2024-09-17T22:30:42.720 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T22:30:42.736 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T22:30:42.826 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909540 branch: squid description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache} email: ceph-qa@ceph.com first_in_suite: false job_id: '7909540' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: rbd cache: false global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b0d356b98e08f6bc59ef15652f0d0e4298fc83c7 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 1983 sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f sleep_before_teardown: 0 subset: 79/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b0d356b98e08f6bc59ef15652f0d0e4298fc83c7 targets: smithi115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPBVDNqFvFCuAcbrQjh38peCguFgt9WodlXcU5H72dyFVtmPEEbHl6aD2cwioUvD8JvgbOZY0zycgkqb2KnXsbA= smithi129.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFWJOirshgZa9espocCz+4g3SQuJpfzo799io9cGRu5IJSf22RKXUEfd1oUMuoY8HPNPG9G/qHerQVZcQqxReAQ= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T22:30:42.826 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa; will attempt to use it 2024-09-17T22:30:42.827 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa/tasks 2024-09-17T22:30:42.827 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T22:30:42.828 INFO:teuthology.task.internal:Checking packages... 2024-09-17T22:30:42.848 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c2ddf9721ef254645201cdbabd9c8f49c30a586f' 2024-09-17T22:30:42.848 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T22:30:42.849 INFO:teuthology.packaging:ref: None 2024-09-17T22:30:42.849 INFO:teuthology.packaging:tag: None 2024-09-17T22:30:42.849 INFO:teuthology.packaging:branch: squid 2024-09-17T22:30:42.849 INFO:teuthology.packaging:sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T22:30:42.849 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-17T22:30:43.051 INFO:teuthology.task.internal:Found packages for ceph version 19.2.0-0 2024-09-17T22:30:43.053 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T22:30:43.058 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T22:30:43.058 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T22:30:43.094 INFO:teuthology.task.internal:Saving configuration 2024-09-17T22:30:43.105 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T22:30:43.128 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T22:30:43.151 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909540', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 22:25:41.515195', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPBVDNqFvFCuAcbrQjh38peCguFgt9WodlXcU5H72dyFVtmPEEbHl6aD2cwioUvD8JvgbOZY0zycgkqb2KnXsbA='} 2024-09-17T22:30:43.174 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi129.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909540', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 22:25:41.516295', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFWJOirshgZa9espocCz+4g3SQuJpfzo799io9cGRu5IJSf22RKXUEfd1oUMuoY8HPNPG9G/qHerQVZcQqxReAQ='} 2024-09-17T22:30:43.174 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T22:30:43.179 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-17T22:30:43.179 INFO:teuthology.task.internal:roles: ubuntu@smithi129.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-17T22:30:43.179 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T22:30:43.266 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f704dd70310>, signals=[15]) 2024-09-17T22:30:43.266 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T22:30:43.271 INFO:teuthology.task.internal:Opening connections... 2024-09-17T22:30:43.271 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2024-09-17T22:30:43.273 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:30:43.351 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T22:30:43.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:30:43.430 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T22:30:43.439 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2024-09-17T22:30:43.457 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2024-09-17T22:30:43.458 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2024-09-17T22:30:43.514 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="CentOS Stream" 2024-09-17T22:30:43.514 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="9" 2024-09-17T22:30:43.514 INFO:teuthology.orchestra.run.smithi115.stdout:ID="centos" 2024-09-17T22:30:43.514 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="rhel fedora" 2024-09-17T22:30:43.514 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="9" 2024-09-17T22:30:43.515 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el9" 2024-09-17T22:30:43.515 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T22:30:43.515 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2024-09-17T22:30:43.515 INFO:teuthology.orchestra.run.smithi115.stdout:LOGO="fedora-logo-icon" 2024-09-17T22:30:43.515 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T22:30:43.515 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://centos.org/" 2024-09-17T22:30:43.515 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T22:30:43.515 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T22:30:43.515 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T22:30:43.516 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2024-09-17T22:30:43.552 DEBUG:teuthology.orchestra.run.smithi129:> uname -m 2024-09-17T22:30:43.570 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2024-09-17T22:30:43.570 DEBUG:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2024-09-17T22:30:43.630 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="CentOS Stream" 2024-09-17T22:30:43.630 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="9" 2024-09-17T22:30:43.630 INFO:teuthology.orchestra.run.smithi129.stdout:ID="centos" 2024-09-17T22:30:43.630 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="rhel fedora" 2024-09-17T22:30:43.630 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="9" 2024-09-17T22:30:43.630 INFO:teuthology.orchestra.run.smithi129.stdout:PLATFORM_ID="platform:el9" 2024-09-17T22:30:43.630 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T22:30:43.630 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2024-09-17T22:30:43.630 INFO:teuthology.orchestra.run.smithi129.stdout:LOGO="fedora-logo-icon" 2024-09-17T22:30:43.630 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T22:30:43.630 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://centos.org/" 2024-09-17T22:30:43.630 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T22:30:43.631 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T22:30:43.631 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T22:30:43.631 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2024-09-17T22:30:43.652 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T22:30:43.661 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T22:30:43.693 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T22:30:43.694 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2024-09-17T22:30:43.696 DEBUG:teuthology.orchestra.run.smithi129:> test '!' -e /home/ubuntu/cephtest 2024-09-17T22:30:43.713 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T22:30:43.727 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T22:30:43.727 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2024-09-17T22:30:43.754 DEBUG:teuthology.orchestra.run.smithi129:> test -z $(ls -A /var/lib/ceph) 2024-09-17T22:30:43.781 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T22:30:43.833 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T22:30:43.849 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-17T22:30:43.849 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-17T22:30:43.849 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T22:30:43.850 DEBUG:teuthology.orchestra.run.smithi129:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T22:30:43.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T22:30:43.866 DEBUG:teuthology.orchestra.run.smithi129:> uname -r 2024-09-17T22:30:43.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T22:30:43.867 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-09-17T22:30:43.924 INFO:teuthology.orchestra.run.smithi129.stdout:5.14.0-503.el9.x86_64 2024-09-17T22:30:43.924 INFO:teuthology.task.kernel:Running kernel on smithi129: 5.14.0-503.el9.x86_64 2024-09-17T22:30:43.924 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum install -y kernel 2024-09-17T22:30:43.925 INFO:teuthology.orchestra.run.smithi115.stdout:5.14.0-503.el9.x86_64 2024-09-17T22:30:43.926 INFO:teuthology.task.kernel:Running kernel on smithi115: 5.14.0-503.el9.x86_64 2024-09-17T22:30:43.926 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum install -y kernel 2024-09-17T22:30:47.639 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.3 MB 00:01 2024-09-17T22:30:48.440 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.3 MB 00:02 2024-09-17T22:30:51.741 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 9.1 MB/s | 20 MB 00:02 2024-09-17T22:30:52.573 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 5.3 MB/s | 20 MB 00:03 2024-09-17T22:30:56.207 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-17T22:30:57.431 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 6.0 MB/s | 6.5 MB 00:01 2024-09-17T22:30:57.866 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-17T22:30:58.899 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 78 kB/s | 19 kB 00:00 2024-09-17T22:30:59.579 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 14 MB/s | 23 MB 00:01 2024-09-17T22:31:00.678 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 14 MB/s | 23 MB 00:01 2024-09-17T22:31:05.067 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-09-17T22:31:06.167 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-17T22:31:06.982 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T22:31:06.982 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T22:31:07.071 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:31:07.076 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:31:07.076 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-17T22:31:07.076 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:31:07.076 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-17T22:31:07.076 INFO:teuthology.orchestra.run.smithi115.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T22:31:07.076 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-17T22:31:07.076 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T22:31:07.076 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T22:31:07.076 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T22:31:07.076 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:31:07.077 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-17T22:31:07.077 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:31:07.077 INFO:teuthology.orchestra.run.smithi115.stdout:Install 4 Packages 2024-09-17T22:31:07.077 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:31:07.077 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 79 M 2024-09-17T22:31:07.078 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 126 M 2024-09-17T22:31:07.078 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-17T22:31:07.211 INFO:teuthology.orchestra.run.smithi115.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 420 kB/s | 36 kB 00:00 2024-09-17T22:31:08.049 INFO:teuthology.orchestra.run.smithi129.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T22:31:08.049 INFO:teuthology.orchestra.run.smithi129.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T22:31:08.142 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:31:08.151 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:31:08.151 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-09-17T22:31:08.151 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:31:08.151 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T22:31:08.152 INFO:teuthology.orchestra.run.smithi129.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T22:31:08.152 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T22:31:08.152 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T22:31:08.152 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T22:31:08.152 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T22:31:08.152 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:31:08.152 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T22:31:08.152 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:31:08.152 INFO:teuthology.orchestra.run.smithi129.stdout:Install 4 Packages 2024-09-17T22:31:08.152 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:31:08.153 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 79 M 2024-09-17T22:31:08.153 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 126 M 2024-09-17T22:31:08.153 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T22:31:08.560 INFO:teuthology.orchestra.run.smithi129.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 125 kB/s | 36 kB 00:00 2024-09-17T22:31:09.694 INFO:teuthology.orchestra.run.smithi115.stdout:(2/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 13 MB/s | 35 MB 00:02 2024-09-17T22:31:09.911 INFO:teuthology.orchestra.run.smithi115.stdout:(3/4): kernel-core-5.14.0-508.el9.x86_64.rpm 5.6 MB/s | 16 MB 00:02 2024-09-17T22:31:10.619 INFO:teuthology.orchestra.run.smithi115.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 8.4 MB/s | 29 MB 00:03 2024-09-17T22:31:10.619 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:31:10.619 INFO:teuthology.orchestra.run.smithi115.stdout:Total 22 MB/s | 79 MB 00:03 2024-09-17T22:31:11.299 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-17T22:31:11.628 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-17T22:31:11.628 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-17T22:31:12.625 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-17T22:31:12.625 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-17T22:31:14.468 INFO:teuthology.orchestra.run.smithi129.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 2.5 MB/s | 16 MB 00:06 2024-09-17T22:31:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-17T22:31:15.757 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T22:31:15.836 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T22:31:16.460 INFO:teuthology.orchestra.run.smithi129.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 3.6 MB/s | 29 MB 00:07 2024-09-17T22:31:16.935 INFO:teuthology.orchestra.run.smithi129.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 4.0 MB/s | 35 MB 00:08 2024-09-17T22:31:16.936 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:31:16.936 INFO:teuthology.orchestra.run.smithi129.stdout:Total 9.0 MB/s | 79 MB 00:08 2024-09-17T22:31:17.614 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T22:31:17.969 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T22:31:17.969 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T22:31:18.599 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T22:31:18.740 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T22:31:18.945 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T22:31:18.945 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T22:31:21.399 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T22:31:22.038 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T22:31:22.119 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T22:31:23.593 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T22:31:23.640 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:31:24.879 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T22:31:25.003 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T22:31:28.489 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:31:29.859 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T22:31:29.888 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:31:34.722 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:32:06.611 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:32:06.654 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:32:06.996 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:32:06.997 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T22:32:06.997 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T22:32:06.997 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T22:32:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:32:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:32:07.822 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-17T22:32:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T22:32:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T22:32:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T22:32:07.823 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T22:32:07.823 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:32:07.823 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:32:08.182 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T22:32:08.261 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T22:32:08.262 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T22:32:08.262 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T22:32:08.262 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T22:32:08.262 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T22:32:08.262 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-17T22:32:08.262 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi115.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T22:32:08.262 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T22:32:08.321 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T22:32:08.321 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T22:32:08.321 DEBUG:teuthology.orchestra.run.smithi115:> sudo rpm -qi grub2-tools 2024-09-17T22:32:08.362 INFO:teuthology.orchestra.run.smithi115.stdout:Name : grub2-tools 2024-09-17T22:32:08.362 INFO:teuthology.orchestra.run.smithi115.stdout:Epoch : 1 2024-09-17T22:32:08.363 INFO:teuthology.orchestra.run.smithi115.stdout:Version : 2.06 2024-09-17T22:32:08.363 INFO:teuthology.orchestra.run.smithi115.stdout:Release : 82.el9 2024-09-17T22:32:08.363 INFO:teuthology.orchestra.run.smithi115.stdout:Architecture: x86_64 2024-09-17T22:32:08.363 INFO:teuthology.orchestra.run.smithi115.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T22:32:08.363 INFO:teuthology.orchestra.run.smithi115.stdout:Group : Unspecified 2024-09-17T22:32:08.363 INFO:teuthology.orchestra.run.smithi115.stdout:Size : 8274670 2024-09-17T22:32:08.363 INFO:teuthology.orchestra.run.smithi115.stdout:License : GPLv3+ 2024-09-17T22:32:08.363 INFO:teuthology.orchestra.run.smithi115.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T22:32:08.363 INFO:teuthology.orchestra.run.smithi115.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T22:32:08.363 INFO:teuthology.orchestra.run.smithi115.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T22:32:08.363 INFO:teuthology.orchestra.run.smithi115.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T22:32:08.363 INFO:teuthology.orchestra.run.smithi115.stdout:Packager : builder@centos.org 2024-09-17T22:32:08.363 INFO:teuthology.orchestra.run.smithi115.stdout:Vendor : CentOS 2024-09-17T22:32:08.363 INFO:teuthology.orchestra.run.smithi115.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T22:32:08.364 INFO:teuthology.orchestra.run.smithi115.stdout:Summary : Support tools for GRUB. 2024-09-17T22:32:08.364 INFO:teuthology.orchestra.run.smithi115.stdout:Description : 2024-09-17T22:32:08.364 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:32:08.364 INFO:teuthology.orchestra.run.smithi115.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T22:32:08.364 INFO:teuthology.orchestra.run.smithi115.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T22:32:08.364 INFO:teuthology.orchestra.run.smithi115.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T22:32:08.364 INFO:teuthology.orchestra.run.smithi115.stdout:hardware devices. 2024-09-17T22:32:08.364 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:32:08.364 INFO:teuthology.orchestra.run.smithi115.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T22:32:08.365 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T22:32:08.365 INFO:teuthology.task.kernel:Updating grub on smithi115 to boot 5.14.0-508.el9.x86_64 2024-09-17T22:32:08.365 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T22:32:09.067 INFO:teuthology.orchestra.run.smithi115.stderr:Generating grub configuration file ... 2024-09-17T22:32:10.420 INFO:teuthology.orchestra.run.smithi115.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T22:32:10.459 INFO:teuthology.orchestra.run.smithi115.stderr:done 2024-09-17T22:32:10.462 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-09-17T22:32:10.477 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.1aJy5yFDK6 2024-09-17T22:32:10.478 DEBUG:teuthology.orchestra.run.smithi115:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.1aJy5yFDK6 2024-09-17T22:32:10.544 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.1aJy5yFDK6 2024-09-17T22:32:10.682 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.1aJy5yFDK6 is 6KB 2024-09-17T22:32:10.731 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.1aJy5yFDK6 2024-09-17T22:32:10.746 DEBUG:teuthology.orchestra.run.smithi115:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T22:32:10.817 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T22:32:10.817 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T22:32:10.817 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T22:32:10.817 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T22:32:10.818 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T22:32:10.985 DEBUG:teuthology.orchestra.run.smithi115:> sudo shutdown -r now 2024-09-17T22:32:12.948 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:32:12.988 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:32:13.301 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:32:13.302 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T22:32:13.302 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T22:32:13.302 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T22:32:14.185 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T22:32:14.185 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:32:14.185 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T22:32:14.185 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T22:32:14.185 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T22:32:14.185 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T22:32:14.185 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T22:32:14.185 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:32:14.185 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:32:14.518 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T22:32:14.645 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T22:32:14.645 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T22:32:14.645 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T22:32:14.645 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T22:32:14.645 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T22:32:14.645 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-17T22:32:14.645 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi129.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T22:32:14.645 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T22:32:14.704 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T22:32:14.704 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T22:32:14.705 DEBUG:teuthology.orchestra.run.smithi129:> sudo rpm -qi grub2-tools 2024-09-17T22:32:14.736 INFO:teuthology.orchestra.run.smithi129.stdout:Name : grub2-tools 2024-09-17T22:32:14.736 INFO:teuthology.orchestra.run.smithi129.stdout:Epoch : 1 2024-09-17T22:32:14.736 INFO:teuthology.orchestra.run.smithi129.stdout:Version : 2.06 2024-09-17T22:32:14.737 INFO:teuthology.orchestra.run.smithi129.stdout:Release : 82.el9 2024-09-17T22:32:14.737 INFO:teuthology.orchestra.run.smithi129.stdout:Architecture: x86_64 2024-09-17T22:32:14.737 INFO:teuthology.orchestra.run.smithi129.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T22:32:14.737 INFO:teuthology.orchestra.run.smithi129.stdout:Group : Unspecified 2024-09-17T22:32:14.737 INFO:teuthology.orchestra.run.smithi129.stdout:Size : 8274670 2024-09-17T22:32:14.737 INFO:teuthology.orchestra.run.smithi129.stdout:License : GPLv3+ 2024-09-17T22:32:14.737 INFO:teuthology.orchestra.run.smithi129.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T22:32:14.737 INFO:teuthology.orchestra.run.smithi129.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T22:32:14.737 INFO:teuthology.orchestra.run.smithi129.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T22:32:14.737 INFO:teuthology.orchestra.run.smithi129.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T22:32:14.737 INFO:teuthology.orchestra.run.smithi129.stdout:Packager : builder@centos.org 2024-09-17T22:32:14.737 INFO:teuthology.orchestra.run.smithi129.stdout:Vendor : CentOS 2024-09-17T22:32:14.737 INFO:teuthology.orchestra.run.smithi129.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T22:32:14.737 INFO:teuthology.orchestra.run.smithi129.stdout:Summary : Support tools for GRUB. 2024-09-17T22:32:14.738 INFO:teuthology.orchestra.run.smithi129.stdout:Description : 2024-09-17T22:32:14.738 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:32:14.738 INFO:teuthology.orchestra.run.smithi129.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T22:32:14.738 INFO:teuthology.orchestra.run.smithi129.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T22:32:14.738 INFO:teuthology.orchestra.run.smithi129.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T22:32:14.738 INFO:teuthology.orchestra.run.smithi129.stdout:hardware devices. 2024-09-17T22:32:14.738 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:32:14.738 INFO:teuthology.orchestra.run.smithi129.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T22:32:14.739 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T22:32:14.739 INFO:teuthology.task.kernel:Updating grub on smithi129 to boot 5.14.0-508.el9.x86_64 2024-09-17T22:32:14.739 DEBUG:teuthology.orchestra.run.smithi129:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T22:32:15.430 INFO:teuthology.orchestra.run.smithi129.stderr:Generating grub configuration file ... 2024-09-17T22:32:16.755 INFO:teuthology.orchestra.run.smithi129.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T22:32:16.788 INFO:teuthology.orchestra.run.smithi129.stderr:done 2024-09-17T22:32:16.792 DEBUG:teuthology.orchestra.run.smithi129:> mktemp 2024-09-17T22:32:16.807 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.bUjRfnv0hn 2024-09-17T22:32:16.807 DEBUG:teuthology.orchestra.run.smithi129:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.bUjRfnv0hn 2024-09-17T22:32:16.876 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.bUjRfnv0hn 2024-09-17T22:32:17.010 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.bUjRfnv0hn is 6KB 2024-09-17T22:32:17.059 DEBUG:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.bUjRfnv0hn 2024-09-17T22:32:17.075 DEBUG:teuthology.orchestra.run.smithi129:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T22:32:17.138 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T22:32:17.139 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T22:32:17.139 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T22:32:17.139 INFO:teuthology.orchestra.run.smithi129.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T22:32:17.140 DEBUG:teuthology.orchestra.run.smithi129:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T22:32:17.297 DEBUG:teuthology.orchestra.run.smithi129:> sudo shutdown -r now 2024-09-17T22:32:41.009 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-17T22:32:41.010 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-17T22:32:41.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:32:47.305 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-17T22:32:47.305 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T22:32:47.306 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:32:59.457 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-09-17T22:33:05.857 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-09-17T22:33:08.461 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-17T22:33:08.462 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:33:11.517 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-09-17T22:33:14.861 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T22:33:14.861 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:33:17.917 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-09-17T22:33:23.522 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-17T22:33:23.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:33:29.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T22:33:29.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:33:41.949 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-09-17T22:33:48.353 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.129 2024-09-17T22:33:56.951 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-17T22:33:56.952 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:33:57.329 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-09-17T22:33:57.677 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-17T22:33:57.677 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-17T22:33:57.677 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-09-17T22:33:57.692 INFO:teuthology.orchestra.run.smithi115.stdout:5.14.0-508.el9.x86_64 2024-09-17T22:33:57.692 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T22:33:57.692 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T22:33:57.692 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T22:33:58.693 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-17T22:33:58.693 DEBUG:teuthology.orchestra.run.smithi115:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T22:33:58.787 INFO:teuthology.orchestra.run.smithi115.stdout:ttyS1 2024-09-17T22:33:58.818 DEBUG:teuthology.parallel:result is None 2024-09-17T22:34:03.354 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T22:34:03.355 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:34:03.780 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-09-17T22:34:04.190 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T22:34:04.190 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-17T22:34:04.190 DEBUG:teuthology.orchestra.run.smithi129:> uname -r 2024-09-17T22:34:04.205 INFO:teuthology.orchestra.run.smithi129.stdout:5.14.0-508.el9.x86_64 2024-09-17T22:34:04.205 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T22:34:04.205 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T22:34:04.205 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T22:34:05.206 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-17T22:34:05.206 DEBUG:teuthology.orchestra.run.smithi129:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T22:34:05.272 INFO:teuthology.orchestra.run.smithi129.stdout:ttyS1 2024-09-17T22:34:05.302 DEBUG:teuthology.parallel:result is None 2024-09-17T22:34:05.302 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T22:34:05.313 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T22:34:05.314 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T22:34:05.317 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T22:34:05.333 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T22:34:05.378 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T22:34:05.412 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T22:34:05.412 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T22:34:05.415 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T22:34:05.451 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T22:34:05.457 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T22:34:05.457 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T22:34:05.489 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T22:34:05.522 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T22:34:05.523 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T22:34:05.532 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T22:34:05.535 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T22:34:05.537 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T22:34:05.544 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T22:34:05.544 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T22:34:05.580 DEBUG:teuthology.orchestra.run.smithi129:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T22:34:05.605 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T22:34:05.615 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T22:34:05.615 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T22:34:05.648 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T22:34:05.665 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T22:34:05.743 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T22:34:05.796 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T22:34:05.876 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T22:34:05.920 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:34:05.920 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T22:34:05.986 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T22:34:06.026 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T22:34:06.077 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T22:34:06.161 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T22:34:06.207 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T22:34:06.207 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T22:34:06.277 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2024-09-17T22:34:06.279 DEBUG:teuthology.orchestra.run.smithi129:> sudo service rsyslog restart 2024-09-17T22:34:06.326 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T22:34:06.395 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T22:34:06.842 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T22:34:06.851 INFO:teuthology.task.internal:Starting timer... 2024-09-17T22:34:06.851 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T22:34:06.897 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T22:34:06.933 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-17T22:34:06.933 DEBUG:teuthology.orchestra.run.smithi115:> sudo service auditd rotate 2024-09-17T22:34:06.987 INFO:teuthology.orchestra.run.smithi115.stdout:Rotating logs: 2024-09-17T22:34:06.989 DEBUG:teuthology.orchestra.run.smithi129:> sudo service auditd rotate 2024-09-17T22:34:07.119 INFO:teuthology.orchestra.run.smithi129.stdout:Rotating logs: 2024-09-17T22:34:07.121 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T22:34:07.122 DEBUG:teuthology.orchestra.run.smithi115:> /usr/sbin/getenforce 2024-09-17T22:34:07.143 INFO:teuthology.orchestra.run.smithi115.stdout:Permissive 2024-09-17T22:34:07.143 DEBUG:teuthology.orchestra.run.smithi129:> /usr/sbin/getenforce 2024-09-17T22:34:07.171 INFO:teuthology.orchestra.run.smithi129.stdout:Permissive 2024-09-17T22:34:07.171 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi115.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi129.front.sepia.ceph.com': 'permissive'} 2024-09-17T22:34:07.171 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T22:34:07.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T22:34:07.210 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T22:34:07.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T22:34:07.241 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T22:34:07.242 DEBUG:teuthology.orchestra.run.smithi115:> sudo /usr/sbin/setenforce permissive 2024-09-17T22:34:07.285 DEBUG:teuthology.orchestra.run.smithi129:> sudo /usr/sbin/setenforce permissive 2024-09-17T22:34:07.592 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T22:34:07.602 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-17T22:34:07.605 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-17T22:34:07.739 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T22:34:07.749 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T22:34:07.750 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi115.front.sepia.ceph.com,smithi129.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T22:37:52.264 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi115.front.sepia.ceph.com'), Remote(name='ubuntu@smithi129.front.sepia.ceph.com')] 2024-09-17T22:37:52.266 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-17T22:37:52.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:37:52.343 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-09-17T22:37:52.420 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-17T22:37:52.421 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T22:37:52.421 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T22:37:52.498 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-09-17T22:37:52.581 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-17T22:37:52.581 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T22:37:52.592 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T22:37:52.592 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T22:37:52.593 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T22:37:52.595 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T22:37:52.595 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T22:37:52.622 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T22:37:52.636 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T22:37:52.650 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T22:37:52.660 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: ntpd: command not found 2024-09-17T22:37:52.665 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T22:37:52.671 INFO:teuthology.orchestra.run.smithi115.stdout:506 Cannot talk to daemon 2024-09-17T22:37:52.684 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T22:37:52.689 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: ntpd: command not found 2024-09-17T22:37:52.698 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T22:37:52.701 INFO:teuthology.orchestra.run.smithi129.stdout:506 Cannot talk to daemon 2024-09-17T22:37:52.716 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T22:37:52.730 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T22:37:52.754 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-09-17T22:37:52.757 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T22:37:52.757 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-09-17T22:37:52.757 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T22:37:52.757 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T22:37:52.757 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T22:37:52.758 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T22:37:52.785 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-09-17T22:37:52.789 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T22:37:52.789 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-09-17T22:37:52.789 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T22:37:52.789 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T22:37:52.789 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T22:37:52.789 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T22:37:52.789 INFO:teuthology.run_tasks:Running task install... 2024-09-17T22:37:52.797 DEBUG:teuthology.task.install:project ceph 2024-09-17T22:37:52.797 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}} 2024-09-17T22:37:52.797 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'} 2024-09-17T22:37:52.798 INFO:teuthology.task.install:Using flavor: default 2024-09-17T22:37:52.804 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T22:37:52.805 INFO:teuthology.task.install:extra packages: [] 2024-09-17T22:37:52.805 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'tag': None, 'wait_for_package': False} 2024-09-17T22:37:52.806 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T22:37:52.808 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'tag': None, 'wait_for_package': False} 2024-09-17T22:37:52.808 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T22:37:52.973 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/ 2024-09-17T22:37:52.973 INFO:teuthology.task.install.rpm:Package version is 19.1.1-330.gc2ddf972 2024-09-17T22:37:52.976 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/ 2024-09-17T22:37:52.976 INFO:teuthology.task.install.rpm:Package version is 19.1.1-330.gc2ddf972 2024-09-17T22:37:53.087 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T22:37:53.088 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T22:37:53.088 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T22:37:53.090 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T22:37:53.090 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:37:53.090 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T22:37:53.116 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-17T22:37:53.117 DEBUG:teuthology.orchestra.run.smithi129:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c2ddf9721ef254645201cdbabd9c8f49c30a586f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T22:37:53.118 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-17T22:37:53.118 DEBUG:teuthology.orchestra.run.smithi115:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c2ddf9721ef254645201cdbabd9c8f49c30a586f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T22:37:53.183 DEBUG:teuthology.orchestra.run.smithi129:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T22:37:53.185 DEBUG:teuthology.orchestra.run.smithi115:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T22:37:53.258 DEBUG:teuthology.orchestra.run.smithi129:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T22:37:53.260 DEBUG:teuthology.orchestra.run.smithi115:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T22:37:53.322 INFO:teuthology.orchestra.run.smithi129.stdout:check_obsoletes = 1 2024-09-17T22:37:53.324 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum clean all 2024-09-17T22:37:53.325 INFO:teuthology.orchestra.run.smithi115.stdout:check_obsoletes = 1 2024-09-17T22:37:53.326 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum clean all 2024-09-17T22:37:53.548 INFO:teuthology.orchestra.run.smithi129.stdout:45 files removed 2024-09-17T22:37:53.575 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-radosgw 2024-09-17T22:37:53.587 INFO:teuthology.orchestra.run.smithi115.stdout:45 files removed 2024-09-17T22:37:53.617 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-radosgw 2024-09-17T22:37:54.127 INFO:teuthology.orchestra.run.smithi129.stdout:ceph packages for x86_64 311 kB/s | 84 kB 00:00 2024-09-17T22:37:54.175 INFO:teuthology.orchestra.run.smithi115.stdout:ceph packages for x86_64 305 kB/s | 84 kB 00:00 2024-09-17T22:37:54.411 INFO:teuthology.orchestra.run.smithi129.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-17T22:37:54.459 INFO:teuthology.orchestra.run.smithi115.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-17T22:37:54.686 INFO:teuthology.orchestra.run.smithi129.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-17T22:37:54.743 INFO:teuthology.orchestra.run.smithi115.stdout:ceph source packages 7.6 kB/s | 1.9 kB 00:00 2024-09-17T22:37:56.553 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.3 MB 00:01 2024-09-17T22:37:57.042 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 3.6 MB/s | 8.3 MB 00:02 2024-09-17T22:37:58.835 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-17T22:38:01.679 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 5.0 MB/s | 20 MB 00:04 2024-09-17T22:38:03.144 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-17T22:38:05.003 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-17T22:38:06.871 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.5 MB 00:01 2024-09-17T22:38:07.099 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 11 MB/s | 23 MB 00:01 2024-09-17T22:38:08.397 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 19 kB 00:00 2024-09-17T22:38:09.993 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 15 MB/s | 23 MB 00:01 2024-09-17T22:38:12.438 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-17T22:38:14.665 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:38:14.668 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:38:14.668 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-17T22:38:14.668 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:38:14.668 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-17T22:38:14.668 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 11 M 2024-09-17T22:38:14.668 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2024-09-17T22:38:14.668 INFO:teuthology.orchestra.run.smithi115.stdout: librados2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.4 M 2024-09-17T22:38:14.668 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.2 M 2024-09-17T22:38:14.668 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-17T22:38:14.668 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T22:38:14.668 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.4 M 2024-09-17T22:38:14.669 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 21 M 2024-09-17T22:38:14.669 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 25 k 2024-09-17T22:38:14.669 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T22:38:14.669 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T22:38:14.669 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T22:38:14.669 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 751 k 2024-09-17T22:38:14.669 INFO:teuthology.orchestra.run.smithi115.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T22:38:14.669 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T22:38:14.669 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T22:38:14.669 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 506 k 2024-09-17T22:38:14.669 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T22:38:14.669 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.3 M 2024-09-17T22:38:14.670 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T22:38:14.670 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T22:38:14.670 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T22:38:14.670 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 45 k 2024-09-17T22:38:14.670 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 136 k 2024-09-17T22:38:14.670 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 162 k 2024-09-17T22:38:14.670 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T22:38:14.670 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 320 k 2024-09-17T22:38:14.670 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 300 k 2024-09-17T22:38:14.670 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 99 k 2024-09-17T22:38:14.670 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T22:38:14.670 INFO:teuthology.orchestra.run.smithi115.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T22:38:14.670 INFO:teuthology.orchestra.run.smithi115.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T22:38:14.671 INFO:teuthology.orchestra.run.smithi115.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T22:38:14.671 INFO:teuthology.orchestra.run.smithi115.stdout:Installing weak dependencies: 2024-09-17T22:38:14.671 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T22:38:14.671 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:38:14.671 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-17T22:38:14.671 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:38:14.671 INFO:teuthology.orchestra.run.smithi115.stdout:Install 30 Packages 2024-09-17T22:38:14.671 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 2 Packages 2024-09-17T22:38:14.671 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:38:14.672 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 60 M 2024-09-17T22:38:14.672 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-17T22:38:15.357 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-09-17T22:38:15.438 INFO:teuthology.orchestra.run.smithi115.stdout:(1/32): ceph-base-19.1.1-330.gc2ddf972.el9.x86_ 9.8 MB/s | 5.4 MB 00:00 2024-09-17T22:38:15.497 INFO:teuthology.orchestra.run.smithi115.stdout:(2/32): ceph-selinux-19.1.1-330.gc2ddf972.el9.x 433 kB/s | 25 kB 00:00 2024-09-17T22:38:15.589 INFO:teuthology.orchestra.run.smithi115.stdout:(3/32): libcephfs2-19.1.1-330.gc2ddf972.el9.x86 8.0 MB/s | 751 kB 00:00 2024-09-17T22:38:15.672 INFO:teuthology.orchestra.run.smithi115.stdout:(4/32): libradosstriper1-19.1.1-330.gc2ddf972.e 5.9 MB/s | 506 kB 00:00 2024-09-17T22:38:15.814 INFO:teuthology.orchestra.run.smithi115.stdout:(5/32): ceph-radosgw-19.1.1-330.gc2ddf972.el9.x 11 MB/s | 11 MB 00:00 2024-09-17T22:38:16.023 INFO:teuthology.orchestra.run.smithi115.stdout:(6/32): ceph-common-19.1.1-330.gc2ddf972.el9.x8 19 MB/s | 21 MB 00:01 2024-09-17T22:38:16.056 INFO:teuthology.orchestra.run.smithi115.stdout:(7/32): python3-ceph-argparse-19.1.1-330.gc2ddf 187 kB/s | 45 kB 00:00 2024-09-17T22:38:16.084 INFO:teuthology.orchestra.run.smithi115.stdout:(8/32): python3-ceph-common-19.1.1-330.gc2ddf97 2.2 MB/s | 136 kB 00:00 2024-09-17T22:38:16.126 INFO:teuthology.orchestra.run.smithi115.stdout:(9/32): python3-cephfs-19.1.1-330.gc2ddf972.el9 2.3 MB/s | 162 kB 00:00 2024-09-17T22:38:16.160 INFO:teuthology.orchestra.run.smithi115.stdout:(10/32): python3-rados-19.1.1-330.gc2ddf972.el9 4.2 MB/s | 320 kB 00:00 2024-09-17T22:38:16.185 INFO:teuthology.orchestra.run.smithi115.stdout:(11/32): python3-rbd-19.1.1-330.gc2ddf972.el9.x 5.0 MB/s | 300 kB 00:00 2024-09-17T22:38:16.210 INFO:teuthology.orchestra.run.smithi115.stdout:(12/32): python3-rgw-19.1.1-330.gc2ddf972.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-17T22:38:16.285 INFO:teuthology.orchestra.run.smithi115.stdout:(13/32): librgw2-19.1.1-330.gc2ddf972.el9.x86_6 8.7 MB/s | 5.3 MB 00:00 2024-09-17T22:38:16.361 INFO:teuthology.orchestra.run.smithi115.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 609 kB/s | 107 kB 00:00 2024-09-17T22:38:16.386 INFO:teuthology.orchestra.run.smithi115.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 452 kB/s | 45 kB 00:00 2024-09-17T22:38:16.411 INFO:teuthology.orchestra.run.smithi115.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 798 kB/s | 160 kB 00:00 2024-09-17T22:38:16.453 INFO:teuthology.orchestra.run.smithi115.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-17T22:38:16.478 INFO:teuthology.orchestra.run.smithi115.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-17T22:38:16.504 INFO:teuthology.orchestra.run.smithi115.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-17T22:38:16.529 INFO:teuthology.orchestra.run.smithi115.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 714 kB/s | 36 kB 00:00 2024-09-17T22:38:16.554 INFO:teuthology.orchestra.run.smithi115.stdout:(21/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-09-17T22:38:16.579 INFO:teuthology.orchestra.run.smithi115.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-09-17T22:38:16.646 INFO:teuthology.orchestra.run.smithi115.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 391 kB/s | 26 kB 00:00 2024-09-17T22:38:16.680 INFO:teuthology.orchestra.run.smithi115.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-17T22:38:16.713 INFO:teuthology.orchestra.run.smithi115.stdout:(25/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.6 MB/s | 308 kB 00:00 2024-09-17T22:38:16.730 INFO:teuthology.orchestra.run.smithi115.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-17T22:38:16.789 INFO:teuthology.orchestra.run.smithi115.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-17T22:38:16.906 INFO:teuthology.orchestra.run.smithi115.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.3 MB/s | 839 kB 00:00 2024-09-17T22:38:17.081 INFO:teuthology.orchestra.run.smithi115.stdout:(29/32): librados2-19.1.1-330.gc2ddf972.el9.x86 20 MB/s | 3.4 MB 00:00 2024-09-17T22:38:17.115 INFO:teuthology.orchestra.run.smithi115.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-09-17T22:38:17.223 INFO:teuthology.orchestra.run.smithi115.stdout:(31/32): librbd1-19.1.1-330.gc2ddf972.el9.x86_6 22 MB/s | 3.2 MB 00:00 2024-09-17T22:38:17.307 INFO:teuthology.orchestra.run.smithi115.stdout:(32/32): libarrow-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 4.4 MB 00:00 2024-09-17T22:38:17.312 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:38:17.312 INFO:teuthology.orchestra.run.smithi115.stdout:Total 23 MB/s | 60 MB 00:02 2024-09-17T22:38:17.419 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-17T22:38:17.455 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-17T22:38:17.455 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-17T22:38:17.566 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:38:17.570 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:38:17.570 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T22:38:17.570 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:38:17.570 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T22:38:17.570 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 11 M 2024-09-17T22:38:17.570 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrading: 2024-09-17T22:38:17.570 INFO:teuthology.orchestra.run.smithi129.stdout: librados2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.4 M 2024-09-17T22:38:17.570 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.2 M 2024-09-17T22:38:17.570 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T22:38:17.570 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T22:38:17.570 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.4 M 2024-09-17T22:38:17.570 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 21 M 2024-09-17T22:38:17.571 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 25 k 2024-09-17T22:38:17.571 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T22:38:17.571 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T22:38:17.571 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T22:38:17.571 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 751 k 2024-09-17T22:38:17.571 INFO:teuthology.orchestra.run.smithi129.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T22:38:17.571 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T22:38:17.571 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T22:38:17.571 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 506 k 2024-09-17T22:38:17.571 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T22:38:17.571 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.3 M 2024-09-17T22:38:17.571 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T22:38:17.572 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T22:38:17.572 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T22:38:17.572 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 45 k 2024-09-17T22:38:17.572 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 136 k 2024-09-17T22:38:17.572 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 162 k 2024-09-17T22:38:17.572 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T22:38:17.572 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 320 k 2024-09-17T22:38:17.572 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 300 k 2024-09-17T22:38:17.572 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 99 k 2024-09-17T22:38:17.572 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T22:38:17.572 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T22:38:17.572 INFO:teuthology.orchestra.run.smithi129.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T22:38:17.572 INFO:teuthology.orchestra.run.smithi129.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T22:38:17.573 INFO:teuthology.orchestra.run.smithi129.stdout:Installing weak dependencies: 2024-09-17T22:38:17.573 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T22:38:17.573 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:38:17.573 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T22:38:17.573 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:38:17.573 INFO:teuthology.orchestra.run.smithi129.stdout:Install 30 Packages 2024-09-17T22:38:17.573 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrade 2 Packages 2024-09-17T22:38:17.573 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:38:17.573 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 60 M 2024-09-17T22:38:17.574 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T22:38:18.130 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-17T22:38:18.131 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-17T22:38:18.233 INFO:teuthology.orchestra.run.smithi129.stdout:(1/32): ceph-base-19.1.1-330.gc2ddf972.el9.x86_ 9.7 MB/s | 5.4 MB 00:00 2024-09-17T22:38:18.426 INFO:teuthology.orchestra.run.smithi129.stdout:(2/32): ceph-selinux-19.1.1-330.gc2ddf972.el9.x 377 kB/s | 25 kB 00:00 2024-09-17T22:38:18.426 INFO:teuthology.orchestra.run.smithi129.stdout:(3/32): ceph-radosgw-19.1.1-330.gc2ddf972.el9.x 14 MB/s | 11 MB 00:00 2024-09-17T22:38:18.470 INFO:teuthology.orchestra.run.smithi129.stdout:(4/32): libcephfs2-19.1.1-330.gc2ddf972.el9.x86 4.3 MB/s | 751 kB 00:00 2024-09-17T22:38:18.503 INFO:teuthology.orchestra.run.smithi129.stdout:(5/32): libradosstriper1-19.1.1-330.gc2ddf972.e 6.4 MB/s | 506 kB 00:00 2024-09-17T22:38:18.562 INFO:teuthology.orchestra.run.smithi129.stdout:(6/32): python3-ceph-argparse-19.1.1-330.gc2ddf 775 kB/s | 45 kB 00:00 2024-09-17T22:38:18.654 INFO:teuthology.orchestra.run.smithi129.stdout:(7/32): python3-ceph-common-19.1.1-330.gc2ddf97 1.5 MB/s | 136 kB 00:00 2024-09-17T22:38:18.737 INFO:teuthology.orchestra.run.smithi129.stdout:(8/32): librgw2-19.1.1-330.gc2ddf972.el9.x86_64 20 MB/s | 5.3 MB 00:00 2024-09-17T22:38:18.763 INFO:teuthology.orchestra.run.smithi129.stdout:(9/32): python3-cephfs-19.1.1-330.gc2ddf972.el9 1.5 MB/s | 162 kB 00:00 2024-09-17T22:38:18.788 INFO:teuthology.orchestra.run.smithi129.stdout:(10/32): python3-rados-19.1.1-330.gc2ddf972.el9 6.2 MB/s | 320 kB 00:00 2024-09-17T22:38:18.813 INFO:teuthology.orchestra.run.smithi129.stdout:(11/32): python3-rbd-19.1.1-330.gc2ddf972.el9.x 5.8 MB/s | 300 kB 00:00 2024-09-17T22:38:18.839 INFO:teuthology.orchestra.run.smithi129.stdout:(12/32): python3-rgw-19.1.1-330.gc2ddf972.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-17T22:38:19.047 INFO:teuthology.orchestra.run.smithi129.stdout:(13/32): ceph-common-19.1.1-330.gc2ddf972.el9.x 15 MB/s | 21 MB 00:01 2024-09-17T22:38:19.156 INFO:teuthology.orchestra.run.smithi129.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 419 kB/s | 45 kB 00:00 2024-09-17T22:38:19.181 INFO:teuthology.orchestra.run.smithi129.stdout:(15/32): boost-program-options-1.75.0-8.el9.x86 291 kB/s | 107 kB 00:00 2024-09-17T22:38:19.206 INFO:teuthology.orchestra.run.smithi129.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 436 kB/s | 160 kB 00:00 2024-09-17T22:38:19.257 INFO:teuthology.orchestra.run.smithi129.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 844 kB/s | 42 kB 00:00 2024-09-17T22:38:19.291 INFO:teuthology.orchestra.run.smithi129.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-17T22:38:19.341 INFO:teuthology.orchestra.run.smithi129.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-17T22:38:19.358 INFO:teuthology.orchestra.run.smithi129.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 537 kB/s | 36 kB 00:00 2024-09-17T22:38:19.391 INFO:teuthology.orchestra.run.smithi129.stdout:(21/32): librdkafka-1.6.1-102.el9.x86_64.rpm 2.8 MB/s | 662 kB 00:00 2024-09-17T22:38:19.408 INFO:teuthology.orchestra.run.smithi129.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 704 kB/s | 47 kB 00:00 2024-09-17T22:38:19.467 INFO:teuthology.orchestra.run.smithi129.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 595 kB/s | 26 kB 00:00 2024-09-17T22:38:19.492 INFO:teuthology.orchestra.run.smithi129.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-17T22:38:19.517 INFO:teuthology.orchestra.run.smithi129.stdout:(25/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.9 MB/s | 308 kB 00:00 2024-09-17T22:38:19.542 INFO:teuthology.orchestra.run.smithi129.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-17T22:38:19.595 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-17T22:38:19.610 INFO:teuthology.orchestra.run.smithi129.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-17T22:38:19.683 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-17T22:38:19.712 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-17T22:38:19.737 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-17T22:38:19.743 INFO:teuthology.orchestra.run.smithi129.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 839 kB 00:00 2024-09-17T22:38:19.760 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T22:38:19.784 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T22:38:19.919 INFO:teuthology.orchestra.run.smithi129.stdout:(29/32): librados2-19.1.1-330.gc2ddf972.el9.x86 20 MB/s | 3.4 MB 00:00 2024-09-17T22:38:19.961 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T22:38:19.985 INFO:teuthology.orchestra.run.smithi129.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 4.2 MB/s | 1.6 MB 00:00 2024-09-17T22:38:20.015 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T22:38:20.061 INFO:teuthology.orchestra.run.smithi129.stdout:(31/32): librbd1-19.1.1-330.gc2ddf972.el9.x86_6 22 MB/s | 3.2 MB 00:00 2024-09-17T22:38:20.153 INFO:teuthology.orchestra.run.smithi129.stdout:(32/32): libarrow-9.0.0-11.el9.x86_64.rpm 5.8 MB/s | 4.4 MB 00:00 2024-09-17T22:38:20.158 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:38:20.158 INFO:teuthology.orchestra.run.smithi129.stdout:Total 23 MB/s | 60 MB 00:02 2024-09-17T22:38:20.263 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T22:38:20.299 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T22:38:20.299 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T22:38:20.383 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-17T22:38:20.455 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T22:38:20.547 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T22:38:20.637 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 8/34 2024-09-17T22:38:20.665 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T22:38:20.727 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T22:38:20.756 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-17T22:38:20.801 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 11/34 2024-09-17T22:38:20.872 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T22:38:20.892 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T22:38:20.948 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T22:38:20.949 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T22:38:20.956 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T22:38:20.990 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-17T22:38:21.101 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-17T22:38:21.142 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-17T22:38:21.189 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-17T22:38:21.245 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-17T22:38:21.277 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-17T22:38:21.310 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-17T22:38:21.629 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-17T22:38:21.673 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T22:38:21.764 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T22:38:21.794 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 23/34 2024-09-17T22:38:21.830 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-17T22:38:22.312 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-17T22:38:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T22:38:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T22:38:22.448 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T22:38:22.483 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 27/34 2024-09-17T22:38:22.511 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 28/34 2024-09-17T22:38:22.528 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-17T22:38:22.558 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-17T22:38:22.591 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-17T22:38:22.614 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T22:38:22.638 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T22:38:22.834 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T22:38:22.886 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T22:38:23.243 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-17T22:38:23.343 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T22:38:23.417 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T22:38:23.514 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 8/34 2024-09-17T22:38:23.545 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T22:38:23.608 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T22:38:23.637 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-17T22:38:23.680 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 11/34 2024-09-17T22:38:23.749 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T22:38:23.772 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T22:38:23.836 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T22:38:23.871 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-17T22:38:23.978 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-17T22:38:24.005 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-17T22:38:24.050 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-17T22:38:24.079 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-17T22:38:24.108 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-17T22:38:24.141 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-17T22:38:24.470 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-17T22:38:24.512 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T22:38:24.604 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T22:38:24.634 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 23/34 2024-09-17T22:38:24.670 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-17T22:38:24.771 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T22:38:24.816 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T22:38:25.158 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-17T22:38:25.190 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T22:38:25.268 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T22:38:25.341 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 27/34 2024-09-17T22:38:25.375 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 28/34 2024-09-17T22:38:25.402 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T22:38:25.435 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T22:38:25.534 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T22:38:25.534 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T22:38:25.534 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T22:38:25.535 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:38:25.562 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T22:38:27.707 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T22:38:27.748 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T22:38:28.306 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T22:38:28.343 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T22:38:28.435 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T22:38:28.435 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T22:38:28.435 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T22:38:28.435 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:38:28.469 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T22:38:35.875 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T22:38:35.875 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /sys 2024-09-17T22:38:35.875 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /proc 2024-09-17T22:38:35.875 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /mnt 2024-09-17T22:38:35.875 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /var/tmp 2024-09-17T22:38:35.875 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /home 2024-09-17T22:38:35.875 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /root 2024-09-17T22:38:35.875 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /tmp 2024-09-17T22:38:35.875 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:38:36.839 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T22:38:36.864 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T22:38:36.864 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:38:36.864 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T22:38:36.864 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T22:38:36.864 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T22:38:36.864 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:38:36.866 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T22:38:36.949 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T22:38:36.950 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T22:38:38.180 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T22:38:38.180 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 1/34 2024-09-17T22:38:38.181 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2/34 2024-09-17T22:38:38.181 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 3/34 2024-09-17T22:38:38.181 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 4/34 2024-09-17T22:38:38.181 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 5/34 2024-09-17T22:38:38.181 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 6/34 2024-09-17T22:38:38.181 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T22:38:38.181 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 8/34 2024-09-17T22:38:38.181 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 9/34 2024-09-17T22:38:38.181 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 10/34 2024-09-17T22:38:38.181 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 11/34 2024-09-17T22:38:38.181 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T22:38:38.181 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T22:38:38.182 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-17T22:38:38.182 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-17T22:38:38.182 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-17T22:38:38.182 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-17T22:38:38.182 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-17T22:38:38.182 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-17T22:38:38.182 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-17T22:38:38.182 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-17T22:38:38.182 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-17T22:38:38.182 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-17T22:38:38.182 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-17T22:38:38.182 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-17T22:38:38.183 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-17T22:38:38.183 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-17T22:38:38.183 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-17T22:38:38.184 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-17T22:38:38.184 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-17T22:38:38.184 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T22:38:38.184 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-17T22:38:38.185 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 33/34 2024-09-17T22:38:38.485 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T22:38:38.485 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /sys 2024-09-17T22:38:38.485 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /proc 2024-09-17T22:38:38.485 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /mnt 2024-09-17T22:38:38.485 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /var/tmp 2024-09-17T22:38:38.485 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /home 2024-09-17T22:38:38.485 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /root 2024-09-17T22:38:38.485 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /tmp 2024-09-17T22:38:38.485 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:38:39.222 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T22:38:39.222 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:38:39.222 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2024-09-17T22:38:39.223 INFO:teuthology.orchestra.run.smithi115.stdout: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:39.223 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:39.223 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-17T22:38:39.223 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T22:38:39.223 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:39.223 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:39.223 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:39.223 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:39.223 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T22:38:39.223 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T22:38:39.223 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T22:38:39.223 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:39.224 INFO:teuthology.orchestra.run.smithi115.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T22:38:39.224 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T22:38:39.224 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T22:38:39.224 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:39.224 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T22:38:39.224 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:39.224 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T22:38:39.224 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T22:38:39.224 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T22:38:39.224 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:39.224 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:39.224 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:39.224 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T22:38:39.225 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:39.225 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:39.225 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:39.225 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T22:38:39.225 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T22:38:39.225 INFO:teuthology.orchestra.run.smithi115.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T22:38:39.225 INFO:teuthology.orchestra.run.smithi115.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T22:38:39.225 INFO:teuthology.orchestra.run.smithi115.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T22:38:39.225 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:38:39.225 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:38:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T22:38:39.519 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T22:38:39.520 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:38:39.520 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T22:38:39.520 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T22:38:39.520 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T22:38:39.520 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:38:39.521 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T22:38:39.553 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-test 2024-09-17T22:38:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T22:38:39.613 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T22:38:40.140 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:28 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:38:40.271 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:38:40.272 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:38:40.272 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-17T22:38:40.272 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:38:40.272 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-17T22:38:40.272 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 49 M 2024-09-17T22:38:40.272 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-17T22:38:40.272 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 163 k 2024-09-17T22:38:40.272 INFO:teuthology.orchestra.run.smithi115.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T22:38:40.272 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T22:38:40.273 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:38:40.273 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-17T22:38:40.273 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:38:40.273 INFO:teuthology.orchestra.run.smithi115.stdout:Install 4 Packages 2024-09-17T22:38:40.273 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:38:40.273 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 49 M 2024-09-17T22:38:40.273 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 209 M 2024-09-17T22:38:40.274 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-17T22:38:40.537 INFO:teuthology.orchestra.run.smithi115.stdout:(1/4): libcephsqlite-19.1.1-330.gc2ddf972.el9.x 755 kB/s | 163 kB 00:00 2024-09-17T22:38:40.562 INFO:teuthology.orchestra.run.smithi115.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.2 MB/s | 304 kB 00:00 2024-09-17T22:38:40.679 INFO:teuthology.orchestra.run.smithi115.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 449 kB/s | 64 kB 00:00 2024-09-17T22:38:40.827 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T22:38:40.827 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 1/34 2024-09-17T22:38:40.827 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2/34 2024-09-17T22:38:40.827 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 3/34 2024-09-17T22:38:40.827 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 4/34 2024-09-17T22:38:40.827 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 5/34 2024-09-17T22:38:40.827 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 6/34 2024-09-17T22:38:40.827 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T22:38:40.827 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 8/34 2024-09-17T22:38:40.827 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 9/34 2024-09-17T22:38:40.827 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 10/34 2024-09-17T22:38:40.828 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 11/34 2024-09-17T22:38:40.828 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T22:38:40.828 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T22:38:40.828 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-17T22:38:40.828 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-17T22:38:40.828 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-17T22:38:40.828 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-17T22:38:40.828 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-17T22:38:40.828 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-17T22:38:40.828 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-17T22:38:40.828 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-17T22:38:40.828 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-17T22:38:40.828 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-17T22:38:40.829 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-17T22:38:40.830 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-17T22:38:40.830 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-17T22:38:40.830 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-17T22:38:40.830 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-17T22:38:40.830 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-17T22:38:40.830 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-17T22:38:40.830 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T22:38:40.830 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-17T22:38:40.830 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 33/34 2024-09-17T22:38:41.747 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T22:38:41.747 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:38:41.748 INFO:teuthology.orchestra.run.smithi129.stdout:Upgraded: 2024-09-17T22:38:41.748 INFO:teuthology.orchestra.run.smithi129.stdout: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:41.748 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:41.748 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T22:38:41.748 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T22:38:41.748 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:41.748 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:41.748 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:41.748 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:41.748 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T22:38:41.748 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T22:38:41.748 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T22:38:41.748 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:41.749 INFO:teuthology.orchestra.run.smithi129.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T22:38:41.749 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T22:38:41.749 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T22:38:41.749 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:41.749 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T22:38:41.749 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:41.749 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T22:38:41.749 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T22:38:41.749 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T22:38:41.749 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:41.749 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:41.749 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:41.750 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T22:38:41.750 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:41.750 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:41.750 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:41.750 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T22:38:41.750 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T22:38:41.750 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T22:38:41.750 INFO:teuthology.orchestra.run.smithi129.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T22:38:41.750 INFO:teuthology.orchestra.run.smithi129.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T22:38:41.750 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:38:41.750 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:38:41.838 INFO:teuthology.orchestra.run.smithi115.stdout:(4/4): ceph-test-19.1.1-330.gc2ddf972.el9.x86_6 32 MB/s | 49 MB 00:01 2024-09-17T22:38:41.840 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:38:41.840 INFO:teuthology.orchestra.run.smithi115.stdout:Total 31 MB/s | 49 MB 00:01 2024-09-17T22:38:41.867 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-17T22:38:41.887 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-17T22:38:41.887 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-17T22:38:42.075 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-test 2024-09-17T22:38:42.275 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-17T22:38:42.275 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-17T22:38:42.655 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:27 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:38:42.786 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:38:42.786 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:38:42.786 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T22:38:42.787 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:38:42.787 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T22:38:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 49 M 2024-09-17T22:38:42.787 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T22:38:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 163 k 2024-09-17T22:38:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T22:38:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T22:38:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:38:42.787 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T22:38:42.787 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:38:42.787 INFO:teuthology.orchestra.run.smithi129.stdout:Install 4 Packages 2024-09-17T22:38:42.787 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:38:42.788 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 49 M 2024-09-17T22:38:42.788 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 209 M 2024-09-17T22:38:42.788 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T22:38:42.813 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-17T22:38:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T22:38:42.937 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T22:38:42.964 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T22:38:43.177 INFO:teuthology.orchestra.run.smithi129.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.3 MB/s | 304 kB 00:00 2024-09-17T22:38:43.219 INFO:teuthology.orchestra.run.smithi129.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-17T22:38:43.277 INFO:teuthology.orchestra.run.smithi129.stdout:(3/4): libcephsqlite-19.1.1-330.gc2ddf972.el9.x 710 kB/s | 163 kB 00:00 2024-09-17T22:38:44.561 INFO:teuthology.orchestra.run.smithi129.stdout:(4/4): ceph-test-19.1.1-330.gc2ddf972.el9.x86_6 32 MB/s | 49 MB 00:01 2024-09-17T22:38:44.563 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:38:44.563 INFO:teuthology.orchestra.run.smithi129.stdout:Total 28 MB/s | 49 MB 00:01 2024-09-17T22:38:44.591 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T22:38:44.611 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T22:38:44.611 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T22:38:45.002 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T22:38:45.002 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T22:38:45.533 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T22:38:45.607 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T22:38:45.645 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T22:38:45.672 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T22:38:47.600 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T22:38:47.679 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T22:38:50.039 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T22:38:50.040 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 1/4 2024-09-17T22:38:50.040 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2/4 2024-09-17T22:38:50.040 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T22:38:50.281 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T22:38:50.361 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T22:38:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T22:38:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:38:50.371 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-17T22:38:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T22:38:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T22:38:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:38:50.371 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:38:50.672 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph 2024-09-17T22:38:51.281 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:39 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:38:51.425 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:38:51.427 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:38:51.427 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-17T22:38:51.427 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:38:51.427 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-17T22:38:51.428 INFO:teuthology.orchestra.run.smithi115.stdout: ceph x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 6.5 k 2024-09-17T22:38:51.428 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-17T22:38:51.428 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 2.4 M 2024-09-17T22:38:51.428 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 1.5 M 2024-09-17T22:38:51.428 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 249 k 2024-09-17T22:38:51.428 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 4.7 M 2024-09-17T22:38:51.428 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 17 M 2024-09-17T22:38:51.428 INFO:teuthology.orchestra.run.smithi115.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T22:38:51.428 INFO:teuthology.orchestra.run.smithi115.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T22:38:51.428 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T22:38:51.428 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T22:38:51.429 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T22:38:51.429 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T22:38:51.429 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T22:38:51.429 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T22:38:51.429 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T22:38:51.429 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T22:38:51.429 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T22:38:51.429 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T22:38:51.429 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T22:38:51.429 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T22:38:51.429 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T22:38:51.429 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T22:38:51.429 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T22:38:51.430 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T22:38:51.430 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T22:38:51.430 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T22:38:51.430 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T22:38:51.430 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T22:38:51.430 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T22:38:51.430 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T22:38:51.430 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T22:38:51.430 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T22:38:51.430 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T22:38:51.430 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T22:38:51.430 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T22:38:51.430 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T22:38:51.431 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T22:38:51.431 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:38:51.431 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-17T22:38:51.431 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:38:51.431 INFO:teuthology.orchestra.run.smithi115.stdout:Install 37 Packages 2024-09-17T22:38:51.431 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:38:51.431 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 30 M 2024-09-17T22:38:51.432 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 108 M 2024-09-17T22:38:51.432 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-17T22:38:52.297 INFO:teuthology.orchestra.run.smithi115.stdout:(1/37): ceph-19.1.1-330.gc2ddf972.el9.x86_64.rp 53 kB/s | 6.5 kB 00:00 2024-09-17T22:38:52.464 INFO:teuthology.orchestra.run.smithi115.stdout:(2/37): ceph-mgr-19.1.1-330.gc2ddf972.el9.x86_6 5.2 MB/s | 1.5 MB 00:00 2024-09-17T22:38:52.531 INFO:teuthology.orchestra.run.smithi115.stdout:(3/37): ceph-mds-19.1.1-330.gc2ddf972.el9.x86_6 6.7 MB/s | 2.4 MB 00:00 2024-09-17T22:38:52.640 INFO:teuthology.orchestra.run.smithi115.stdout:(4/37): ceph-mon-19.1.1-330.gc2ddf972.el9.x86_6 14 MB/s | 4.7 MB 00:00 2024-09-17T22:38:52.665 INFO:teuthology.orchestra.run.smithi115.stdout:(5/37): ceph-mgr-modules-core-19.1.1-330.gc2ddf 1.8 MB/s | 249 kB 00:00 2024-09-17T22:38:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T22:38:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 1/4 2024-09-17T22:38:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2/4 2024-09-17T22:38:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T22:38:52.790 INFO:teuthology.orchestra.run.smithi115.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-17T22:38:52.832 INFO:teuthology.orchestra.run.smithi115.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-17T22:38:52.874 INFO:teuthology.orchestra.run.smithi115.stdout:(8/37): python3-cryptography-36.0.1-4.el9.x86_6 5.9 MB/s | 1.2 MB 00:00 2024-09-17T22:38:52.899 INFO:teuthology.orchestra.run.smithi115.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-09-17T22:38:52.924 INFO:teuthology.orchestra.run.smithi115.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-09-17T22:38:52.950 INFO:teuthology.orchestra.run.smithi115.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-09-17T22:38:53.066 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T22:38:53.066 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:38:53.066 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T22:38:53.066 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:53.066 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:38:53.067 INFO:teuthology.orchestra.run.smithi129.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T22:38:53.067 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T22:38:53.067 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:38:53.067 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:38:53.117 INFO:teuthology.orchestra.run.smithi115.stdout:(12/37): ceph-osd-19.1.1-330.gc2ddf972.el9.x86_ 26 MB/s | 17 MB 00:00 2024-09-17T22:38:53.320 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph 2024-09-17T22:38:53.400 INFO:teuthology.orchestra.run.smithi115.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 123 kB/s | 35 kB 00:00 2024-09-17T22:38:53.575 INFO:teuthology.orchestra.run.smithi115.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 275 kB/s | 172 kB 00:00 2024-09-17T22:38:53.601 INFO:teuthology.orchestra.run.smithi115.stdout:(15/37): lua-5.4.4-4.el9.x86_64.rpm 279 kB/s | 188 kB 00:00 2024-09-17T22:38:53.684 INFO:teuthology.orchestra.run.smithi115.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 272 kB/s | 77 kB 00:00 2024-09-17T22:38:53.709 INFO:teuthology.orchestra.run.smithi115.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 381 kB/s | 51 kB 00:00 2024-09-17T22:38:53.735 INFO:teuthology.orchestra.run.smithi115.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 311 kB/s | 42 kB 00:00 2024-09-17T22:38:53.810 INFO:teuthology.orchestra.run.smithi115.stdout:(19/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 581 kB/s | 43 kB 00:00 2024-09-17T22:38:53.835 INFO:teuthology.orchestra.run.smithi115.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-09-17T22:38:53.860 INFO:teuthology.orchestra.run.smithi115.stdout:(21/37): lua-devel-5.4.4-4.el9.x86_64.rpm 127 kB/s | 22 kB 00:00 2024-09-17T22:38:53.903 INFO:teuthology.orchestra.run.smithi115.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-09-17T22:38:53.916 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:38 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:38:53.928 INFO:teuthology.orchestra.run.smithi115.stdout:(23/37): python3-cherrypy-18.6.1-2.el9.noarch.r 3.8 MB/s | 358 kB 00:00 2024-09-17T22:38:53.953 INFO:teuthology.orchestra.run.smithi115.stdout:(24/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 116 kB/s | 11 kB 00:00 2024-09-17T22:38:53.978 INFO:teuthology.orchestra.run.smithi115.stdout:(25/37): python3-jaraco-classes-3.2.1-5.el9.noa 239 kB/s | 18 kB 00:00 2024-09-17T22:38:54.004 INFO:teuthology.orchestra.run.smithi115.stdout:(26/37): python3-jaraco-collections-3.0.0-8.el9 307 kB/s | 23 kB 00:00 2024-09-17T22:38:54.029 INFO:teuthology.orchestra.run.smithi115.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-09-17T22:38:54.054 INFO:teuthology.orchestra.run.smithi115.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-09-17T22:38:54.058 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:38:54.060 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:38:54.060 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T22:38:54.061 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:38:54.061 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T22:38:54.061 INFO:teuthology.orchestra.run.smithi129.stdout: ceph x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 6.5 k 2024-09-17T22:38:54.061 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T22:38:54.061 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 2.4 M 2024-09-17T22:38:54.061 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 1.5 M 2024-09-17T22:38:54.061 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 249 k 2024-09-17T22:38:54.061 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 4.7 M 2024-09-17T22:38:54.061 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 17 M 2024-09-17T22:38:54.061 INFO:teuthology.orchestra.run.smithi129.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T22:38:54.062 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T22:38:54.062 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T22:38:54.062 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T22:38:54.062 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T22:38:54.062 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T22:38:54.062 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T22:38:54.062 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T22:38:54.062 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T22:38:54.062 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T22:38:54.062 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T22:38:54.062 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T22:38:54.063 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T22:38:54.063 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T22:38:54.063 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T22:38:54.063 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T22:38:54.063 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T22:38:54.063 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T22:38:54.063 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T22:38:54.063 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T22:38:54.063 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T22:38:54.063 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T22:38:54.063 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T22:38:54.063 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T22:38:54.064 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T22:38:54.064 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T22:38:54.064 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T22:38:54.064 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T22:38:54.064 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T22:38:54.064 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T22:38:54.064 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T22:38:54.064 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:38:54.064 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T22:38:54.064 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:38:54.064 INFO:teuthology.orchestra.run.smithi129.stdout:Install 37 Packages 2024-09-17T22:38:54.065 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:38:54.065 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 30 M 2024-09-17T22:38:54.065 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 108 M 2024-09-17T22:38:54.065 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T22:38:54.079 INFO:teuthology.orchestra.run.smithi115.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 925 kB/s | 46 kB 00:00 2024-09-17T22:38:54.105 INFO:teuthology.orchestra.run.smithi115.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-17T22:38:54.122 INFO:teuthology.orchestra.run.smithi115.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 390 kB/s | 16 kB 00:00 2024-09-17T22:38:54.147 INFO:teuthology.orchestra.run.smithi115.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2024-09-17T22:38:54.164 INFO:teuthology.orchestra.run.smithi115.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 856 kB/s | 36 kB 00:00 2024-09-17T22:38:54.189 INFO:teuthology.orchestra.run.smithi115.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-09-17T22:38:54.215 INFO:teuthology.orchestra.run.smithi115.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-09-17T22:38:54.232 INFO:teuthology.orchestra.run.smithi115.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 479 kB/s | 20 kB 00:00 2024-09-17T22:38:54.257 INFO:teuthology.orchestra.run.smithi115.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.5 MB/s | 427 kB 00:00 2024-09-17T22:38:54.261 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:38:54.261 INFO:teuthology.orchestra.run.smithi115.stdout:Total 11 MB/s | 30 MB 00:02 2024-09-17T22:38:54.413 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-17T22:38:54.453 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-17T22:38:54.453 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-17T22:38:54.498 INFO:teuthology.orchestra.run.smithi129.stdout:(1/37): ceph-19.1.1-330.gc2ddf972.el9.x86_64.rp 54 kB/s | 6.5 kB 00:00 2024-09-17T22:38:54.698 INFO:teuthology.orchestra.run.smithi129.stdout:(2/37): ceph-mgr-19.1.1-330.gc2ddf972.el9.x86_6 4.7 MB/s | 1.5 MB 00:00 2024-09-17T22:38:54.749 INFO:teuthology.orchestra.run.smithi129.stdout:(3/37): ceph-mds-19.1.1-330.gc2ddf972.el9.x86_6 6.4 MB/s | 2.4 MB 00:00 2024-09-17T22:38:54.832 INFO:teuthology.orchestra.run.smithi129.stdout:(4/37): ceph-mgr-modules-core-19.1.1-330.gc2ddf 2.9 MB/s | 249 kB 00:00 2024-09-17T22:38:54.916 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-17T22:38:54.916 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-17T22:38:55.033 INFO:teuthology.orchestra.run.smithi129.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-17T22:38:55.100 INFO:teuthology.orchestra.run.smithi129.stdout:(6/37): ceph-mon-19.1.1-330.gc2ddf972.el9.x86_6 7.8 MB/s | 4.7 MB 00:00 2024-09-17T22:38:55.292 INFO:teuthology.orchestra.run.smithi129.stdout:(7/37): ceph-osd-19.1.1-330.gc2ddf972.el9.x86_6 28 MB/s | 17 MB 00:00 2024-09-17T22:38:55.400 INFO:teuthology.orchestra.run.smithi129.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 355 kB/s | 106 kB 00:00 2024-09-17T22:38:55.450 INFO:teuthology.orchestra.run.smithi129.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 854 kB/s | 135 kB 00:00 2024-09-17T22:38:55.476 INFO:teuthology.orchestra.run.smithi129.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 1.6 MB/s | 125 kB 00:00 2024-09-17T22:38:55.509 INFO:teuthology.orchestra.run.smithi129.stdout:(11/37): python3-cryptography-36.0.1-4.el9.x86_ 2.6 MB/s | 1.2 MB 00:00 2024-09-17T22:38:55.535 INFO:teuthology.orchestra.run.smithi129.stdout:(12/37): python3-urllib3-1.26.5-6.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-09-17T22:38:56.085 INFO:teuthology.orchestra.run.smithi129.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 63 kB/s | 35 kB 00:00 2024-09-17T22:38:56.227 INFO:teuthology.orchestra.run.smithi129.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 240 kB/s | 172 kB 00:00 2024-09-17T22:38:56.252 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-17T22:38:56.294 INFO:teuthology.orchestra.run.smithi129.stdout:(15/37): lua-5.4.4-4.el9.x86_64.rpm 230 kB/s | 188 kB 00:00 2024-09-17T22:38:56.329 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T22:38:56.379 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T22:38:56.415 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T22:38:56.485 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T22:38:56.531 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T22:38:56.572 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T22:38:56.664 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T22:38:56.719 INFO:teuthology.orchestra.run.smithi129.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 122 kB/s | 77 kB 00:00 2024-09-17T22:38:56.733 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T22:38:56.783 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T22:38:56.818 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T22:38:56.836 INFO:teuthology.orchestra.run.smithi129.stdout:(17/37): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-09-17T22:38:56.848 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T22:38:56.865 INFO:teuthology.orchestra.run.smithi129.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 73 kB/s | 42 kB 00:00 2024-09-17T22:38:56.882 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T22:38:56.899 INFO:teuthology.orchestra.run.smithi129.stdout:(19/37): python3-pytz-2021.1-5.el9.noarch.rpm 76 kB/s | 51 kB 00:00 2024-09-17T22:38:56.916 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T22:38:56.941 INFO:teuthology.orchestra.run.smithi129.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-09-17T22:38:56.956 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T22:38:56.966 INFO:teuthology.orchestra.run.smithi129.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 472 kB/s | 43 kB 00:00 2024-09-17T22:38:56.991 INFO:teuthology.orchestra.run.smithi129.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 430 kB/s | 11 kB 00:00 2024-09-17T22:38:57.000 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T22:38:57.016 INFO:teuthology.orchestra.run.smithi129.stdout:(23/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-09-17T22:38:57.034 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T22:38:57.043 INFO:teuthology.orchestra.run.smithi129.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-09-17T22:38:57.067 INFO:teuthology.orchestra.run.smithi129.stdout:(25/37): python3-cherrypy-18.6.1-2.el9.noarch.r 2.8 MB/s | 358 kB 00:00 2024-09-17T22:38:57.083 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T22:38:57.092 INFO:teuthology.orchestra.run.smithi129.stdout:(26/37): python3-jaraco-collections-3.0.0-8.el9 307 kB/s | 23 kB 00:00 2024-09-17T22:38:57.115 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T22:38:57.118 INFO:teuthology.orchestra.run.smithi129.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 257 kB/s | 19 kB 00:00 2024-09-17T22:38:57.143 INFO:teuthology.orchestra.run.smithi129.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 260 kB/s | 20 kB 00:00 2024-09-17T22:38:57.168 INFO:teuthology.orchestra.run.smithi129.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 613 kB/s | 46 kB 00:00 2024-09-17T22:38:57.187 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T22:38:57.194 INFO:teuthology.orchestra.run.smithi129.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-17T22:38:57.230 INFO:teuthology.orchestra.run.smithi129.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 269 kB/s | 16 kB 00:00 2024-09-17T22:38:57.270 INFO:teuthology.orchestra.run.smithi129.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-17T22:38:57.295 INFO:teuthology.orchestra.run.smithi129.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 551 kB/s | 36 kB 00:00 2024-09-17T22:38:57.320 INFO:teuthology.orchestra.run.smithi129.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-09-17T22:38:57.333 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T22:38:57.346 INFO:teuthology.orchestra.run.smithi129.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-17T22:38:57.371 INFO:teuthology.orchestra.run.smithi129.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 399 kB/s | 20 kB 00:00 2024-09-17T22:38:57.412 INFO:teuthology.orchestra.run.smithi129.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-09-17T22:38:57.414 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T22:38:57.417 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:38:57.417 INFO:teuthology.orchestra.run.smithi129.stdout:Total 9.0 MB/s | 30 MB 00:03 2024-09-17T22:38:57.468 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T22:38:57.506 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T22:38:57.555 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T22:38:57.570 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T22:38:57.610 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T22:38:57.610 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T22:38:57.610 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T22:38:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T22:38:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T22:38:57.948 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T22:38:58.037 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T22:38:58.078 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T22:38:58.079 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T22:38:58.082 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T22:38:58.202 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T22:38:58.358 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 32/37 2024-09-17T22:38:58.389 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T22:38:58.415 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T22:38:58.415 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:38:58.415 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T22:38:58.415 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T22:38:58.415 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T22:38:58.415 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:38:59.411 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T22:38:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T22:38:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T22:38:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T22:38:59.597 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T22:38:59.642 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T22:38:59.675 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T22:38:59.767 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T22:38:59.846 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T22:38:59.856 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T22:38:59.884 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T22:38:59.884 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:38:59.884 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T22:38:59.885 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T22:38:59.885 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T22:38:59.885 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:38:59.894 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T22:38:59.938 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T22:38:59.968 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T22:39:00.002 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T22:39:00.036 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T22:39:00.076 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T22:39:00.111 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T22:39:00.146 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T22:39:00.186 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T22:39:00.226 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T22:39:00.299 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T22:39:00.324 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T22:39:00.347 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T22:39:00.347 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:39:00.347 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T22:39:00.347 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T22:39:00.348 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T22:39:00.348 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:00.447 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T22:39:00.531 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T22:39:00.574 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T22:39:00.592 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T22:39:00.598 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T22:39:00.598 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:39:00.598 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T22:39:00.598 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T22:39:00.598 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T22:39:00.598 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:00.631 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T22:39:00.662 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T22:39:00.679 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T22:39:00.735 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T22:39:00.930 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T22:39:00.977 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T22:39:01.055 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T22:39:01.152 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T22:39:01.199 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T22:39:01.317 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T22:39:01.472 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 32/37 2024-09-17T22:39:01.503 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T22:39:01.529 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T22:39:01.529 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:39:01.529 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T22:39:01.529 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T22:39:01.529 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T22:39:01.529 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T22:39:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 1/37 2024-09-17T22:39:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2/37 2024-09-17T22:39:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 3/37 2024-09-17T22:39:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 4/37 2024-09-17T22:39:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 5/37 2024-09-17T22:39:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 6/37 2024-09-17T22:39:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T22:39:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T22:39:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T22:39:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T22:39:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T22:39:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T22:39:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T22:39:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T22:39:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T22:39:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T22:39:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T22:39:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T22:39:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T22:39:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T22:39:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T22:39:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T22:39:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T22:39:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T22:39:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T22:39:02.535 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T22:39:02.535 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T22:39:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T22:39:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T22:39:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T22:39:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T22:39:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T22:39:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T22:39:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T22:39:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T22:39:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T22:39:02.963 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T22:39:02.990 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T22:39:02.990 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:39:02.990 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T22:39:02.990 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T22:39:02.991 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T22:39:02.991 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:03.424 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T22:39:03.446 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T22:39:03.446 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:39:03.446 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T22:39:03.446 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T22:39:03.446 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T22:39:03.446 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:03.494 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T22:39:03.494 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:03.494 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-17T22:39:03.494 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:39:03.494 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:39:03.494 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:39:03.494 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:39:03.494 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:39:03.494 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:39:03.494 INFO:teuthology.orchestra.run.smithi115.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T22:39:03.495 INFO:teuthology.orchestra.run.smithi115.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T22:39:03.495 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T22:39:03.495 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T22:39:03.495 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T22:39:03.495 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T22:39:03.495 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T22:39:03.495 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T22:39:03.495 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T22:39:03.495 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T22:39:03.495 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T22:39:03.495 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T22:39:03.495 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T22:39:03.495 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T22:39:03.495 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T22:39:03.496 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T22:39:03.496 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T22:39:03.496 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T22:39:03.496 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T22:39:03.496 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T22:39:03.496 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T22:39:03.496 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T22:39:03.496 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T22:39:03.496 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T22:39:03.496 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T22:39:03.496 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T22:39:03.496 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T22:39:03.496 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T22:39:03.496 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T22:39:03.497 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T22:39:03.497 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T22:39:03.497 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:03.497 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:03.672 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T22:39:03.696 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T22:39:03.696 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:39:03.696 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T22:39:03.696 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T22:39:03.696 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T22:39:03.696 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:03.761 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T22:39:03.895 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-base 2024-09-17T22:39:04.479 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:52 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:04.568 INFO:teuthology.orchestra.run.smithi115.stdout:Package ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:39:04.611 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:04.612 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-17T22:39:04.612 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:04.683 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install cephadm 2024-09-17T22:39:05.262 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:53 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:05.394 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:05.395 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:05.395 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-17T22:39:05.395 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:05.395 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-17T22:39:05.395 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 765 k 2024-09-17T22:39:05.395 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:05.396 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-17T22:39:05.396 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:05.396 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-17T22:39:05.396 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:05.396 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 765 k 2024-09-17T22:39:05.396 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 771 k 2024-09-17T22:39:05.396 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-17T22:39:05.484 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T22:39:05.484 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 1/37 2024-09-17T22:39:05.484 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2/37 2024-09-17T22:39:05.484 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 3/37 2024-09-17T22:39:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 4/37 2024-09-17T22:39:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 5/37 2024-09-17T22:39:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 6/37 2024-09-17T22:39:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T22:39:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T22:39:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T22:39:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T22:39:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T22:39:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T22:39:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T22:39:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T22:39:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T22:39:05.485 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T22:39:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T22:39:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T22:39:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T22:39:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T22:39:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T22:39:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T22:39:05.486 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T22:39:05.487 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T22:39:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T22:39:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T22:39:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T22:39:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T22:39:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T22:39:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T22:39:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T22:39:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T22:39:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T22:39:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T22:39:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T22:39:05.488 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T22:39:05.665 INFO:teuthology.orchestra.run.smithi115.stdout:cephadm-19.1.1-330.gc2ddf972.el9.noarch.rpm 2.8 MB/s | 765 kB 00:00 2024-09-17T22:39:05.665 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:05.666 INFO:teuthology.orchestra.run.smithi115.stdout:Total 2.8 MB/s | 765 kB 00:00 2024-09-17T22:39:05.666 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-17T22:39:05.671 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-17T22:39:05.671 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-17T22:39:05.677 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-17T22:39:05.677 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-17T22:39:05.756 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-17T22:39:06.234 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:39:06.318 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:39:06.634 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T22:39:06.634 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:06.635 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T22:39:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:39:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:39:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:39:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:39:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:39:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:39:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T22:39:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T22:39:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T22:39:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T22:39:06.635 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T22:39:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T22:39:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T22:39:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T22:39:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T22:39:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T22:39:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T22:39:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T22:39:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T22:39:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T22:39:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T22:39:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T22:39:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T22:39:06.636 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T22:39:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T22:39:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T22:39:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T22:39:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T22:39:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T22:39:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T22:39:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T22:39:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T22:39:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T22:39:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T22:39:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T22:39:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T22:39:06.637 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T22:39:06.638 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:06.638 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:06.677 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:39:06.939 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:39:06.939 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:06.939 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-17T22:39:06.939 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:39:06.939 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:06.939 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:06.996 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-base 2024-09-17T22:39:07.144 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T22:39:07.581 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:52 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:39:07.669 INFO:teuthology.orchestra.run.smithi129.stdout:Package ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:39:07.711 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:39:07.712 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T22:39:07.712 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:07.735 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:55 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:07.774 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install cephadm 2024-09-17T22:39:07.871 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:07.871 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:07.871 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-17T22:39:07.872 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:07.872 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-17T22:39:07.872 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 145 k 2024-09-17T22:39:07.872 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:07.872 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-17T22:39:07.872 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:07.872 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-17T22:39:07.872 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:07.873 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 145 k 2024-09-17T22:39:07.873 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 435 k 2024-09-17T22:39:07.873 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-17T22:39:08.079 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-immutable-object-cache-19.1.1-330.gc2ddf97 703 kB/s | 145 kB 00:00 2024-09-17T22:39:08.079 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:08.080 INFO:teuthology.orchestra.run.smithi115.stdout:Total 698 kB/s | 145 kB 00:00 2024-09-17T22:39:08.080 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-17T22:39:08.089 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-17T22:39:08.089 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-17T22:39:08.145 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-17T22:39:08.145 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-17T22:39:08.303 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-17T22:39:08.353 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:53 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:39:08.380 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T22:39:08.405 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T22:39:08.406 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:39:08.406 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T22:39:08.406 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:08.486 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:39:08.487 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:08.487 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T22:39:08.487 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:08.487 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T22:39:08.487 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 765 k 2024-09-17T22:39:08.487 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:08.487 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T22:39:08.487 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:08.488 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T22:39:08.488 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:08.488 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 765 k 2024-09-17T22:39:08.488 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 771 k 2024-09-17T22:39:08.488 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T22:39:08.765 INFO:teuthology.orchestra.run.smithi129.stdout:cephadm-19.1.1-330.gc2ddf972.el9.noarch.rpm 2.7 MB/s | 765 kB 00:00 2024-09-17T22:39:08.766 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:08.766 INFO:teuthology.orchestra.run.smithi129.stdout:Total 2.7 MB/s | 765 kB 00:00 2024-09-17T22:39:08.766 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T22:39:08.771 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T22:39:08.771 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T22:39:08.777 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T22:39:08.778 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T22:39:08.855 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T22:39:09.297 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T22:39:09.297 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:09.297 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-17T22:39:09.297 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:39:09.297 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:09.297 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:09.318 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:39:09.402 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:39:09.536 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr 2024-09-17T22:39:09.798 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:39:10.040 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:39:10.040 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:10.040 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T22:39:10.040 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:39:10.040 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:10.040 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:10.125 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:10.215 INFO:teuthology.orchestra.run.smithi115.stdout:Package ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:39:10.258 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:10.259 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-17T22:39:10.259 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:10.261 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T22:39:10.332 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T22:39:10.860 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:55 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:39:10.900 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:10.996 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:39:10.997 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:10.997 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-17T22:39:10.997 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:10.997 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T22:39:10.997 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 145 k 2024-09-17T22:39:10.997 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:10.997 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T22:39:10.997 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:10.997 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T22:39:10.997 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:10.997 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 145 k 2024-09-17T22:39:10.997 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 435 k 2024-09-17T22:39:10.998 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T22:39:11.031 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:11.032 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:11.032 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-17T22:39:11.032 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:11.032 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-17T22:39:11.032 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 3.6 M 2024-09-17T22:39:11.033 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-17T22:39:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T22:39:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 32 k 2024-09-17T22:39:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 16 k 2024-09-17T22:39:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T22:39:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T22:39:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T22:39:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T22:39:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T22:39:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T22:39:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T22:39:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T22:39:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:11.033 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-17T22:39:11.034 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:11.034 INFO:teuthology.orchestra.run.smithi115.stdout:Install 12 Packages 2024-09-17T22:39:11.034 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:11.034 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 8.7 M 2024-09-17T22:39:11.035 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 104 M 2024-09-17T22:39:11.035 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-17T22:39:11.200 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-immutable-object-cache-19.1.1-330.gc2ddf97 719 kB/s | 145 kB 00:00 2024-09-17T22:39:11.200 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:11.201 INFO:teuthology.orchestra.run.smithi129.stdout:Total 713 kB/s | 145 kB 00:00 2024-09-17T22:39:11.201 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T22:39:11.208 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T22:39:11.208 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T22:39:11.266 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T22:39:11.266 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T22:39:11.293 INFO:teuthology.orchestra.run.smithi115.stdout:(1/12): ceph-grafana-dashboards-19.1.1-330.gc2d 201 kB/s | 32 kB 00:00 2024-09-17T22:39:11.318 INFO:teuthology.orchestra.run.smithi115.stdout:(2/12): ceph-prometheus-alerts-19.1.1-330.gc2dd 90 kB/s | 16 kB 00:00 2024-09-17T22:39:11.410 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T22:39:11.468 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T22:39:11.493 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T22:39:11.493 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:39:11.493 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T22:39:11.493 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:11.544 INFO:teuthology.orchestra.run.smithi115.stdout:(3/12): ceph-mgr-dashboard-19.1.1-330.gc2ddf972 8.7 MB/s | 3.6 MB 00:00 2024-09-17T22:39:11.594 INFO:teuthology.orchestra.run.smithi115.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 978 kB/s | 269 kB 00:00 2024-09-17T22:39:11.697 INFO:teuthology.orchestra.run.smithi115.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.5 MB/s | 1.0 MB 00:00 2024-09-17T22:39:11.745 INFO:teuthology.orchestra.run.smithi115.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 3.6 MB/s | 548 kB 00:00 2024-09-17T22:39:11.770 INFO:teuthology.orchestra.run.smithi115.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-09-17T22:39:11.837 INFO:teuthology.orchestra.run.smithi115.stdout:(8/12): protobuf-compiler-3.14.0-13.el9.x86_64. 2.9 MB/s | 863 kB 00:00 2024-09-17T22:39:11.879 INFO:teuthology.orchestra.run.smithi115.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 741 kB/s | 31 kB 00:00 2024-09-17T22:39:11.937 INFO:teuthology.orchestra.run.smithi115.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.2 MB/s | 188 kB 00:00 2024-09-17T22:39:11.996 INFO:teuthology.orchestra.run.smithi115.stdout:(11/12): python3-grpcio-tools-1.46.7-10.el9.x86 660 kB/s | 144 kB 00:00 2024-09-17T22:39:12.129 INFO:teuthology.orchestra.run.smithi115.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 5.3 MB/s | 2.0 MB 00:00 2024-09-17T22:39:12.133 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:12.133 INFO:teuthology.orchestra.run.smithi115.stdout:Total 7.9 MB/s | 8.7 MB 00:01 2024-09-17T22:39:12.181 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-17T22:39:12.197 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-17T22:39:12.197 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-17T22:39:12.409 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-17T22:39:12.409 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-17T22:39:12.481 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T22:39:12.482 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:12.482 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T22:39:12.482 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:39:12.482 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:12.482 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:12.732 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr 2024-09-17T22:39:12.950 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-17T22:39:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T22:39:13.073 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T22:39:13.128 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T22:39:13.159 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T22:39:13.263 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T22:39:13.322 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:39:13.384 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T22:39:13.410 INFO:teuthology.orchestra.run.smithi129.stdout:Package ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:39:13.451 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:39:13.452 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T22:39:13.453 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:13.469 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T22:39:13.514 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T22:39:13.521 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T22:39:13.559 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T22:39:13.590 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 10/12 2024-09-17T22:39:14.088 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:59 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:39:14.219 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:39:14.220 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:14.220 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T22:39:14.220 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:14.221 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T22:39:14.221 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 3.6 M 2024-09-17T22:39:14.221 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T22:39:14.221 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T22:39:14.221 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 32 k 2024-09-17T22:39:14.221 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 16 k 2024-09-17T22:39:14.221 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T22:39:14.221 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T22:39:14.221 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T22:39:14.221 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T22:39:14.221 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T22:39:14.221 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T22:39:14.221 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T22:39:14.222 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T22:39:14.222 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:14.222 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T22:39:14.222 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:14.222 INFO:teuthology.orchestra.run.smithi129.stdout:Install 12 Packages 2024-09-17T22:39:14.222 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:14.222 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 8.7 M 2024-09-17T22:39:14.222 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 104 M 2024-09-17T22:39:14.223 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T22:39:14.638 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 11/12 2024-09-17T22:39:14.674 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T22:39:15.021 INFO:teuthology.orchestra.run.smithi129.stdout:(1/12): ceph-prometheus-alerts-19.1.1-330.gc2dd 109 kB/s | 16 kB 00:00 2024-09-17T22:39:15.046 INFO:teuthology.orchestra.run.smithi129.stdout:(2/12): ceph-grafana-dashboards-19.1.1-330.gc2d 180 kB/s | 32 kB 00:00 2024-09-17T22:39:15.163 INFO:teuthology.orchestra.run.smithi129.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.3 MB/s | 269 kB 00:00 2024-09-17T22:39:15.222 INFO:teuthology.orchestra.run.smithi129.stdout:(4/12): ceph-mgr-dashboard-19.1.1-330.gc2ddf972 10 MB/s | 3.6 MB 00:00 2024-09-17T22:39:15.256 INFO:teuthology.orchestra.run.smithi129.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.3 MB/s | 1.0 MB 00:00 2024-09-17T22:39:15.315 INFO:teuthology.orchestra.run.smithi129.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 379 kB/s | 19 kB 00:00 2024-09-17T22:39:15.381 INFO:teuthology.orchestra.run.smithi129.stdout:(7/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.9 MB/s | 863 kB 00:00 2024-09-17T22:39:15.423 INFO:teuthology.orchestra.run.smithi129.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 2.7 MB/s | 548 kB 00:00 2024-09-17T22:39:15.457 INFO:teuthology.orchestra.run.smithi129.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 976 kB/s | 31 kB 00:00 2024-09-17T22:39:15.499 INFO:teuthology.orchestra.run.smithi129.stdout:(10/12): python3-grpcio-tools-1.46.7-10.el9.x86 1.2 MB/s | 144 kB 00:00 2024-09-17T22:39:15.533 INFO:teuthology.orchestra.run.smithi129.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.4 MB/s | 188 kB 00:00 2024-09-17T22:39:15.658 INFO:teuthology.orchestra.run.smithi129.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 5.9 MB/s | 2.0 MB 00:00 2024-09-17T22:39:15.662 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:15.662 INFO:teuthology.orchestra.run.smithi129.stdout:Total 6.0 MB/s | 8.7 MB 00:01 2024-09-17T22:39:15.710 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T22:39:15.727 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T22:39:15.728 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T22:39:15.930 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T22:39:15.931 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T22:39:16.121 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T22:39:16.121 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 1/12 2024-09-17T22:39:16.121 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 2/12 2024-09-17T22:39:16.122 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 3/12 2024-09-17T22:39:16.122 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T22:39:16.122 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T22:39:16.122 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T22:39:16.122 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T22:39:16.122 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T22:39:16.122 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T22:39:16.122 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T22:39:16.122 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T22:39:16.455 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T22:39:16.455 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:16.455 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-17T22:39:16.455 INFO:teuthology.orchestra.run.smithi115.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T22:39:16.455 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:39:16.455 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:39:16.456 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:39:16.456 INFO:teuthology.orchestra.run.smithi115.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T22:39:16.456 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T22:39:16.456 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T22:39:16.456 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T22:39:16.456 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T22:39:16.456 INFO:teuthology.orchestra.run.smithi115.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T22:39:16.456 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T22:39:16.456 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T22:39:16.456 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:16.457 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:16.460 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T22:39:16.542 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T22:39:16.585 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T22:39:16.633 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T22:39:16.662 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T22:39:16.710 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T22:39:16.769 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T22:39:16.897 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T22:39:16.991 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T22:39:17.037 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T22:39:17.063 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T22:39:17.086 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 10/12 2024-09-17T22:39:17.292 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:05 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:17.425 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:17.425 INFO:teuthology.orchestra.run.smithi115.stdout:====================================================================================== 2024-09-17T22:39:17.426 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-17T22:39:17.426 INFO:teuthology.orchestra.run.smithi115.stdout:====================================================================================== 2024-09-17T22:39:17.426 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-17T22:39:17.426 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 7.4 M 2024-09-17T22:39:17.426 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-17T22:39:17.426 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T22:39:17.426 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T22:39:17.426 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T22:39:17.426 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T22:39:17.426 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T22:39:17.426 INFO:teuthology.orchestra.run.smithi115.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T22:39:17.426 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T22:39:17.427 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T22:39:17.427 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T22:39:17.427 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T22:39:17.427 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T22:39:17.427 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:17.427 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-17T22:39:17.427 INFO:teuthology.orchestra.run.smithi115.stdout:====================================================================================== 2024-09-17T22:39:17.427 INFO:teuthology.orchestra.run.smithi115.stdout:Install 12 Packages 2024-09-17T22:39:17.427 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:17.428 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 38 M 2024-09-17T22:39:17.428 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 212 M 2024-09-17T22:39:17.428 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-17T22:39:17.595 INFO:teuthology.orchestra.run.smithi115.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 191 kB 00:00 2024-09-17T22:39:17.812 INFO:teuthology.orchestra.run.smithi115.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-17T22:39:17.987 INFO:teuthology.orchestra.run.smithi115.stdout:(3/12): ceph-mgr-diskprediction-local-19.1.1-33 15 MB/s | 7.4 MB 00:00 2024-09-17T22:39:18.136 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 11/12 2024-09-17T22:39:18.171 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T22:39:18.271 INFO:teuthology.orchestra.run.smithi115.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 48 kB/s | 33 kB 00:00 2024-09-17T22:39:18.471 INFO:teuthology.orchestra.run.smithi115.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 38 kB/s | 18 kB 00:00 2024-09-17T22:39:18.838 INFO:teuthology.orchestra.run.smithi115.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 68 kB/s | 38 kB 00:00 2024-09-17T22:39:18.897 INFO:teuthology.orchestra.run.smithi115.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.8 MB/s | 3.0 MB 00:01 2024-09-17T22:39:19.493 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T22:39:19.493 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 1/12 2024-09-17T22:39:19.493 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 2/12 2024-09-17T22:39:19.493 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 3/12 2024-09-17T22:39:19.493 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T22:39:19.494 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T22:39:19.494 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T22:39:19.494 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T22:39:19.494 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T22:39:19.494 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T22:39:19.494 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T22:39:19.494 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T22:39:19.589 INFO:teuthology.orchestra.run.smithi115.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 326 kB/s | 245 kB 00:00 2024-09-17T22:39:19.748 INFO:teuthology.orchestra.run.smithi115.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 3.9 MB/s | 4.9 MB 00:01 2024-09-17T22:39:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T22:39:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:19.821 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T22:39:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T22:39:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:39:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:39:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:39:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T22:39:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T22:39:19.821 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T22:39:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T22:39:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T22:39:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T22:39:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T22:39:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T22:39:19.822 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:19.822 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:20.081 INFO:teuthology.orchestra.run.smithi115.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.3 MB/s | 5.1 MB 00:01 2024-09-17T22:39:20.105 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T22:39:20.431 INFO:teuthology.orchestra.run.smithi115.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 298 kB/s | 250 kB 00:00 2024-09-17T22:39:20.683 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:05 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:39:20.815 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:39:20.816 INFO:teuthology.orchestra.run.smithi129.stdout:====================================================================================== 2024-09-17T22:39:20.816 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T22:39:20.817 INFO:teuthology.orchestra.run.smithi129.stdout:====================================================================================== 2024-09-17T22:39:20.817 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T22:39:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 7.4 M 2024-09-17T22:39:20.817 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T22:39:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T22:39:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T22:39:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T22:39:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T22:39:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T22:39:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T22:39:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T22:39:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T22:39:20.817 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T22:39:20.818 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T22:39:20.818 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T22:39:20.818 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:20.818 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T22:39:20.818 INFO:teuthology.orchestra.run.smithi129.stdout:====================================================================================== 2024-09-17T22:39:20.818 INFO:teuthology.orchestra.run.smithi129.stdout:Install 12 Packages 2024-09-17T22:39:20.818 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:20.818 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 38 M 2024-09-17T22:39:20.819 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 212 M 2024-09-17T22:39:20.819 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T22:39:21.206 INFO:teuthology.orchestra.run.smithi115.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-17T22:39:21.208 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:21.208 INFO:teuthology.orchestra.run.smithi115.stdout:Total 10 MB/s | 38 MB 00:03 2024-09-17T22:39:21.570 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-17T22:39:21.597 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-17T22:39:21.597 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-17T22:39:21.792 INFO:teuthology.orchestra.run.smithi129.stdout:(1/12): ceph-mgr-diskprediction-local-19.1.1-33 17 MB/s | 7.4 MB 00:00 2024-09-17T22:39:21.859 INFO:teuthology.orchestra.run.smithi129.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 492 kB/s | 33 kB 00:00 2024-09-17T22:39:21.901 INFO:teuthology.orchestra.run.smithi129.stdout:(3/12): libquadmath-11.5.0-2.el9.x86_64.rpm 347 kB/s | 191 kB 00:00 2024-09-17T22:39:21.993 INFO:teuthology.orchestra.run.smithi129.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 199 kB/s | 18 kB 00:00 2024-09-17T22:39:22.029 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-17T22:39:22.029 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-17T22:39:22.068 INFO:teuthology.orchestra.run.smithi129.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 512 kB/s | 38 kB 00:00 2024-09-17T22:39:22.260 INFO:teuthology.orchestra.run.smithi129.stdout:(6/12): libgfortran-11.5.0-2.el9.x86_64.rpm 881 kB/s | 801 kB 00:00 2024-09-17T22:39:22.502 INFO:teuthology.orchestra.run.smithi129.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.0 MB/s | 245 kB 00:00 2024-09-17T22:39:22.586 INFO:teuthology.orchestra.run.smithi129.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.1 MB/s | 3.0 MB 00:00 2024-09-17T22:39:22.711 INFO:teuthology.orchestra.run.smithi129.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.0 MB/s | 250 kB 00:00 2024-09-17T22:39:22.820 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-17T22:39:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T22:39:22.937 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T22:39:23.046 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T22:39:23.089 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T22:39:23.236 INFO:teuthology.orchestra.run.smithi129.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.2 MB/s | 4.9 MB 00:01 2024-09-17T22:39:23.276 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T22:39:23.304 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T22:39:23.380 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T22:39:23.745 INFO:teuthology.orchestra.run.smithi129.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.1 MB/s | 5.1 MB 00:01 2024-09-17T22:39:24.043 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T22:39:24.151 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T22:39:24.803 INFO:teuthology.orchestra.run.smithi129.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.8 MB/s | 16 MB 00:02 2024-09-17T22:39:24.804 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:24.804 INFO:teuthology.orchestra.run.smithi129.stdout:Total 9.6 MB/s | 38 MB 00:03 2024-09-17T22:39:25.159 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T22:39:25.186 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T22:39:25.186 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T22:39:25.407 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T22:39:25.612 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T22:39:25.612 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T22:39:26.387 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T22:39:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T22:39:26.452 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T22:39:26.470 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T22:39:26.501 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T22:39:26.602 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T22:39:26.645 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T22:39:26.829 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T22:39:26.860 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T22:39:26.937 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T22:39:27.604 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T22:39:27.718 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T22:39:28.975 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T22:39:29.068 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T22:39:29.068 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 1/12 2024-09-17T22:39:29.068 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T22:39:29.068 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T22:39:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T22:39:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T22:39:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T22:39:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T22:39:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T22:39:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T22:39:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T22:39:29.069 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T22:39:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T22:39:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:29.539 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-17T22:39:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:39:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T22:39:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T22:39:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T22:39:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T22:39:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T22:39:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T22:39:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T22:39:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T22:39:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T22:39:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T22:39:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T22:39:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:29.540 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:29.891 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-rook 2024-09-17T22:39:30.001 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T22:39:30.028 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T22:39:30.482 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:18 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:30.615 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:30.616 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-17T22:39:30.617 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:30.617 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-17T22:39:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 49 k 2024-09-17T22:39:30.617 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-17T22:39:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T22:39:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T22:39:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T22:39:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T22:39:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T22:39:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T22:39:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T22:39:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T22:39:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T22:39:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T22:39:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T22:39:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T22:39:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:30.618 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-17T22:39:30.618 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:30.618 INFO:teuthology.orchestra.run.smithi115.stdout:Install 13 Packages 2024-09-17T22:39:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:30.619 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 2.2 M 2024-09-17T22:39:30.619 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 26 M 2024-09-17T22:39:30.619 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-17T22:39:30.997 INFO:teuthology.orchestra.run.smithi115.stdout:(1/13): ceph-mgr-rook-19.1.1-330.gc2ddf972.el9. 288 kB/s | 49 kB 00:00 2024-09-17T22:39:31.380 INFO:teuthology.orchestra.run.smithi115.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 34 kB/s | 19 kB 00:00 2024-09-17T22:39:31.431 INFO:teuthology.orchestra.run.smithi115.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 42 kB/s | 26 kB 00:00 2024-09-17T22:39:31.839 INFO:teuthology.orchestra.run.smithi115.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 264 kB/s | 222 kB 00:00 2024-09-17T22:39:32.214 INFO:teuthology.orchestra.run.smithi115.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 190 kB/s | 159 kB 00:00 2024-09-17T22:39:32.265 INFO:teuthology.orchestra.run.smithi115.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 651 kB/s | 32 kB 00:00 2024-09-17T22:39:32.298 INFO:teuthology.orchestra.run.smithi115.stdout:(7/13): python3-certifi-2023.05.07-4.el9.noarch 426 kB/s | 14 kB 00:00 2024-09-17T22:39:32.323 INFO:teuthology.orchestra.run.smithi115.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 312 kB/s | 279 kB 00:00 2024-09-17T22:39:32.382 INFO:teuthology.orchestra.run.smithi115.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 2.6 MB/s | 220 kB 00:00 2024-09-17T22:39:32.399 INFO:teuthology.orchestra.run.smithi115.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-17T22:39:32.441 INFO:teuthology.orchestra.run.smithi115.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 2.1 MB/s | 90 kB 00:00 2024-09-17T22:39:32.457 INFO:teuthology.orchestra.run.smithi115.stdout:(12/13): python3-requests-oauthlib-1.3.0-12.el9 87 kB/s | 54 kB 00:00 2024-09-17T22:39:32.516 INFO:teuthology.orchestra.run.smithi115.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.3 MB/s | 1.0 MB 00:00 2024-09-17T22:39:32.520 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:32.520 INFO:teuthology.orchestra.run.smithi115.stdout:Total 1.2 MB/s | 2.2 MB 00:01 2024-09-17T22:39:32.591 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-17T22:39:32.607 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-17T22:39:32.607 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-17T22:39:32.622 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T22:39:32.622 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 1/12 2024-09-17T22:39:32.622 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T22:39:32.622 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T22:39:32.622 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T22:39:32.622 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T22:39:32.622 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T22:39:32.622 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T22:39:32.622 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T22:39:32.622 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T22:39:32.622 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T22:39:32.622 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T22:39:32.699 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-17T22:39:32.699 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-17T22:39:33.024 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T22:39:33.024 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:33.024 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T22:39:33.024 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:39:33.024 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T22:39:33.024 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T22:39:33.024 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T22:39:33.024 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T22:39:33.025 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T22:39:33.025 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T22:39:33.025 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T22:39:33.025 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T22:39:33.025 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T22:39:33.025 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T22:39:33.025 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T22:39:33.025 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:33.025 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:33.207 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-17T22:39:33.380 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T22:39:33.425 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-rook 2024-09-17T22:39:33.443 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T22:39:33.498 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T22:39:33.543 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T22:39:33.593 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T22:39:33.695 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T22:39:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T22:39:33.845 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T22:39:34.016 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:19 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:39:34.149 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:39:34.150 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:34.150 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T22:39:34.150 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:34.150 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T22:39:34.150 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 49 k 2024-09-17T22:39:34.150 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T22:39:34.150 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T22:39:34.150 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T22:39:34.150 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T22:39:34.150 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T22:39:34.150 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T22:39:34.151 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T22:39:34.151 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T22:39:34.151 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T22:39:34.151 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T22:39:34.151 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T22:39:34.151 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T22:39:34.151 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T22:39:34.151 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:34.151 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T22:39:34.151 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:34.151 INFO:teuthology.orchestra.run.smithi129.stdout:Install 13 Packages 2024-09-17T22:39:34.151 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:34.152 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 2.2 M 2024-09-17T22:39:34.152 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 26 M 2024-09-17T22:39:34.152 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T22:39:34.559 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T22:39:34.609 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T22:39:34.631 INFO:teuthology.orchestra.run.smithi129.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 252 kB/s | 19 kB 00:00 2024-09-17T22:39:34.642 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T22:39:34.656 INFO:teuthology.orchestra.run.smithi129.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 253 kB/s | 26 kB 00:00 2024-09-17T22:39:34.691 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T22:39:34.715 INFO:teuthology.orchestra.run.smithi129.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 222 kB 00:00 2024-09-17T22:39:34.721 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T22:39:34.741 INFO:teuthology.orchestra.run.smithi129.stdout:(4/13): ceph-mgr-rook-19.1.1-330.gc2ddf972.el9. 266 kB/s | 49 kB 00:00 2024-09-17T22:39:34.766 INFO:teuthology.orchestra.run.smithi129.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-17T22:39:34.791 INFO:teuthology.orchestra.run.smithi129.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.6 MB/s | 279 kB 00:00 2024-09-17T22:39:34.833 INFO:teuthology.orchestra.run.smithi129.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 581 kB/s | 54 kB 00:00 2024-09-17T22:39:34.858 INFO:teuthology.orchestra.run.smithi129.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 350 kB/s | 32 kB 00:00 2024-09-17T22:39:34.884 INFO:teuthology.orchestra.run.smithi129.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 165 kB/s | 14 kB 00:00 2024-09-17T22:39:34.917 INFO:teuthology.orchestra.run.smithi129.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.8 MB/s | 59 kB 00:00 2024-09-17T22:39:34.959 INFO:teuthology.orchestra.run.smithi129.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 2.1 MB/s | 90 kB 00:00 2024-09-17T22:39:34.984 INFO:teuthology.orchestra.run.smithi129.stdout:(12/13): python3-google-auth-2.34.0-1.el9.noarc 1.4 MB/s | 220 kB 00:00 2024-09-17T22:39:35.060 INFO:teuthology.orchestra.run.smithi129.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.3 MB/s | 1.0 MB 00:00 2024-09-17T22:39:35.062 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:35.062 INFO:teuthology.orchestra.run.smithi129.stdout:Total 2.4 MB/s | 2.2 MB 00:00 2024-09-17T22:39:35.126 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T22:39:35.141 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T22:39:35.141 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T22:39:35.232 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T22:39:35.232 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T22:39:35.552 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T22:39:35.552 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 1/13 2024-09-17T22:39:35.552 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T22:39:35.552 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T22:39:35.553 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T22:39:35.553 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T22:39:35.553 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T22:39:35.553 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T22:39:35.553 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T22:39:35.554 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T22:39:35.554 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T22:39:35.554 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T22:39:35.554 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T22:39:35.753 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T22:39:35.874 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T22:39:35.874 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:35.874 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-17T22:39:35.874 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:39:35.874 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T22:39:35.874 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T22:39:35.874 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T22:39:35.874 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T22:39:35.874 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T22:39:35.874 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T22:39:35.874 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T22:39:35.875 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T22:39:35.875 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T22:39:35.875 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T22:39:35.875 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T22:39:35.875 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T22:39:35.875 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:35.875 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:35.923 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T22:39:35.977 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T22:39:36.041 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T22:39:36.079 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T22:39:36.121 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T22:39:36.148 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T22:39:36.222 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T22:39:36.357 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T22:39:36.414 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T22:39:36.728 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:24 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:36.860 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:36.860 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:36.860 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-17T22:39:36.860 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:36.861 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-17T22:39:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 146 k 2024-09-17T22:39:36.861 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-17T22:39:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T22:39:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T22:39:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T22:39:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T22:39:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T22:39:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:36.861 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-17T22:39:36.861 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:36.861 INFO:teuthology.orchestra.run.smithi115.stdout:Install 6 Packages 2024-09-17T22:39:36.861 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:36.862 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 7.0 M 2024-09-17T22:39:36.862 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 33 M 2024-09-17T22:39:36.862 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-17T22:39:37.129 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T22:39:37.208 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T22:39:37.246 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T22:39:37.293 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T22:39:37.315 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T22:39:37.319 INFO:teuthology.orchestra.run.smithi115.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.6 MB/s | 248 kB 00:00 2024-09-17T22:39:37.386 INFO:teuthology.orchestra.run.smithi115.stdout:(2/6): ceph-mgr-cephadm-19.1.1-330.gc2ddf972.el 676 kB/s | 146 kB 00:00 2024-09-17T22:39:37.494 INFO:teuthology.orchestra.run.smithi115.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-09-17T22:39:37.520 INFO:teuthology.orchestra.run.smithi115.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 431 kB/s | 58 kB 00:00 2024-09-17T22:39:37.554 INFO:teuthology.orchestra.run.smithi115.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.3 MB/s | 548 kB 00:00 2024-09-17T22:39:37.587 INFO:teuthology.orchestra.run.smithi115.stdout:(6/6): python3-typing-extensions-4.12.2-2.el9.n 808 kB/s | 74 kB 00:00 2024-09-17T22:39:37.590 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:37.590 INFO:teuthology.orchestra.run.smithi115.stdout:Total 9.6 MB/s | 7.0 MB 00:00 2024-09-17T22:39:37.660 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-17T22:39:37.671 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-17T22:39:37.671 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-17T22:39:37.769 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-17T22:39:37.770 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-17T22:39:38.068 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T22:39:38.068 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 1/13 2024-09-17T22:39:38.068 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T22:39:38.068 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T22:39:38.068 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T22:39:38.068 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T22:39:38.068 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T22:39:38.068 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T22:39:38.069 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T22:39:38.069 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T22:39:38.069 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T22:39:38.069 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T22:39:38.069 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T22:39:38.079 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-17T22:39:38.253 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T22:39:38.305 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T22:39:38.452 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T22:39:38.452 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:38.452 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T22:39:38.452 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:39:38.453 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T22:39:38.453 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T22:39:38.453 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T22:39:38.453 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T22:39:38.453 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T22:39:38.453 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T22:39:38.454 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T22:39:38.454 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T22:39:38.454 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T22:39:38.454 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T22:39:38.454 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T22:39:38.455 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T22:39:38.455 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:38.455 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:38.719 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T22:39:38.823 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T22:39:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T22:39:38.980 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T22:39:39.009 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T22:39:39.301 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:24 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:39:39.435 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:39:39.435 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:39.435 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T22:39:39.436 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:39.436 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T22:39:39.436 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 146 k 2024-09-17T22:39:39.436 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T22:39:39.436 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T22:39:39.436 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T22:39:39.436 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T22:39:39.436 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T22:39:39.436 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T22:39:39.436 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:39.436 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T22:39:39.436 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:39.436 INFO:teuthology.orchestra.run.smithi129.stdout:Install 6 Packages 2024-09-17T22:39:39.437 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:39.437 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 7.0 M 2024-09-17T22:39:39.437 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 33 M 2024-09-17T22:39:39.437 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T22:39:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T22:39:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/6 2024-09-17T22:39:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T22:39:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T22:39:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T22:39:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T22:39:39.865 INFO:teuthology.orchestra.run.smithi129.stdout:(1/6): ceph-mgr-cephadm-19.1.1-330.gc2ddf972.el 668 kB/s | 146 kB 00:00 2024-09-17T22:39:39.906 INFO:teuthology.orchestra.run.smithi129.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 953 kB/s | 248 kB 00:00 2024-09-17T22:39:39.957 INFO:teuthology.orchestra.run.smithi129.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.2 MB/s | 58 kB 00:00 2024-09-17T22:39:40.015 INFO:teuthology.orchestra.run.smithi129.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 1.2 MB/s | 74 kB 00:00 2024-09-17T22:39:40.040 INFO:teuthology.orchestra.run.smithi129.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.0 MB/s | 548 kB 00:00 2024-09-17T22:39:40.090 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T22:39:40.090 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:40.090 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-17T22:39:40.090 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:39:40.090 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T22:39:40.090 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T22:39:40.090 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T22:39:40.090 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T22:39:40.090 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T22:39:40.091 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:40.091 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:40.342 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-fuse 2024-09-17T22:39:40.916 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:28 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:41.049 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:41.049 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:41.050 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-17T22:39:41.050 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:41.050 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-17T22:39:41.050 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 897 k 2024-09-17T22:39:41.050 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-17T22:39:41.050 INFO:teuthology.orchestra.run.smithi115.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T22:39:41.050 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:41.050 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-17T22:39:41.050 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:41.050 INFO:teuthology.orchestra.run.smithi115.stdout:Install 2 Packages 2024-09-17T22:39:41.050 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:41.051 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 977 k 2024-09-17T22:39:41.051 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 2.9 M 2024-09-17T22:39:41.051 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-17T22:39:41.572 INFO:teuthology.orchestra.run.smithi115.stdout:(1/2): ceph-fuse-19.1.1-330.gc2ddf972.el9.x86_6 3.3 MB/s | 897 kB 00:00 2024-09-17T22:39:41.857 INFO:teuthology.orchestra.run.smithi129.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.7 MB/s | 6.0 MB 00:02 2024-09-17T22:39:41.860 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:41.860 INFO:teuthology.orchestra.run.smithi129.stdout:Total 2.9 MB/s | 7.0 MB 00:02 2024-09-17T22:39:41.933 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T22:39:41.944 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T22:39:41.944 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T22:39:41.947 INFO:teuthology.orchestra.run.smithi115.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 125 kB/s | 80 kB 00:00 2024-09-17T22:39:41.948 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:41.949 INFO:teuthology.orchestra.run.smithi115.stdout:Total 1.1 MB/s | 977 kB 00:00 2024-09-17T22:39:41.962 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-17T22:39:41.985 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-17T22:39:41.985 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-17T22:39:42.045 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T22:39:42.046 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T22:39:42.068 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-17T22:39:42.069 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-17T22:39:42.251 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-17T22:39:42.356 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T22:39:42.393 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T22:39:42.463 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T22:39:42.513 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T22:39:42.575 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T22:39:43.105 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T22:39:43.126 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T22:39:43.126 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/2 2024-09-17T22:39:43.195 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T22:39:43.259 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T22:39:43.288 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T22:39:43.415 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T22:39:43.415 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:43.415 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-17T22:39:43.415 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T22:39:43.415 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:43.416 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:43.710 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-volume 2024-09-17T22:39:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T22:39:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/6 2024-09-17T22:39:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T22:39:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T22:39:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T22:39:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T22:39:44.307 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:32 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T22:39:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:44.353 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T22:39:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:39:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T22:39:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T22:39:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T22:39:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T22:39:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T22:39:44.354 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:44.354 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:44.443 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:44.443 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-17T22:39:44.444 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:44.444 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-17T22:39:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 285 k 2024-09-17T22:39:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:44.444 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-17T22:39:44.444 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:44.444 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-17T22:39:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:44.444 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 285 k 2024-09-17T22:39:44.444 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 1.3 M 2024-09-17T22:39:44.444 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-17T22:39:44.629 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-fuse 2024-09-17T22:39:44.777 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-volume-19.1.1-330.gc2ddf972.el9.noarch.rpm 857 kB/s | 285 kB 00:00 2024-09-17T22:39:44.777 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:44.778 INFO:teuthology.orchestra.run.smithi115.stdout:Total 854 kB/s | 285 kB 00:00 2024-09-17T22:39:44.778 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-17T22:39:44.786 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-17T22:39:44.787 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-17T22:39:44.811 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-17T22:39:44.812 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-17T22:39:45.036 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-17T22:39:45.088 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:39:45.104 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:39:45.104 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:39:45.104 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T22:39:45.104 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:45.207 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:30 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:39:45.340 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:39:45.341 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:45.341 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T22:39:45.341 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:45.341 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T22:39:45.341 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 897 k 2024-09-17T22:39:45.341 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-17T22:39:45.341 INFO:teuthology.orchestra.run.smithi129.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T22:39:45.341 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:45.341 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T22:39:45.341 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:45.342 INFO:teuthology.orchestra.run.smithi129.stdout:Install 2 Packages 2024-09-17T22:39:45.342 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:45.342 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 977 k 2024-09-17T22:39:45.342 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 2.9 M 2024-09-17T22:39:45.342 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T22:39:45.543 INFO:teuthology.orchestra.run.smithi129.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 521 kB/s | 80 kB 00:00 2024-09-17T22:39:45.718 INFO:teuthology.orchestra.run.smithi129.stdout:(2/2): ceph-fuse-19.1.1-330.gc2ddf972.el9.x86_6 2.7 MB/s | 897 kB 00:00 2024-09-17T22:39:45.719 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:45.719 INFO:teuthology.orchestra.run.smithi129.stdout:Total 2.5 MB/s | 977 kB 00:00 2024-09-17T22:39:45.731 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T22:39:45.753 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T22:39:45.753 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T22:39:45.835 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T22:39:45.835 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T22:39:46.022 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T22:39:46.161 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T22:39:46.196 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:39:46.197 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:46.197 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-17T22:39:46.197 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:39:46.197 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:46.197 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:46.226 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T22:39:46.490 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librados-devel 2024-09-17T22:39:46.946 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T22:39:46.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/2 2024-09-17T22:39:47.091 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:35 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:47.209 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T22:39:47.209 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:47.209 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T22:39:47.209 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T22:39:47.209 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:47.209 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:47.224 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:47.224 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:47.224 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-17T22:39:47.224 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:47.224 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-17T22:39:47.225 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 127 k 2024-09-17T22:39:47.225 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:47.225 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-17T22:39:47.225 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:47.225 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-17T22:39:47.225 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:47.225 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 127 k 2024-09-17T22:39:47.226 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 456 k 2024-09-17T22:39:47.226 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-17T22:39:47.424 INFO:teuthology.orchestra.run.smithi115.stdout:librados-devel-19.1.1-330.gc2ddf972.el9.x86_64. 639 kB/s | 127 kB 00:00 2024-09-17T22:39:47.425 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:47.425 INFO:teuthology.orchestra.run.smithi115.stdout:Total 634 kB/s | 127 kB 00:00 2024-09-17T22:39:47.425 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-17T22:39:47.430 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-17T22:39:47.430 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-17T22:39:47.494 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-17T22:39:47.495 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-17T22:39:47.501 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-volume 2024-09-17T22:39:47.649 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-17T22:39:47.755 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:39:48.109 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:33 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:39:48.180 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:39:48.247 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:39:48.248 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:48.248 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-17T22:39:48.248 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:48.248 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T22:39:48.248 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 285 k 2024-09-17T22:39:48.248 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:48.249 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T22:39:48.249 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:48.249 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T22:39:48.249 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:48.249 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 285 k 2024-09-17T22:39:48.249 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 1.3 M 2024-09-17T22:39:48.249 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T22:39:48.454 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:39:48.454 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:48.454 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-17T22:39:48.454 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:39:48.455 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:48.455 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:48.483 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-volume-19.1.1-330.gc2ddf972.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-17T22:39:48.483 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:48.484 INFO:teuthology.orchestra.run.smithi129.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-17T22:39:48.484 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T22:39:48.493 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T22:39:48.493 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T22:39:48.517 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T22:39:48.517 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T22:39:48.708 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install libcephfs2 2024-09-17T22:39:48.744 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T22:39:48.818 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:39:48.835 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:39:48.835 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:39:48.835 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T22:39:48.835 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:49.308 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:37 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:49.398 INFO:teuthology.orchestra.run.smithi115.stdout:Package libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:39:49.443 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:49.444 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-17T22:39:49.444 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:49.528 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install libcephfs-devel 2024-09-17T22:39:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T22:39:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:49.818 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T22:39:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T22:39:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:49.818 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:50.100 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:38 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:50.121 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install librados-devel 2024-09-17T22:39:50.230 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:50.231 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-17T22:39:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-17T22:39:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 32 k 2024-09-17T22:39:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-17T22:39:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:50.232 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-17T22:39:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:50.233 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 32 k 2024-09-17T22:39:50.233 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 142 k 2024-09-17T22:39:50.233 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-17T22:39:50.369 INFO:teuthology.orchestra.run.smithi115.stdout:libcephfs-devel-19.1.1-330.gc2ddf972.el9.x86_64 231 kB/s | 32 kB 00:00 2024-09-17T22:39:50.370 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:50.370 INFO:teuthology.orchestra.run.smithi115.stdout:Total 229 kB/s | 32 kB 00:00 2024-09-17T22:39:50.370 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-17T22:39:50.374 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-17T22:39:50.374 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-17T22:39:50.400 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-17T22:39:50.401 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-17T22:39:50.480 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-17T22:39:50.591 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:39:50.710 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:35 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:39:50.849 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:39:50.850 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:50.850 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-17T22:39:50.850 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:50.850 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T22:39:50.850 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 127 k 2024-09-17T22:39:50.850 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:50.850 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T22:39:50.850 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:50.850 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T22:39:50.850 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:50.850 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 127 k 2024-09-17T22:39:50.851 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 456 k 2024-09-17T22:39:50.851 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T22:39:50.970 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:39:51.056 INFO:teuthology.orchestra.run.smithi129.stdout:librados-devel-19.1.1-330.gc2ddf972.el9.x86_64. 616 kB/s | 127 kB 00:00 2024-09-17T22:39:51.056 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:51.057 INFO:teuthology.orchestra.run.smithi129.stdout:Total 612 kB/s | 127 kB 00:00 2024-09-17T22:39:51.057 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T22:39:51.062 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T22:39:51.063 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T22:39:51.125 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T22:39:51.125 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T22:39:51.194 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:39:51.194 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:51.194 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-17T22:39:51.194 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:39:51.194 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:51.194 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:51.278 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T22:39:51.379 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:39:51.390 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librados2 2024-09-17T22:39:51.792 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:39:51.967 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:39 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:52.055 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:39:52.055 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:52.056 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T22:39:52.056 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:39:52.056 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:52.056 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:52.056 INFO:teuthology.orchestra.run.smithi115.stdout:Package librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:39:52.100 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:52.101 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-17T22:39:52.101 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:52.170 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librbd1 2024-09-17T22:39:52.284 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install libcephfs2 2024-09-17T22:39:52.749 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:40 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:52.840 INFO:teuthology.orchestra.run.smithi115.stdout:Package librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:39:52.873 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:37 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:39:52.882 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:52.883 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-17T22:39:52.883 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:52.954 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rados 2024-09-17T22:39:52.963 INFO:teuthology.orchestra.run.smithi129.stdout:Package libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:39:53.007 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:39:53.008 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T22:39:53.008 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:53.068 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install libcephfs-devel 2024-09-17T22:39:53.533 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:41 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:53.620 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:39:53.649 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:38 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:39:53.664 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:53.664 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-17T22:39:53.665 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:53.736 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rgw 2024-09-17T22:39:53.782 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:39:53.782 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:53.783 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-17T22:39:53.783 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:53.783 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T22:39:53.783 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 32 k 2024-09-17T22:39:53.783 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:53.783 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T22:39:53.783 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:39:53.784 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T22:39:53.784 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:53.784 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 32 k 2024-09-17T22:39:53.784 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 142 k 2024-09-17T22:39:53.784 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T22:39:53.927 INFO:teuthology.orchestra.run.smithi129.stdout:libcephfs-devel-19.1.1-330.gc2ddf972.el9.x86_64 221 kB/s | 32 kB 00:00 2024-09-17T22:39:53.928 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:53.928 INFO:teuthology.orchestra.run.smithi129.stdout:Total 218 kB/s | 32 kB 00:00 2024-09-17T22:39:53.928 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T22:39:53.931 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T22:39:53.932 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T22:39:53.957 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T22:39:53.957 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T22:39:54.047 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T22:39:54.191 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:39:54.318 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:54.406 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:39:54.450 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:54.451 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-17T22:39:54.451 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:54.518 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-cephfs 2024-09-17T22:39:54.574 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:39:54.802 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:39:54.802 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:54.802 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T22:39:54.802 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:39:54.802 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:39:54.802 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:54.993 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install librados2 2024-09-17T22:39:55.099 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:43 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:55.188 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:39:55.231 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:55.232 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-17T22:39:55.232 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:55.302 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rbd 2024-09-17T22:39:55.575 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:40 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:39:55.665 INFO:teuthology.orchestra.run.smithi129.stdout:Package librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:39:55.708 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:39:55.709 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T22:39:55.709 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:55.771 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install librbd1 2024-09-17T22:39:55.890 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:43 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:55.979 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:39:56.022 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:56.023 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-17T22:39:56.023 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:56.100 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-fuse 2024-09-17T22:39:56.354 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:41 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:39:56.444 INFO:teuthology.orchestra.run.smithi129.stdout:Package librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:39:56.488 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:39:56.489 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T22:39:56.489 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:56.550 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-rados 2024-09-17T22:39:56.675 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:56.806 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:56.806 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-17T22:39:56.807 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:56.807 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-17T22:39:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 85 k 2024-09-17T22:39:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:56.807 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-17T22:39:56.807 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:56.807 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-17T22:39:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:56.807 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 85 k 2024-09-17T22:39:56.807 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 231 k 2024-09-17T22:39:56.807 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-17T22:39:57.005 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-fuse-19.1.1-330.gc2ddf972.el9.x86_64.rpm 432 kB/s | 85 kB 00:00 2024-09-17T22:39:57.006 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:57.006 INFO:teuthology.orchestra.run.smithi115.stdout:Total 429 kB/s | 85 kB 00:00 2024-09-17T22:39:57.006 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-17T22:39:57.013 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-17T22:39:57.013 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-17T22:39:57.077 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-17T22:39:57.077 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-17T22:39:57.141 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:39:57.229 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:39:57.238 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-17T22:39:57.272 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:39:57.273 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T22:39:57.273 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:57.333 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-rgw 2024-09-17T22:39:57.344 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:39:57.720 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:39:57.918 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:39:57.996 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:39:57.996 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:57.996 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-17T22:39:57.996 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:39:57.996 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:57.996 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:39:58.007 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:39:58.052 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:39:58.053 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T22:39:58.053 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:58.112 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-cephfs 2024-09-17T22:39:58.250 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-mirror 2024-09-17T22:39:58.694 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:43 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:39:58.783 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:39:58.826 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:39:58.827 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T22:39:58.827 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:58.855 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:46 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:39:58.886 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install python3-rbd 2024-09-17T22:39:58.987 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:39:58.987 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:58.988 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-17T22:39:58.988 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:58.988 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-17T22:39:58.988 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.1 M 2024-09-17T22:39:58.988 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:58.988 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-17T22:39:58.988 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:39:58.988 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-17T22:39:58.988 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:39:58.988 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 3.1 M 2024-09-17T22:39:58.988 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 13 M 2024-09-17T22:39:58.988 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-17T22:39:59.468 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:39:59.557 INFO:teuthology.orchestra.run.smithi129.stdout:Package python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T22:39:59.600 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:39:59.601 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-17T22:39:59.601 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:39:59.669 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install rbd-fuse 2024-09-17T22:39:59.951 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-mirror-19.1.1-330.gc2ddf972.el9.x86_64.rpm 3.2 MB/s | 3.1 MB 00:00 2024-09-17T22:39:59.951 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:39:59.952 INFO:teuthology.orchestra.run.smithi115.stdout:Total 3.2 MB/s | 3.1 MB 00:00 2024-09-17T22:39:59.952 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-17T22:39:59.961 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-17T22:39:59.961 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-17T22:40:00.043 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-17T22:40:00.044 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-17T22:40:00.247 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:40:00.382 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:40:00.383 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:40:00.383 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-09-17T22:40:00.383 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:40:00.383 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T22:40:00.383 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 85 k 2024-09-17T22:40:00.383 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:00.384 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T22:40:00.384 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:40:00.384 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T22:40:00.384 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:00.384 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 85 k 2024-09-17T22:40:00.384 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 231 k 2024-09-17T22:40:00.384 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T22:40:00.474 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-17T22:40:00.535 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:40:00.573 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:40:00.573 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:40:00.574 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T22:40:00.574 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T22:40:00.574 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T22:40:00.574 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:00.698 INFO:teuthology.orchestra.run.smithi129.stdout:rbd-fuse-19.1.1-330.gc2ddf972.el9.x86_64.rpm 272 kB/s | 85 kB 00:00 2024-09-17T22:40:00.699 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:40:00.699 INFO:teuthology.orchestra.run.smithi129.stdout:Total 271 kB/s | 85 kB 00:00 2024-09-17T22:40:00.699 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T22:40:00.706 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T22:40:00.706 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T22:40:00.770 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T22:40:00.770 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T22:40:00.919 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T22:40:01.014 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:40:01.463 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:40:01.643 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:40:01.643 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:01.644 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-17T22:40:01.644 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:40:01.644 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:01.644 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:40:01.725 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:40:01.725 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:01.725 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T22:40:01.725 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:40:01.726 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:01.726 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:40:01.881 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-nbd 2024-09-17T22:40:01.993 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install rbd-mirror 2024-09-17T22:40:02.478 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:50 ago on Tue 17 Sep 2024 10:38:12 PM UTC. 2024-09-17T22:40:02.592 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:40:02.610 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-17T22:40:02.611 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:40:02.612 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-17T22:40:02.612 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:40:02.612 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-17T22:40:02.612 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 170 k 2024-09-17T22:40:02.612 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:02.612 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-17T22:40:02.612 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-17T22:40:02.613 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-17T22:40:02.613 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:02.613 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 170 k 2024-09-17T22:40:02.613 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 494 k 2024-09-17T22:40:02.613 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-17T22:40:02.731 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:40:02.732 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:40:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repo Size 2024-09-17T22:40:02.732 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:40:02.732 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T22:40:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.1 M 2024-09-17T22:40:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:02.733 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T22:40:02.733 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:40:02.733 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T22:40:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:02.733 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 3.1 M 2024-09-17T22:40:02.733 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 13 M 2024-09-17T22:40:02.734 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T22:40:02.829 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-nbd-19.1.1-330.gc2ddf972.el9.x86_64.rpm 789 kB/s | 170 kB 00:00 2024-09-17T22:40:02.830 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:40:02.830 INFO:teuthology.orchestra.run.smithi115.stdout:Total 783 kB/s | 170 kB 00:00 2024-09-17T22:40:02.830 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-17T22:40:02.837 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-17T22:40:02.837 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-17T22:40:02.901 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-17T22:40:02.902 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-17T22:40:03.057 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-17T22:40:03.077 INFO:teuthology.orchestra.run.smithi129.stdout:rbd-mirror-19.1.1-330.gc2ddf972.el9.x86_64.rpm 9.0 MB/s | 3.1 MB 00:00 2024-09-17T22:40:03.078 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:40:03.078 INFO:teuthology.orchestra.run.smithi129.stdout:Total 9.0 MB/s | 3.1 MB 00:00 2024-09-17T22:40:03.078 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T22:40:03.087 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T22:40:03.087 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T22:40:03.168 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T22:40:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:40:03.168 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T22:40:03.596 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T22:40:03.635 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:40:03.661 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:40:03.687 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:40:03.688 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T22:40:03.688 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T22:40:03.688 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T22:40:03.688 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T22:40:03.688 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:03.927 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:40:03.927 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:03.927 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-17T22:40:03.927 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:40:03.928 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:03.928 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-17T22:40:04.183 DEBUG:teuthology.parallel:result is None 2024-09-17T22:40:04.832 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:40:04.832 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:04.832 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T22:40:04.832 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:40:04.832 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:04.832 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:40:05.087 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install rbd-nbd 2024-09-17T22:40:05.686 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:01:50 ago on Tue 17 Sep 2024 10:38:15 PM UTC. 2024-09-17T22:40:05.821 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-17T22:40:05.822 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:40:05.822 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-09-17T22:40:05.822 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:40:05.822 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-17T22:40:05.822 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 170 k 2024-09-17T22:40:05.822 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:05.822 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-17T22:40:05.822 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-17T22:40:05.822 INFO:teuthology.orchestra.run.smithi129.stdout:Install 1 Package 2024-09-17T22:40:05.822 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:05.822 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 170 k 2024-09-17T22:40:05.822 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 494 k 2024-09-17T22:40:05.822 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-17T22:40:06.007 INFO:teuthology.orchestra.run.smithi129.stdout:rbd-nbd-19.1.1-330.gc2ddf972.el9.x86_64.rpm 926 kB/s | 170 kB 00:00 2024-09-17T22:40:06.007 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-17T22:40:06.008 INFO:teuthology.orchestra.run.smithi129.stdout:Total 918 kB/s | 170 kB 00:00 2024-09-17T22:40:06.008 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-17T22:40:06.015 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-17T22:40:06.016 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-17T22:40:06.080 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-17T22:40:06.080 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-17T22:40:06.232 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-17T22:40:06.330 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:40:06.754 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:40:07.033 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T22:40:07.033 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:07.033 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-17T22:40:07.033 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T22:40:07.033 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:07.033 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-17T22:40:07.279 DEBUG:teuthology.parallel:result is None 2024-09-17T22:40:07.279 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T22:40:07.422 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T22:40:07.447 INFO:teuthology.orchestra.run.smithi115.stdout:19.1.1-330.gc2ddf972.el9 2024-09-17T22:40:07.448 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-330.gc2ddf972.el9 2024-09-17T22:40:07.448 INFO:teuthology.task.install:The correct ceph version 19.1.1-330.gc2ddf972 is installed. 2024-09-17T22:40:07.450 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T22:40:07.587 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T22:40:07.611 INFO:teuthology.orchestra.run.smithi129.stdout:19.1.1-330.gc2ddf972.el9 2024-09-17T22:40:07.611 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-330.gc2ddf972.el9 2024-09-17T22:40:07.611 INFO:teuthology.task.install:The correct ceph version 19.1.1-330.gc2ddf972 is installed. 2024-09-17T22:40:07.613 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T22:40:07.613 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:07.613 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T22:40:07.643 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T22:40:07.643 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T22:40:07.679 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T22:40:07.680 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:07.680 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T22:40:07.712 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T22:40:07.782 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T22:40:07.783 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T22:40:07.811 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T22:40:07.882 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T22:40:07.883 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:07.883 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T22:40:07.913 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T22:40:07.981 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T22:40:07.982 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T22:40:08.029 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T22:40:08.097 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T22:40:08.098 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:08.098 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T22:40:08.128 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T22:40:08.196 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T22:40:08.196 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T22:40:08.224 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T22:40:08.291 INFO:teuthology.run_tasks:Running task ceph... 2024-09-17T22:40:08.370 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-17T22:40:08.370 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /var/log/ceph 2024-09-17T22:40:08.373 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 777 /var/log/ceph 2024-09-17T22:40:08.400 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-17T22:40:08.400 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T22:40:08.440 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T22:40:08.466 INFO:tasks.ceph:Creating extra log directories... 2024-09-17T22:40:08.466 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T22:40:08.506 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T22:40:08.541 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-17T22:40:08.541 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-17T22:40:08.542 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909540', 'branch': 'squid', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7909540', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 1983, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'sleep_before_teardown': 0, 'subset': '79/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7', 'targets': {'smithi115.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPBVDNqFvFCuAcbrQjh38peCguFgt9WodlXcU5H72dyFVtmPEEbHl6aD2cwioUvD8JvgbOZY0zycgkqb2KnXsbA=', 'smithi129.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFWJOirshgZa9espocCz+4g3SQuJpfzo799io9cGRu5IJSf22RKXUEfd1oUMuoY8HPNPG9G/qHerQVZcQqxReAQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-17T22:40:08.542 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T22:40:08.572 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T22:40:08.588 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T22:40:08.628 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T22:40:08.654 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:08.654 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T22:40:08.707 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T22:40:08.707 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 2024-09-17T22:40:08.763 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T22:40:08.764 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T22:40:08.764 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-17T22:40:08.764 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T22:40:08.764 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T22:40:08.764 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-17 22:40:00.825394510 +0000 2024-09-17T22:40:08.764 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-17 22:37:33.995538185 +0000 2024-09-17T22:40:08.764 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-17 22:37:33.995538185 +0000 2024-09-17T22:40:08.764 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-17 22:37:33.995538185 +0000 2024-09-17T22:40:08.765 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T22:40:08.827 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-17T22:40:08.827 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-17T22:40:08.827 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000258354 s, 2.0 MB/s 2024-09-17T22:40:08.829 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T22:40:08.885 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 2024-09-17T22:40:08.939 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T22:40:08.940 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T22:40:08.940 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-17T22:40:08.940 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T22:40:08.940 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T22:40:08.940 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-17 22:40:00.825394510 +0000 2024-09-17T22:40:08.940 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-17 22:37:34.315531934 +0000 2024-09-17T22:40:08.940 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-17 22:37:34.315531934 +0000 2024-09-17T22:40:08.940 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-17 22:37:34.315531934 +0000 2024-09-17T22:40:08.940 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T22:40:09.003 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-17T22:40:09.003 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-17T22:40:09.003 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000271186 s, 1.9 MB/s 2024-09-17T22:40:09.004 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T22:40:09.059 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 2024-09-17T22:40:09.115 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T22:40:09.115 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T22:40:09.115 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-17T22:40:09.115 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T22:40:09.115 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T22:40:09.115 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-17 22:40:00.826394490 +0000 2024-09-17T22:40:09.115 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-17 22:37:34.656525273 +0000 2024-09-17T22:40:09.115 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-17 22:37:34.656525273 +0000 2024-09-17T22:40:09.115 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-17 22:37:34.656525273 +0000 2024-09-17T22:40:09.116 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T22:40:09.177 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-17T22:40:09.177 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-17T22:40:09.177 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000190396 s, 2.7 MB/s 2024-09-17T22:40:09.179 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T22:40:09.234 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 2024-09-17T22:40:09.289 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T22:40:09.290 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T22:40:09.290 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-17T22:40:09.290 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T22:40:09.290 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T22:40:09.290 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-17 22:40:00.826394490 +0000 2024-09-17T22:40:09.290 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-17 22:37:34.964519256 +0000 2024-09-17T22:40:09.290 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-17 22:37:34.964519256 +0000 2024-09-17T22:40:09.290 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-17 22:37:34.964519256 +0000 2024-09-17T22:40:09.290 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T22:40:09.352 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-17T22:40:09.352 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-17T22:40:09.352 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000182656 s, 2.8 MB/s 2024-09-17T22:40:09.354 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T22:40:09.409 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T22:40:09.410 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T22:40:09.410 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T22:40:09.428 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T22:40:09.428 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_1 2024-09-17T22:40:09.484 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T22:40:09.484 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T22:40:09.484 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-17T22:40:09.484 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T22:40:09.484 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T22:40:09.484 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-17 22:40:03.941956267 +0000 2024-09-17T22:40:09.485 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-17 22:37:31.395211542 +0000 2024-09-17T22:40:09.485 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-17 22:37:31.395211542 +0000 2024-09-17T22:40:09.485 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-17 22:37:31.395211542 +0000 2024-09-17T22:40:09.485 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T22:40:09.550 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-17T22:40:09.550 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-17T22:40:09.550 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000215246 s, 2.4 MB/s 2024-09-17T22:40:09.551 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T22:40:09.609 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_2 2024-09-17T22:40:09.665 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T22:40:09.666 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T22:40:09.666 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-17T22:40:09.666 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T22:40:09.666 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T22:40:09.666 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-17 22:40:03.942956245 +0000 2024-09-17T22:40:09.666 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-17 22:37:31.735204041 +0000 2024-09-17T22:40:09.666 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-17 22:37:31.735204041 +0000 2024-09-17T22:40:09.666 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-17 22:37:31.735204041 +0000 2024-09-17T22:40:09.666 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T22:40:09.730 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-17T22:40:09.730 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-17T22:40:09.730 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000241401 s, 2.1 MB/s 2024-09-17T22:40:09.731 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T22:40:09.788 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_3 2024-09-17T22:40:09.844 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T22:40:09.844 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T22:40:09.844 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-17T22:40:09.844 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T22:40:09.845 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T22:40:09.845 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-17 22:40:03.942956245 +0000 2024-09-17T22:40:09.845 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-17 22:37:32.067196716 +0000 2024-09-17T22:40:09.845 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-17 22:37:32.067196716 +0000 2024-09-17T22:40:09.845 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-17 22:37:32.067196716 +0000 2024-09-17T22:40:09.845 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T22:40:09.908 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-17T22:40:09.908 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-17T22:40:09.908 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000256823 s, 2.0 MB/s 2024-09-17T22:40:09.909 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T22:40:09.968 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_4 2024-09-17T22:40:10.023 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T22:40:10.023 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T22:40:10.024 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-17T22:40:10.024 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T22:40:10.024 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T22:40:10.024 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-17 22:40:03.942956245 +0000 2024-09-17T22:40:10.024 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-17 22:37:32.409189171 +0000 2024-09-17T22:40:10.024 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-17 22:37:32.409189171 +0000 2024-09-17T22:40:10.024 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-17 22:37:32.409189171 +0000 2024-09-17T22:40:10.024 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T22:40:10.086 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-17T22:40:10.086 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-17T22:40:10.086 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000233532 s, 2.2 MB/s 2024-09-17T22:40:10.088 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T22:40:10.144 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T22:40:10.144 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi115.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi129.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T22:40:10.144 INFO:tasks.ceph:Generating config... 2024-09-17T22:40:10.145 INFO:tasks.ceph:[client] rbd cache = False 2024-09-17T22:40:10.145 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-17T22:40:10.145 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-17T22:40:10.145 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-17T22:40:10.145 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-17T22:40:10.145 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-17T22:40:10.145 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-17T22:40:10.146 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-17T22:40:10.146 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-17T22:40:10.146 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-17T22:40:10.146 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-09-17T22:40:10.146 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-17T22:40:10.146 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-17T22:40:10.146 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-17T22:40:10.146 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-17T22:40:10.146 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-17T22:40:10.146 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-17T22:40:10.146 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-17T22:40:10.146 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-17T22:40:10.146 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-17T22:40:10.146 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-17T22:40:10.146 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-17T22:40:10.146 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-17T22:40:10.147 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-17T22:40:10.147 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-17T22:40:10.147 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-17T22:40:10.147 INFO:tasks.ceph:Setting up mon.a... 2024-09-17T22:40:10.147 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-17T22:40:10.210 INFO:teuthology.orchestra.run.smithi115.stdout:creating /etc/ceph/ceph.keyring 2024-09-17T22:40:10.213 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-17T22:40:10.257 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T22:40:10.323 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.115'), ('mon.c', '[v2:172.21.15.115:3301,v1:172.21.15.115:6790]'), ('mon.b', '172.21.15.129')] 2024-09-17T22:40:10.324 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.115,[v2:172.21.15.115:3301,v1:172.21.15.115:6790],172.21.15.129', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': False}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-17T22:40:10.324 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:10.325 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T22:40:10.379 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.115 --addv c '[v2:172.21.15.115:3301,v1:172.21.15.115:6790]' --add b 172.21.15.129 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:40:10.467 INFO:teuthology.orchestra.run.smithi115.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:40:10.467 INFO:teuthology.orchestra.run.smithi115.stdout:monmaptool: generated fsid 551c5ed8-e790-485a-bc95-a9578a29275c 2024-09-17T22:40:10.467 INFO:teuthology.orchestra.run.smithi115.stdout:setting min_mon_release = quincy 2024-09-17T22:40:10.467 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 0 2024-09-17T22:40:10.467 INFO:teuthology.orchestra.run.smithi115.stdout:fsid 551c5ed8-e790-485a-bc95-a9578a29275c 2024-09-17T22:40:10.467 INFO:teuthology.orchestra.run.smithi115.stdout:last_changed 2024-09-17T22:40:10.465164+0000 2024-09-17T22:40:10.467 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-09-17T22:40:10.465164+0000 2024-09-17T22:40:10.467 INFO:teuthology.orchestra.run.smithi115.stdout:min_mon_release 17 (quincy) 2024-09-17T22:40:10.467 INFO:teuthology.orchestra.run.smithi115.stdout:election_strategy: 1 2024-09-17T22:40:10.467 INFO:teuthology.orchestra.run.smithi115.stdout:0: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.a 2024-09-17T22:40:10.467 INFO:teuthology.orchestra.run.smithi115.stdout:1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-09-17T22:40:10.467 INFO:teuthology.orchestra.run.smithi115.stdout:2: [v2:172.21.15.115:3301/0,v1:172.21.15.115:6790/0] mon.c 2024-09-17T22:40:10.467 INFO:teuthology.orchestra.run.smithi115.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-17T22:40:10.468 DEBUG:teuthology.orchestra.run.smithi115:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T22:40:10.484 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 551c5ed8-e790-485a-bc95-a9578a29275c... 2024-09-17T22:40:10.484 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T22:40:10.526 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T22:40:10.565 INFO:teuthology.orchestra.run.smithi115.stdout:[global] 2024-09-17T22:40:10.566 INFO:teuthology.orchestra.run.smithi115.stdout: chdir = "" 2024-09-17T22:40:10.566 INFO:teuthology.orchestra.run.smithi115.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T22:40:10.566 INFO:teuthology.orchestra.run.smithi115.stdout: auth supported = cephx 2024-09-17T22:40:10.566 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.566 INFO:teuthology.orchestra.run.smithi115.stdout: filestore xattr use omap = true 2024-09-17T22:40:10.566 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.566 INFO:teuthology.orchestra.run.smithi115.stdout: mon clock drift allowed = 1.000 2024-09-17T22:40:10.566 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.566 INFO:teuthology.orchestra.run.smithi115.stdout: osd crush chooseleaf type = 0 2024-09-17T22:40:10.566 INFO:teuthology.orchestra.run.smithi115.stdout: auth debug = true 2024-09-17T22:40:10.566 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.566 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on old message = true 2024-09-17T22:40:10.567 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on bug = true 2024-09-17T22:40:10.567 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.567 INFO:teuthology.orchestra.run.smithi115.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T22:40:10.567 INFO:teuthology.orchestra.run.smithi115.stdout: mon pg warn max object skew = 0 2024-09-17T22:40:10.567 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.567 INFO:teuthology.orchestra.run.smithi115.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T22:40:10.567 INFO:teuthology.orchestra.run.smithi115.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T22:40:10.567 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.567 INFO:teuthology.orchestra.run.smithi115.stdout: osd pool default size = 2 2024-09-17T22:40:10.567 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.567 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd allow primary affinity = true 2024-09-17T22:40:10.567 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd allow pg remap = true 2024-09-17T22:40:10.567 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on legacy crush tunables = false 2024-09-17T22:40:10.567 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on crush straw calc version zero = false 2024-09-17T22:40:10.567 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on no sortbitwise = false 2024-09-17T22:40:10.568 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on osd down out interval zero = false 2024-09-17T22:40:10.568 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on too few osds = false 2024-09-17T22:40:10.568 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T22:40:10.568 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T22:40:10.568 INFO:teuthology.orchestra.run.smithi115.stdout: mon_allow_pool_size_one = true 2024-09-17T22:40:10.568 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.568 INFO:teuthology.orchestra.run.smithi115.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T22:40:10.568 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.568 INFO:teuthology.orchestra.run.smithi115.stdout: osd default data pool replay window = 5 2024-09-17T22:40:10.568 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.568 INFO:teuthology.orchestra.run.smithi115.stdout: mon allow pool delete = true 2024-09-17T22:40:10.568 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.568 INFO:teuthology.orchestra.run.smithi115.stdout: mon cluster log file level = debug 2024-09-17T22:40:10.568 INFO:teuthology.orchestra.run.smithi115.stdout: debug asserts on shutdown = true 2024-09-17T22:40:10.569 INFO:teuthology.orchestra.run.smithi115.stdout: mon health detail to clog = false 2024-09-17T22:40:10.569 INFO:teuthology.orchestra.run.smithi115.stdout: mon host = "172.21.15.115,[v2:172.21.15.115:3301,v1:172.21.15.115:6790],172.21.15.129" 2024-09-17T22:40:10.569 INFO:teuthology.orchestra.run.smithi115.stdout: mon client directed command retry = 5 2024-09-17T22:40:10.569 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on pool no app = False 2024-09-17T22:40:10.569 INFO:teuthology.orchestra.run.smithi115.stdout: ms inject socket failures = 5000 2024-09-17T22:40:10.569 INFO:teuthology.orchestra.run.smithi115.stdout: fsid = 551c5ed8-e790-485a-bc95-a9578a29275c 2024-09-17T22:40:10.569 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.569 INFO:teuthology.orchestra.run.smithi115.stdout:[osd] 2024-09-17T22:40:10.569 INFO:teuthology.orchestra.run.smithi115.stdout: osd journal size = 100 2024-09-17T22:40:10.569 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.569 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub load threshold = 5.0 2024-09-17T22:40:10.569 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub max interval = 600 2024-09-17T22:40:10.569 INFO:teuthology.orchestra.run.smithi115.stdout: osd mclock profile = high_recovery_ops 2024-09-17T22:40:10.570 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.570 INFO:teuthology.orchestra.run.smithi115.stdout: osd recover clone overlap = true 2024-09-17T22:40:10.570 INFO:teuthology.orchestra.run.smithi115.stdout: osd recovery max chunk = 1048576 2024-09-17T22:40:10.570 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.570 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug shutdown = true 2024-09-17T22:40:10.570 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug op order = true 2024-09-17T22:40:10.570 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug verify stray on activate = true 2024-09-17T22:40:10.570 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug trim objects = true 2024-09-17T22:40:10.570 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.570 INFO:teuthology.orchestra.run.smithi115.stdout: osd open classes on start = true 2024-09-17T22:40:10.570 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug pg log writeout = true 2024-09-17T22:40:10.570 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.570 INFO:teuthology.orchestra.run.smithi115.stdout: osd deep scrub update digest min age = 30 2024-09-17T22:40:10.571 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.571 INFO:teuthology.orchestra.run.smithi115.stdout: osd map max advance = 10 2024-09-17T22:40:10.571 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.571 INFO:teuthology.orchestra.run.smithi115.stdout: journal zero on create = true 2024-09-17T22:40:10.571 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.571 INFO:teuthology.orchestra.run.smithi115.stdout: filestore ondisk finisher threads = 3 2024-09-17T22:40:10.571 INFO:teuthology.orchestra.run.smithi115.stdout: filestore apply finisher threads = 3 2024-09-17T22:40:10.571 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.571 INFO:teuthology.orchestra.run.smithi115.stdout: bdev debug aio = true 2024-09-17T22:40:10.571 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug misdirected ops = true 2024-09-17T22:40:10.571 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore block size = 96636764160 2024-09-17T22:40:10.571 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore compression algorithm = zstd 2024-09-17T22:40:10.571 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore compression mode = aggressive 2024-09-17T22:40:10.571 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore fsck on mount = True 2024-09-17T22:40:10.571 INFO:teuthology.orchestra.run.smithi115.stdout: debug bluefs = 1/20 2024-09-17T22:40:10.572 INFO:teuthology.orchestra.run.smithi115.stdout: debug bluestore = 1/20 2024-09-17T22:40:10.572 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-09-17T22:40:10.572 INFO:teuthology.orchestra.run.smithi115.stdout: debug osd = 20 2024-09-17T22:40:10.572 INFO:teuthology.orchestra.run.smithi115.stdout: debug rocksdb = 4/10 2024-09-17T22:40:10.572 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T22:40:10.572 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd full ratio = 0.9 2024-09-17T22:40:10.572 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd nearfull ratio = 0.8 2024-09-17T22:40:10.572 INFO:teuthology.orchestra.run.smithi115.stdout: osd blocked scrub grace period = 3600 2024-09-17T22:40:10.572 INFO:teuthology.orchestra.run.smithi115.stdout: osd failsafe full ratio = 0.95 2024-09-17T22:40:10.572 INFO:teuthology.orchestra.run.smithi115.stdout: osd max markdown count = 1000 2024-09-17T22:40:10.572 INFO:teuthology.orchestra.run.smithi115.stdout: osd objectstore = bluestore 2024-09-17T22:40:10.572 INFO:teuthology.orchestra.run.smithi115.stdout: osd shutdown pgref assert = True 2024-09-17T22:40:10.572 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.572 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr] 2024-09-17T22:40:10.572 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-09-17T22:40:10.573 INFO:teuthology.orchestra.run.smithi115.stdout: debug mgr = 20 2024-09-17T22:40:10.573 INFO:teuthology.orchestra.run.smithi115.stdout: debug mon = 20 2024-09-17T22:40:10.573 INFO:teuthology.orchestra.run.smithi115.stdout: debug auth = 20 2024-09-17T22:40:10.573 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min pgs per osd = 4 2024-09-17T22:40:10.573 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min bytes per osd = 10 2024-09-17T22:40:10.573 INFO:teuthology.orchestra.run.smithi115.stdout: mgr/telemetry/nag = false 2024-09-17T22:40:10.573 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.573 INFO:teuthology.orchestra.run.smithi115.stdout:[mon] 2024-09-17T22:40:10.573 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-09-17T22:40:10.573 INFO:teuthology.orchestra.run.smithi115.stdout: debug mon = 20 2024-09-17T22:40:10.573 INFO:teuthology.orchestra.run.smithi115.stdout: debug paxos = 20 2024-09-17T22:40:10.573 INFO:teuthology.orchestra.run.smithi115.stdout: debug auth = 20 2024-09-17T22:40:10.573 INFO:teuthology.orchestra.run.smithi115.stdout: mon data avail warn = 5 2024-09-17T22:40:10.574 INFO:teuthology.orchestra.run.smithi115.stdout: mon mgr mkfs grace = 240 2024-09-17T22:40:10.574 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min pgs per osd = 4 2024-09-17T22:40:10.574 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd reporter subtree level = osd 2024-09-17T22:40:10.574 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd prime pg temp = true 2024-09-17T22:40:10.574 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min bytes per osd = 10 2024-09-17T22:40:10.574 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.574 INFO:teuthology.orchestra.run.smithi115.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T22:40:10.574 INFO:teuthology.orchestra.run.smithi115.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T22:40:10.574 INFO:teuthology.orchestra.run.smithi115.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T22:40:10.574 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.574 INFO:teuthology.orchestra.run.smithi115.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T22:40:10.574 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T22:40:10.574 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T22:40:10.574 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.574 INFO:teuthology.orchestra.run.smithi115.stdout: # 1m isn't quite enough 2024-09-17T22:40:10.575 INFO:teuthology.orchestra.run.smithi115.stdout: mon_down_mkfs_grace = 2m 2024-09-17T22:40:10.575 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.575 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_filestore_osds = false 2024-09-17T22:40:10.575 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:10.575 INFO:teuthology.orchestra.run.smithi115.stdout:[client] 2024-09-17T22:40:10.575 INFO:teuthology.orchestra.run.smithi115.stdout: rgw cache enabled = true 2024-09-17T22:40:10.575 INFO:teuthology.orchestra.run.smithi115.stdout: rgw enable ops log = true 2024-09-17T22:40:10.575 INFO:teuthology.orchestra.run.smithi115.stdout: rgw enable usage log = true 2024-09-17T22:40:10.575 INFO:teuthology.orchestra.run.smithi115.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T22:40:10.575 INFO:teuthology.orchestra.run.smithi115.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T22:40:10.575 INFO:teuthology.orchestra.run.smithi115.stdout: rbd cache = False 2024-09-17T22:40:10.575 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.a] 2024-09-17T22:40:10.575 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.c] 2024-09-17T22:40:10.576 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.b] 2024-09-17T22:40:10.576 INFO:teuthology.orchestra.run.smithi129.stdout:[global] 2024-09-17T22:40:10.576 INFO:teuthology.orchestra.run.smithi129.stdout: chdir = "" 2024-09-17T22:40:10.576 INFO:teuthology.orchestra.run.smithi129.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T22:40:10.576 INFO:teuthology.orchestra.run.smithi129.stdout: auth supported = cephx 2024-09-17T22:40:10.576 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.576 INFO:teuthology.orchestra.run.smithi129.stdout: filestore xattr use omap = true 2024-09-17T22:40:10.577 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.577 INFO:teuthology.orchestra.run.smithi129.stdout: mon clock drift allowed = 1.000 2024-09-17T22:40:10.577 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.577 INFO:teuthology.orchestra.run.smithi129.stdout: osd crush chooseleaf type = 0 2024-09-17T22:40:10.577 INFO:teuthology.orchestra.run.smithi129.stdout: auth debug = true 2024-09-17T22:40:10.577 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.577 INFO:teuthology.orchestra.run.smithi129.stdout: ms die on old message = true 2024-09-17T22:40:10.577 INFO:teuthology.orchestra.run.smithi129.stdout: ms die on bug = true 2024-09-17T22:40:10.577 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.577 INFO:teuthology.orchestra.run.smithi129.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T22:40:10.577 INFO:teuthology.orchestra.run.smithi129.stdout: mon pg warn max object skew = 0 2024-09-17T22:40:10.577 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.577 INFO:teuthology.orchestra.run.smithi129.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T22:40:10.578 INFO:teuthology.orchestra.run.smithi129.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T22:40:10.578 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.578 INFO:teuthology.orchestra.run.smithi129.stdout: osd pool default size = 2 2024-09-17T22:40:10.578 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.578 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd allow primary affinity = true 2024-09-17T22:40:10.578 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd allow pg remap = true 2024-09-17T22:40:10.578 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on legacy crush tunables = false 2024-09-17T22:40:10.578 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on crush straw calc version zero = false 2024-09-17T22:40:10.578 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on no sortbitwise = false 2024-09-17T22:40:10.578 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on osd down out interval zero = false 2024-09-17T22:40:10.578 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on too few osds = false 2024-09-17T22:40:10.578 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T22:40:10.578 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T22:40:10.578 INFO:teuthology.orchestra.run.smithi129.stdout: mon_allow_pool_size_one = true 2024-09-17T22:40:10.578 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.579 INFO:teuthology.orchestra.run.smithi129.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T22:40:10.579 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.579 INFO:teuthology.orchestra.run.smithi129.stdout: osd default data pool replay window = 5 2024-09-17T22:40:10.579 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.579 INFO:teuthology.orchestra.run.smithi129.stdout: mon allow pool delete = true 2024-09-17T22:40:10.579 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.579 INFO:teuthology.orchestra.run.smithi129.stdout: mon cluster log file level = debug 2024-09-17T22:40:10.579 INFO:teuthology.orchestra.run.smithi129.stdout: debug asserts on shutdown = true 2024-09-17T22:40:10.579 INFO:teuthology.orchestra.run.smithi129.stdout: mon health detail to clog = false 2024-09-17T22:40:10.579 INFO:teuthology.orchestra.run.smithi129.stdout: mon host = "172.21.15.115,[v2:172.21.15.115:3301,v1:172.21.15.115:6790],172.21.15.129" 2024-09-17T22:40:10.579 INFO:teuthology.orchestra.run.smithi129.stdout: mon client directed command retry = 5 2024-09-17T22:40:10.579 INFO:teuthology.orchestra.run.smithi129.stdout: mon warn on pool no app = False 2024-09-17T22:40:10.579 INFO:teuthology.orchestra.run.smithi129.stdout: ms inject socket failures = 5000 2024-09-17T22:40:10.579 INFO:teuthology.orchestra.run.smithi129.stdout: fsid = 551c5ed8-e790-485a-bc95-a9578a29275c 2024-09-17T22:40:10.579 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.580 INFO:teuthology.orchestra.run.smithi129.stdout:[osd] 2024-09-17T22:40:10.580 INFO:teuthology.orchestra.run.smithi129.stdout: osd journal size = 100 2024-09-17T22:40:10.580 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.580 INFO:teuthology.orchestra.run.smithi129.stdout: osd scrub load threshold = 5.0 2024-09-17T22:40:10.580 INFO:teuthology.orchestra.run.smithi129.stdout: osd scrub max interval = 600 2024-09-17T22:40:10.580 INFO:teuthology.orchestra.run.smithi129.stdout: osd mclock profile = high_recovery_ops 2024-09-17T22:40:10.580 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.580 INFO:teuthology.orchestra.run.smithi129.stdout: osd recover clone overlap = true 2024-09-17T22:40:10.580 INFO:teuthology.orchestra.run.smithi129.stdout: osd recovery max chunk = 1048576 2024-09-17T22:40:10.580 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.580 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug shutdown = true 2024-09-17T22:40:10.580 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug op order = true 2024-09-17T22:40:10.580 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug verify stray on activate = true 2024-09-17T22:40:10.580 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug trim objects = true 2024-09-17T22:40:10.581 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.581 INFO:teuthology.orchestra.run.smithi129.stdout: osd open classes on start = true 2024-09-17T22:40:10.581 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug pg log writeout = true 2024-09-17T22:40:10.581 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.581 INFO:teuthology.orchestra.run.smithi129.stdout: osd deep scrub update digest min age = 30 2024-09-17T22:40:10.581 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.581 INFO:teuthology.orchestra.run.smithi129.stdout: osd map max advance = 10 2024-09-17T22:40:10.581 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.581 INFO:teuthology.orchestra.run.smithi129.stdout: journal zero on create = true 2024-09-17T22:40:10.581 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.581 INFO:teuthology.orchestra.run.smithi129.stdout: filestore ondisk finisher threads = 3 2024-09-17T22:40:10.581 INFO:teuthology.orchestra.run.smithi129.stdout: filestore apply finisher threads = 3 2024-09-17T22:40:10.581 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.581 INFO:teuthology.orchestra.run.smithi129.stdout: bdev debug aio = true 2024-09-17T22:40:10.581 INFO:teuthology.orchestra.run.smithi129.stdout: osd debug misdirected ops = true 2024-09-17T22:40:10.582 INFO:teuthology.orchestra.run.smithi129.stdout: bluestore block size = 96636764160 2024-09-17T22:40:10.582 INFO:teuthology.orchestra.run.smithi129.stdout: bluestore compression algorithm = zstd 2024-09-17T22:40:10.582 INFO:teuthology.orchestra.run.smithi129.stdout: bluestore compression mode = aggressive 2024-09-17T22:40:10.582 INFO:teuthology.orchestra.run.smithi129.stdout: bluestore fsck on mount = True 2024-09-17T22:40:10.582 INFO:teuthology.orchestra.run.smithi129.stdout: debug bluefs = 1/20 2024-09-17T22:40:10.582 INFO:teuthology.orchestra.run.smithi129.stdout: debug bluestore = 1/20 2024-09-17T22:40:10.582 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-09-17T22:40:10.582 INFO:teuthology.orchestra.run.smithi129.stdout: debug osd = 20 2024-09-17T22:40:10.582 INFO:teuthology.orchestra.run.smithi129.stdout: debug rocksdb = 4/10 2024-09-17T22:40:10.582 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T22:40:10.582 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd full ratio = 0.9 2024-09-17T22:40:10.582 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd nearfull ratio = 0.8 2024-09-17T22:40:10.582 INFO:teuthology.orchestra.run.smithi129.stdout: osd blocked scrub grace period = 3600 2024-09-17T22:40:10.582 INFO:teuthology.orchestra.run.smithi129.stdout: osd failsafe full ratio = 0.95 2024-09-17T22:40:10.583 INFO:teuthology.orchestra.run.smithi129.stdout: osd max markdown count = 1000 2024-09-17T22:40:10.583 INFO:teuthology.orchestra.run.smithi129.stdout: osd objectstore = bluestore 2024-09-17T22:40:10.583 INFO:teuthology.orchestra.run.smithi129.stdout: osd shutdown pgref assert = True 2024-09-17T22:40:10.583 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.583 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr] 2024-09-17T22:40:10.583 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-09-17T22:40:10.583 INFO:teuthology.orchestra.run.smithi129.stdout: debug mgr = 20 2024-09-17T22:40:10.583 INFO:teuthology.orchestra.run.smithi129.stdout: debug mon = 20 2024-09-17T22:40:10.583 INFO:teuthology.orchestra.run.smithi129.stdout: debug auth = 20 2024-09-17T22:40:10.583 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min pgs per osd = 4 2024-09-17T22:40:10.583 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min bytes per osd = 10 2024-09-17T22:40:10.583 INFO:teuthology.orchestra.run.smithi129.stdout: mgr/telemetry/nag = false 2024-09-17T22:40:10.583 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.583 INFO:teuthology.orchestra.run.smithi129.stdout:[mon] 2024-09-17T22:40:10.583 INFO:teuthology.orchestra.run.smithi129.stdout: debug ms = 1 2024-09-17T22:40:10.584 INFO:teuthology.orchestra.run.smithi129.stdout: debug mon = 20 2024-09-17T22:40:10.584 INFO:teuthology.orchestra.run.smithi129.stdout: debug paxos = 20 2024-09-17T22:40:10.584 INFO:teuthology.orchestra.run.smithi129.stdout: debug auth = 20 2024-09-17T22:40:10.584 INFO:teuthology.orchestra.run.smithi129.stdout: mon data avail warn = 5 2024-09-17T22:40:10.584 INFO:teuthology.orchestra.run.smithi129.stdout: mon mgr mkfs grace = 240 2024-09-17T22:40:10.584 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min pgs per osd = 4 2024-09-17T22:40:10.584 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd reporter subtree level = osd 2024-09-17T22:40:10.584 INFO:teuthology.orchestra.run.smithi129.stdout: mon osd prime pg temp = true 2024-09-17T22:40:10.584 INFO:teuthology.orchestra.run.smithi129.stdout: mon reweight min bytes per osd = 10 2024-09-17T22:40:10.584 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.584 INFO:teuthology.orchestra.run.smithi129.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T22:40:10.584 INFO:teuthology.orchestra.run.smithi129.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T22:40:10.584 INFO:teuthology.orchestra.run.smithi129.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T22:40:10.584 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.584 INFO:teuthology.orchestra.run.smithi129.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T22:40:10.584 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T22:40:10.585 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T22:40:10.585 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.585 INFO:teuthology.orchestra.run.smithi129.stdout: # 1m isn't quite enough 2024-09-17T22:40:10.585 INFO:teuthology.orchestra.run.smithi129.stdout: mon_down_mkfs_grace = 2m 2024-09-17T22:40:10.585 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.585 INFO:teuthology.orchestra.run.smithi129.stdout: mon_warn_on_filestore_osds = false 2024-09-17T22:40:10.585 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-17T22:40:10.585 INFO:teuthology.orchestra.run.smithi129.stdout:[client] 2024-09-17T22:40:10.585 INFO:teuthology.orchestra.run.smithi129.stdout: rgw cache enabled = true 2024-09-17T22:40:10.585 INFO:teuthology.orchestra.run.smithi129.stdout: rgw enable ops log = true 2024-09-17T22:40:10.585 INFO:teuthology.orchestra.run.smithi129.stdout: rgw enable usage log = true 2024-09-17T22:40:10.585 INFO:teuthology.orchestra.run.smithi129.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T22:40:10.585 INFO:teuthology.orchestra.run.smithi129.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T22:40:10.585 INFO:teuthology.orchestra.run.smithi129.stdout: rbd cache = False 2024-09-17T22:40:10.586 INFO:teuthology.orchestra.run.smithi129.stdout:[mon.a] 2024-09-17T22:40:10.586 INFO:teuthology.orchestra.run.smithi129.stdout:[mon.c] 2024-09-17T22:40:10.586 INFO:teuthology.orchestra.run.smithi129.stdout:[mon.b] 2024-09-17T22:40:10.587 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-17T22:40:10.587 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-17T22:40:10.681 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-17T22:40:10.682 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:10.682 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-17T22:40:10.697 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:10.697 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-17T22:40:10.751 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi115.front.sepia.ceph.com 2024-09-17T22:40:10.751 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:10.752 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T22:40:10.752 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T22:40:10.826 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:10.826 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:40:10.881 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T22:40:10.882 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T22:40:10.882 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T22:40:10.882 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T22:40:10.917 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T22:40:10.917 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:40:10.973 INFO:tasks.ceph:Setting up mon nodes... 2024-09-17T22:40:10.973 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-17T22:40:10.973 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T22:40:11.023 INFO:teuthology.orchestra.run.smithi115.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T22:40:11.026 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T22:40:11.082 INFO:teuthology.orchestra.run.smithi129.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T22:40:11.085 INFO:tasks.ceph:Setting up mds nodes... 2024-09-17T22:40:11.085 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-17T22:40:11.085 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T22:40:11.126 INFO:teuthology.orchestra.run.smithi115.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-17T22:40:11.139 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-17T22:40:11.179 INFO:teuthology.orchestra.run.smithi129.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-17T22:40:11.193 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-17T22:40:11.193 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi115.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi129.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T22:40:11.193 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-17T22:40:11.217 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T22:40:11.217 INFO:tasks.ceph:role: osd.0 2024-09-17T22:40:11.217 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-17T22:40:11.218 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T22:40:11.289 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T22:40:11.290 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T22:40:11.290 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T22:40:11.290 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T22:40:11.290 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T22:40:11.290 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-09-17T22:40:11.290 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T22:40:11.290 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T22:40:11.290 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T22:40:11.290 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T22:40:11.812 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-09-17T22:40:11.814 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-09-17T22:40:11.814 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-17T22:40:11.948 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-17T22:40:11.981 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-17T22:40:12.047 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T22:40:12.047 INFO:tasks.ceph:role: osd.1 2024-09-17T22:40:12.047 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-17T22:40:12.047 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T22:40:12.118 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T22:40:12.118 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T22:40:12.118 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T22:40:12.118 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T22:40:12.118 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T22:40:12.118 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-09-17T22:40:12.119 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T22:40:12.119 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T22:40:12.119 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T22:40:12.119 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T22:40:12.564 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-09-17T22:40:12.566 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-09-17T22:40:12.566 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-17T22:40:12.651 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-17T22:40:12.679 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-17T22:40:12.744 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T22:40:12.744 INFO:tasks.ceph:role: osd.2 2024-09-17T22:40:12.745 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-17T22:40:12.745 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T22:40:12.816 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T22:40:12.816 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T22:40:12.816 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T22:40:12.816 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T22:40:12.816 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T22:40:12.817 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-09-17T22:40:12.817 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T22:40:12.817 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T22:40:12.817 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T22:40:12.817 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T22:40:13.307 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-09-17T22:40:13.309 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-09-17T22:40:13.310 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-17T22:40:13.447 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-17T22:40:13.475 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-17T22:40:13.540 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T22:40:13.540 INFO:tasks.ceph:role: osd.3 2024-09-17T22:40:13.540 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-17T22:40:13.540 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T22:40:13.612 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T22:40:13.613 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T22:40:13.613 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T22:40:13.613 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T22:40:13.613 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T22:40:13.613 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-09-17T22:40:13.613 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T22:40:13.613 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T22:40:13.613 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T22:40:13.613 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T22:40:14.117 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-09-17T22:40:14.120 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-09-17T22:40:14.120 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-17T22:40:14.260 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-17T22:40:14.292 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:40:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-17T22:40:14.413+0000 7fd220323640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-17T22:40:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-17T22:40:14.413+0000 7fd220323640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-17T22:40:14.416 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-17T22:40:14.413+0000 7fd220323640 -1 bdev(0x55897d8eb800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-17T22:40:14.417 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-17T22:40:14.413+0000 7fd220323640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-17T22:40:17.335 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-17T22:40:17.363 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:40:17.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-17T22:40:17.449+0000 7f3554334640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-17T22:40:17.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-17T22:40:17.450+0000 7f3554334640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-17T22:40:17.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-17T22:40:17.450+0000 7f3554334640 -1 bdev(0x5619af56f800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-17T22:40:17.453 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-17T22:40:17.450+0000 7f3554334640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-17T22:40:20.362 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-17T22:40:20.389 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:40:20.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-17T22:40:20.475+0000 7f9d11a0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-17T22:40:20.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-17T22:40:20.475+0000 7f9d11a0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-17T22:40:20.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-17T22:40:20.475+0000 7f9d11a0f640 -1 bdev(0x5584b69af800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-17T22:40:20.477 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-17T22:40:20.475+0000 7f9d11a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-17T22:40:23.396 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T22:40:23.421 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:40:23.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-17T22:40:23.505+0000 7f4ac542c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-17T22:40:23.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-17T22:40:23.505+0000 7f4ac542c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-17T22:40:23.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-17T22:40:23.505+0000 7f4ac542c640 -1 bdev(0x55867f5ab800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-17T22:40:23.508 INFO:teuthology.orchestra.run.smithi115.stderr:2024-09-17T22:40:23.505+0000 7f4ac542c640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-17T22:40:26.665 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-17T22:40:26.694 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-17T22:40:26.724 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T22:40:26.724 INFO:tasks.ceph:role: osd.4 2024-09-17T22:40:26.725 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T22:40:26.725 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T22:40:26.801 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T22:40:26.801 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T22:40:26.801 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T22:40:26.801 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T22:40:26.801 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T22:40:26.801 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-09-17T22:40:26.801 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T22:40:26.801 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T22:40:26.801 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T22:40:26.802 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T22:40:27.305 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-09-17T22:40:27.307 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-09-17T22:40:27.308 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-17T22:40:27.449 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-17T22:40:27.484 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-17T22:40:27.553 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T22:40:27.553 INFO:tasks.ceph:role: osd.5 2024-09-17T22:40:27.553 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T22:40:27.553 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T22:40:27.627 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T22:40:27.628 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T22:40:27.628 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T22:40:27.628 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T22:40:27.628 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T22:40:27.628 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-09-17T22:40:27.628 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T22:40:27.628 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T22:40:27.628 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T22:40:27.629 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T22:40:28.072 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-09-17T22:40:28.074 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-09-17T22:40:28.074 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-17T22:40:28.168 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-17T22:40:28.197 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-17T22:40:28.262 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T22:40:28.277 INFO:tasks.ceph:role: osd.6 2024-09-17T22:40:28.277 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T22:40:28.277 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T22:40:28.335 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T22:40:28.335 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T22:40:28.335 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T22:40:28.335 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T22:40:28.335 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T22:40:28.335 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-09-17T22:40:28.335 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T22:40:28.335 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T22:40:28.335 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T22:40:28.335 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T22:40:28.819 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-09-17T22:40:28.821 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-09-17T22:40:28.824 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-17T22:40:28.972 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-17T22:40:29.003 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-17T22:40:29.067 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T22:40:29.068 INFO:tasks.ceph:role: osd.7 2024-09-17T22:40:29.068 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T22:40:29.068 DEBUG:teuthology.orchestra.run.smithi129:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T22:40:29.139 INFO:teuthology.orchestra.run.smithi129.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T22:40:29.140 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T22:40:29.140 INFO:teuthology.orchestra.run.smithi129.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T22:40:29.140 INFO:teuthology.orchestra.run.smithi129.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T22:40:29.140 INFO:teuthology.orchestra.run.smithi129.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T22:40:29.140 INFO:teuthology.orchestra.run.smithi129.stdout: = sunit=0 swidth=0 blks 2024-09-17T22:40:29.140 INFO:teuthology.orchestra.run.smithi129.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T22:40:29.140 INFO:teuthology.orchestra.run.smithi129.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T22:40:29.140 INFO:teuthology.orchestra.run.smithi129.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T22:40:29.140 INFO:teuthology.orchestra.run.smithi129.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T22:40:29.659 INFO:teuthology.orchestra.run.smithi129.stdout:Discarding blocks...Done. 2024-09-17T22:40:29.661 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi129.front.sepia.ceph.com -o noatime 2024-09-17T22:40:29.661 DEBUG:teuthology.orchestra.run.smithi129:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-17T22:40:29.793 DEBUG:teuthology.orchestra.run.smithi129:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-17T22:40:29.824 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:40:29.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T22:40:29.916+0000 7f269841c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-17T22:40:29.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T22:40:29.916+0000 7f269841c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-17T22:40:29.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T22:40:29.916+0000 7f269841c640 -1 bdev(0x559ea3ebb800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-17T22:40:29.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T22:40:29.916+0000 7f269841c640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-17T22:40:33.068 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-17T22:40:33.096 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:40:33.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T22:40:33.182+0000 7f56b502e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-17T22:40:33.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T22:40:33.183+0000 7f56b502e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-17T22:40:33.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T22:40:33.183+0000 7f56b502e640 -1 bdev(0x557f5f9d3800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-17T22:40:33.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T22:40:33.183+0000 7f56b502e640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-17T22:40:36.348 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-17T22:40:36.378 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:40:36.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T22:40:36.466+0000 7fef5f7a2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-17T22:40:36.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T22:40:36.467+0000 7fef5f7a2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-17T22:40:36.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T22:40:36.467+0000 7fef5f7a2640 -1 bdev(0x556fda28b800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-17T22:40:36.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T22:40:36.467+0000 7fef5f7a2640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-17T22:40:39.626 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-17T22:40:39.652 DEBUG:teuthology.orchestra.run.smithi129:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:40:39.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T22:40:39.736+0000 7f5524eb6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-17T22:40:39.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T22:40:39.736+0000 7f5524eb6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-17T22:40:39.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T22:40:39.736+0000 7f5524eb6640 -1 bdev(0x561da3201800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-17T22:40:39.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-17T22:40:39.736+0000 7f5524eb6640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-17T22:40:42.916 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-17T22:40:42.946 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-17T22:40:42.946 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:42.946 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-17T22:40:42.977 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:42.977 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-17T22:40:43.045 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:43.045 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-17T22:40:43.112 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:43.112 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-17T22:40:43.180 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:43.180 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-17T22:40:43.247 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T22:40:43.247 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-17T22:40:43.276 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T22:40:43.276 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-17T22:40:43.345 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T22:40:43.345 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-17T22:40:43.414 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T22:40:43.414 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-17T22:40:43.481 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T22:40:43.481 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-17T22:40:43.547 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:43.547 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-17T22:40:43.564 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T22:40:43.564 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-17T22:40:43.603 INFO:tasks.ceph:Adding keys to all mons... 2024-09-17T22:40:43.604 DEBUG:teuthology.orchestra.run.smithi115:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T22:40:43.608 DEBUG:teuthology.orchestra.run.smithi129:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T22:40:43.646 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr.y] 2024-09-17T22:40:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBLBepmUUFKARAAgzvSmVi89Vp5C3mA5LOwiw== 2024-09-17T22:40:43.647 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.0] 2024-09-17T22:40:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBOBepmVH22GBAAvvHwtx8azmT1d6jZ9vWMCQ== 2024-09-17T22:40:43.647 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.1] 2024-09-17T22:40:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBRBepmderfGhAAaKysiX6njNud2nc/VbO3Cw== 2024-09-17T22:40:43.647 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.2] 2024-09-17T22:40:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBUBepmoBxWHBAA6WOazGPm7JAneog5xFWRYw== 2024-09-17T22:40:43.647 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.3] 2024-09-17T22:40:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBXBepm9YsqHhAA+siqShJnZi5WbmRQooQB3A== 2024-09-17T22:40:43.647 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr.x] 2024-09-17T22:40:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBLBepm3jHVBBAAgj3Wt4gfYKIndLkuus88Pg== 2024-09-17T22:40:43.647 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.4] 2024-09-17T22:40:43.647 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBdBepmh+qrNhAAHmpJb9yJL1I8VBZ7pWsl1A== 2024-09-17T22:40:43.647 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.5] 2024-09-17T22:40:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBhBepmA/n0ChAAsdhtXFesQ5UZpVfq7qNRsQ== 2024-09-17T22:40:43.648 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.6] 2024-09-17T22:40:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBkBepmcJ3gGxAAe9xRKZiDt0BmQvLz68gd6A== 2024-09-17T22:40:43.648 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.7] 2024-09-17T22:40:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBnBepmwhXwKxAAy/vTZe7xay+3AuKoi/J92w== 2024-09-17T22:40:43.648 INFO:teuthology.orchestra.run.smithi115.stdout:[client.0] 2024-09-17T22:40:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBLBepmTU9yBxAAroGi2Dau4IDBbyy7gGkpPg== 2024-09-17T22:40:43.648 INFO:teuthology.orchestra.run.smithi115.stdout:[client.1] 2024-09-17T22:40:43.648 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQBLBepmt/igChAAK+kJ9EL7xOLRQIGAK+vBDg== 2024-09-17T22:40:43.667 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr.y] 2024-09-17T22:40:43.668 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQBLBepmUUFKARAAgzvSmVi89Vp5C3mA5LOwiw== 2024-09-17T22:40:43.668 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.0] 2024-09-17T22:40:43.668 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQBOBepmVH22GBAAvvHwtx8azmT1d6jZ9vWMCQ== 2024-09-17T22:40:43.668 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.1] 2024-09-17T22:40:43.668 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQBRBepmderfGhAAaKysiX6njNud2nc/VbO3Cw== 2024-09-17T22:40:43.668 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.2] 2024-09-17T22:40:43.668 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQBUBepmoBxWHBAA6WOazGPm7JAneog5xFWRYw== 2024-09-17T22:40:43.668 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.3] 2024-09-17T22:40:43.668 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQBXBepm9YsqHhAA+siqShJnZi5WbmRQooQB3A== 2024-09-17T22:40:43.668 INFO:teuthology.orchestra.run.smithi129.stdout:[mgr.x] 2024-09-17T22:40:43.668 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQBLBepm3jHVBBAAgj3Wt4gfYKIndLkuus88Pg== 2024-09-17T22:40:43.669 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.4] 2024-09-17T22:40:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQBdBepmh+qrNhAAHmpJb9yJL1I8VBZ7pWsl1A== 2024-09-17T22:40:43.669 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.5] 2024-09-17T22:40:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQBhBepmA/n0ChAAsdhtXFesQ5UZpVfq7qNRsQ== 2024-09-17T22:40:43.669 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.6] 2024-09-17T22:40:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQBkBepmcJ3gGxAAe9xRKZiDt0BmQvLz68gd6A== 2024-09-17T22:40:43.669 INFO:teuthology.orchestra.run.smithi129.stdout:[osd.7] 2024-09-17T22:40:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQBnBepmwhXwKxAAy/vTZe7xay+3AuKoi/J92w== 2024-09-17T22:40:43.669 INFO:teuthology.orchestra.run.smithi129.stdout:[client.0] 2024-09-17T22:40:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQBLBepmTU9yBxAAroGi2Dau4IDBbyy7gGkpPg== 2024-09-17T22:40:43.669 INFO:teuthology.orchestra.run.smithi129.stdout:[client.1] 2024-09-17T22:40:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQBLBepmt/igChAAK+kJ9EL7xOLRQIGAK+vBDg== 2024-09-17T22:40:43.670 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T22:40:43.692 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T22:40:43.763 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:40:43.765 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:40:43.816 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:40:43.818 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:40:43.869 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:40:43.871 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:40:43.921 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:40:43.924 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:40:43.974 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T22:40:43.977 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T22:40:44.028 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:40:44.031 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:40:44.081 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:40:44.084 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:40:44.134 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:40:44.137 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:40:44.187 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:40:44.190 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T22:40:44.240 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T22:40:44.242 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T22:40:44.296 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T22:40:44.298 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T22:40:44.352 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-17T22:40:44.352 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-17T22:40:44.377 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T22:40:44.474 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-17T22:40:44.498 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-17T22:40:44.563 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T22:40:44.658 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-17T22:40:44.683 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-17T22:40:44.709 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T22:40:44.807 DEBUG:teuthology.orchestra.run.smithi129:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-17T22:40:44.831 DEBUG:teuthology.orchestra.run.smithi115:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:40:44.834 DEBUG:teuthology.orchestra.run.smithi129:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T22:40:44.886 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-17T22:40:44.887 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T22:40:44.887 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T22:40:44.889 INFO:tasks.ceph.mon.a:Started 2024-09-17T22:40:44.889 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T22:40:44.889 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T22:40:44.892 INFO:tasks.ceph.mon.c:Started 2024-09-17T22:40:44.892 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T22:40:44.892 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T22:40:44.929 INFO:tasks.ceph.mon.b:Started 2024-09-17T22:40:44.929 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-17T22:40:44.929 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-17T22:40:44.929 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-17T22:40:44.931 INFO:tasks.ceph.mgr.y:Started 2024-09-17T22:40:44.932 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-17T22:40:44.932 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-17T22:40:44.933 INFO:tasks.ceph.mgr.x:Started 2024-09-17T22:40:44.934 INFO:tasks.ceph:Setting crush tunables to default 2024-09-17T22:40:44.934 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd crush tunables default 2024-09-17T22:40:50.284 INFO:teuthology.orchestra.run.smithi115.stderr:adjusted tunables profile to default 2024-09-17T22:40:50.296 INFO:tasks.ceph:check_enable_crimson: False 2024-09-17T22:40:50.296 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-17T22:40:50.296 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:50.296 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-17T22:40:50.321 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:50.321 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-17T22:40:50.324 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:50.320+0000 7faf2505c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T22:40:50.365 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:50.362+0000 7faf2505c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T22:40:50.386 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:50.387 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-17T22:40:50.452 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-17T22:40:50.452 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-17T22:40:50.504 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:50.500+0000 7faf2505c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T22:40:50.510 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:50.508+0000 7fe86e525040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T22:40:50.516 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T22:40:50.516 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-17T22:40:50.542 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T22:40:50.542 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-17T22:40:50.553 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:50.550+0000 7fe86e525040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T22:40:50.558 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:50.554+0000 7faf2505c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T22:40:50.582 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T22:40:50.582 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-17T22:40:50.646 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-17T22:40:50.647 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-17T22:40:50.691 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:50.689+0000 7fe86e525040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T22:40:50.711 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new 3bd5c11f-e7c2-4968-839a-2799c00d93a6 0 2024-09-17T22:40:50.733 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:50.729+0000 7faf2505c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T22:40:50.742 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:50.739+0000 7fe86e525040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T22:40:50.849 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:50.846+0000 7faf2505c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T22:40:50.902 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:50.899+0000 7fe86e525040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T22:40:50.916 INFO:teuthology.orchestra.run.smithi129.stdout:0 2024-09-17T22:40:50.927 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new 02918203-cf2e-49fc-b994-743b5d81b4a1 1 2024-09-17T22:40:51.011 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:51.008+0000 7fe86e525040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T22:40:51.111 INFO:teuthology.orchestra.run.smithi129.stdout:1 2024-09-17T22:40:51.122 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new 22ee34ef-522e-4121-85b7-38bbafeca5de 2 2024-09-17T22:40:51.191 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:51.188+0000 7faf2505c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T22:40:51.243 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:51.240+0000 7faf2505c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T22:40:51.305 INFO:teuthology.orchestra.run.smithi129.stdout:2 2024-09-17T22:40:51.317 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new 5d464921-5696-43c5-b89d-430a54ddefb5 3 2024-09-17T22:40:51.374 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:51.371+0000 7fe86e525040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T22:40:51.399 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:51.396+0000 7faf2505c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T22:40:51.430 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:51.427+0000 7fe86e525040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T22:40:51.499 INFO:teuthology.orchestra.run.smithi129.stdout:3 2024-09-17T22:40:51.511 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new 4cb6973a-63ae-4f48-af94-37610aee3c06 4 2024-09-17T22:40:51.579 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:51.576+0000 7fe86e525040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T22:40:51.693 INFO:teuthology.orchestra.run.smithi129.stdout:4 2024-09-17T22:40:51.704 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new ef9fe11b-b719-48f4-9c6c-e1b1b355acdc 5 2024-09-17T22:40:51.885 INFO:teuthology.orchestra.run.smithi129.stdout:5 2024-09-17T22:40:51.897 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new e4045aec-22c1-401b-bfbf-64f115bc32af 6 2024-09-17T22:40:52.075 INFO:teuthology.orchestra.run.smithi129.stdout:6 2024-09-17T22:40:52.086 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --cluster ceph osd new 4a05a5d7-8281-4572-a9d0-6095ee5e168b 7 2024-09-17T22:40:52.267 INFO:teuthology.orchestra.run.smithi129.stdout:7 2024-09-17T22:40:52.278 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T22:40:52.278 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-17T22:40:52.281 INFO:tasks.ceph.osd.0:Started 2024-09-17T22:40:52.281 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T22:40:52.281 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T22:40:52.284 INFO:tasks.ceph.osd.1:Started 2024-09-17T22:40:52.284 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T22:40:52.284 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T22:40:52.286 INFO:tasks.ceph.osd.2:Started 2024-09-17T22:40:52.286 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-17T22:40:52.286 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-17T22:40:52.288 INFO:tasks.ceph.osd.3:Started 2024-09-17T22:40:52.288 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-17T22:40:52.289 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-17T22:40:52.291 INFO:tasks.ceph.osd.4:Started 2024-09-17T22:40:52.291 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-17T22:40:52.291 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-17T22:40:52.293 INFO:tasks.ceph.osd.5:Started 2024-09-17T22:40:52.293 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-17T22:40:52.293 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-17T22:40:52.295 INFO:tasks.ceph.osd.6:Started 2024-09-17T22:40:52.295 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-17T22:40:52.296 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-17T22:40:52.298 INFO:tasks.ceph.osd.7:Started 2024-09-17T22:40:52.298 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T22:40:52.417 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:52.414+0000 7faf2505c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T22:40:52.470 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:52.466+0000 7faf2505c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T22:40:52.547 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:40:52.548 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":10,"fsid":"551c5ed8-e790-485a-bc95-a9578a29275c","created":"2024-09-17T22:40:50.189230+0000","modified":"2024-09-17T22:40:52.260526+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T22:40:52.260526+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3bd5c11f-e7c2-4968-839a-2799c00d93a6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"02918203-cf2e-49fc-b994-743b5d81b4a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"22ee34ef-522e-4121-85b7-38bbafeca5de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5d464921-5696-43c5-b89d-430a54ddefb5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4cb6973a-63ae-4f48-af94-37610aee3c06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ef9fe11b-b719-48f4-9c6c-e1b1b355acdc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e4045aec-22c1-401b-bfbf-64f115bc32af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4a05a5d7-8281-4572-a9d0-6095ee5e168b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T22:40:52.559 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-17T22:40:52.559 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-17T22:40:52.626 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:52.623+0000 7fe86e525040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T22:40:52.680 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:52.677+0000 7fe86e525040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T22:40:52.834 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:52.831+0000 7faf2505c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T22:40:52.918 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:52.914+0000 7faf2505c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T22:40:53.026 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:53.023+0000 7faf2505c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T22:40:53.049 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:53.046+0000 7fe86e525040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T22:40:53.071 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:53.068+0000 7faf2505c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T22:40:53.133 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:53.130+0000 7fe86e525040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T22:40:53.183 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:53.179+0000 7faf2505c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T22:40:53.237 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:53.234+0000 7faf2505c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T22:40:53.241 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:53.238+0000 7fe86e525040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T22:40:53.284 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:53.280+0000 7faf2505c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T22:40:53.288 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:53.286+0000 7fe86e525040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T22:40:53.364 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T22:40:53.394 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:53.391+0000 7faf2505c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T22:40:53.403 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:53.400+0000 7fe86e525040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T22:40:53.443 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:53.439+0000 7faf2505c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T22:40:53.459 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:53.456+0000 7fe86e525040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T22:40:53.494 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:53.490+0000 7faf2505c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T22:40:53.505 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:53.503+0000 7fe86e525040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T22:40:53.519 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:40:53.516+0000 7f2540c17640 -1 Falling back to public interface 2024-09-17T22:40:53.523 INFO:teuthology.misc.health.smithi115.stdout: 2024-09-17T22:40:53.523 INFO:teuthology.misc.health.smithi115.stdout:{"epoch":10,"fsid":"551c5ed8-e790-485a-bc95-a9578a29275c","created":"2024-09-17T22:40:50.189230+0000","modified":"2024-09-17T22:40:52.260526+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T22:40:52.260526+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3bd5c11f-e7c2-4968-839a-2799c00d93a6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"02918203-cf2e-49fc-b994-743b5d81b4a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"22ee34ef-522e-4121-85b7-38bbafeca5de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5d464921-5696-43c5-b89d-430a54ddefb5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4cb6973a-63ae-4f48-af94-37610aee3c06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ef9fe11b-b719-48f4-9c6c-e1b1b355acdc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e4045aec-22c1-401b-bfbf-64f115bc32af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4a05a5d7-8281-4572-a9d0-6095ee5e168b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T22:40:53.533 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-17T22:40:53.545 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:40:53.542+0000 7fc5785b0640 -1 Falling back to public interface 2024-09-17T22:40:53.562 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:53.559+0000 7faf2505c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T22:40:53.606 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:53.602+0000 7faf2505c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T22:40:53.619 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:53.617+0000 7fe86e525040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T22:40:53.649 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:53.645+0000 7faf2505c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T22:40:53.669 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:53.666+0000 7fe86e525040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T22:40:53.719 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:53.717+0000 7fe86e525040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T22:40:53.772 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:40:53.768+0000 7f3eb0bb0640 -1 Falling back to public interface 2024-09-17T22:40:53.778 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:40:53.774+0000 7f797a20f640 -1 Falling back to public interface 2024-09-17T22:40:53.780 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:40:53.776+0000 7f2b042f1640 -1 Falling back to public interface 2024-09-17T22:40:53.787 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:53.784+0000 7fe86e525040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T22:40:53.831 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:53.828+0000 7fe86e525040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T22:40:53.874 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:53.872+0000 7fe86e525040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T22:40:54.017 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:40:54.014+0000 7fc37620c640 -1 Falling back to public interface 2024-09-17T22:40:54.029 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:40:54.025+0000 7f668b9b2640 -1 Falling back to public interface 2024-09-17T22:40:54.043 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:40:54.040+0000 7f372a40f640 -1 Falling back to public interface 2024-09-17T22:40:54.299 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:54.295+0000 7faf2505c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T22:40:54.535 INFO:tasks.ceph.mgr.y.smithi115.stderr:2024-09-17T22:40:54.532+0000 7fe86e525040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T22:40:55.313 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:55.310+0000 7faec5938640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T22:40:55.313 INFO:tasks.ceph.mgr.x.smithi129.stderr:2024-09-17T22:40:55.311+0000 7faec5938640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-17T22:40:57.612 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:40:57.609+0000 7f2b042f1640 -1 osd.6 0 log_to_monitors true 2024-09-17T22:40:57.616 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:40:57.613+0000 7f797a20f640 -1 osd.5 0 log_to_monitors true 2024-09-17T22:40:57.826 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:40:57.823+0000 7f2540c17640 -1 osd.3 0 log_to_monitors true 2024-09-17T22:40:57.837 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:40:57.835+0000 7f3eb0bb0640 -1 osd.4 0 log_to_monitors true 2024-09-17T22:40:57.885 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:40:57.882+0000 7fc5785b0640 -1 osd.2 0 log_to_monitors true 2024-09-17T22:40:58.024 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:40:58.021+0000 7f668b9b2640 -1 osd.7 0 log_to_monitors true 2024-09-17T22:40:58.032 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:40:58.029+0000 7fc37620c640 -1 osd.1 0 log_to_monitors true 2024-09-17T22:40:58.345 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:40:58.342+0000 7f372a40f640 -1 osd.0 0 log_to_monitors true 2024-09-17T22:41:00.280 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:00.277+0000 7f253c601640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T22:41:00.301 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:00.298+0000 7f2534421640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T22:41:00.303 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:00.300+0000 7fc574d53640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T22:41:00.318 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:00.315+0000 7fc371ac9640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T22:41:00.322 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:00.320+0000 7fc56c35f640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T22:41:00.327 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:00.325+0000 7fc3698e9640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T22:41:00.339 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T22:41:00.431 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:00.428+0000 7f3eacb3f640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T22:41:00.443 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:00.441+0000 7f3ea495f640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T22:41:00.452 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:00.450+0000 7f6687941640 -1 osd.7 0 waiting for initial osdmap 2024-09-17T22:41:00.463 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:00.461+0000 7f667f761640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T22:41:00.464 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:00.462+0000 7f2b00280640 -1 osd.6 0 waiting for initial osdmap 2024-09-17T22:41:00.476 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:00.474+0000 7f2af80a0640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T22:41:00.482 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:00.479+0000 7f7975b1e640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T22:41:00.496 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:00.494+0000 7f796d93e640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T22:41:00.694 INFO:teuthology.misc.health.smithi115.stdout: 2024-09-17T22:41:00.694 INFO:teuthology.misc.health.smithi115.stdout:{"epoch":13,"fsid":"551c5ed8-e790-485a-bc95-a9578a29275c","created":"2024-09-17T22:40:50.189230+0000","modified":"2024-09-17T22:41:00.322284+0000","last_up_change":"2024-09-17T22:41:00.322284+0000","last_in_change":"2024-09-17T22:40:52.260526+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3bd5c11f-e7c2-4968-839a-2799c00d93a6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"02918203-cf2e-49fc-b994-743b5d81b4a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"22ee34ef-522e-4121-85b7-38bbafeca5de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6809","nonce":689014698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6811","nonce":689014698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6815","nonce":689014698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6813","nonce":689014698}]},"public_addr":"172.21.15.115:6809/689014698","cluster_addr":"172.21.15.115:6811/689014698","heartbeat_back_addr":"172.21.15.115:6815/689014698","heartbeat_front_addr":"172.21.15.115:6813/689014698","state":["exists","up"]},{"osd":3,"uuid":"5d464921-5696-43c5-b89d-430a54ddefb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6801","nonce":453192890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6803","nonce":453192890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6807","nonce":453192890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6805","nonce":453192890}]},"public_addr":"172.21.15.115:6801/453192890","cluster_addr":"172.21.15.115:6803/453192890","heartbeat_back_addr":"172.21.15.115:6807/453192890","heartbeat_front_addr":"172.21.15.115:6805/453192890","state":["exists","up"]},{"osd":4,"uuid":"4cb6973a-63ae-4f48-af94-37610aee3c06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ef9fe11b-b719-48f4-9c6c-e1b1b355acdc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e4045aec-22c1-401b-bfbf-64f115bc32af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4a05a5d7-8281-4572-a9d0-6095ee5e168b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T22:41:00.708 DEBUG:teuthology.misc:2 of 8 OSDs are up 2024-09-17T22:41:01.069 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:01.066+0000 7f37265a6640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T22:41:01.079 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:01.077+0000 7f371dbb2640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T22:41:07.516 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T22:41:07.796 INFO:teuthology.misc.health.smithi115.stdout: 2024-09-17T22:41:07.796 INFO:teuthology.misc.health.smithi115.stdout:{"epoch":17,"fsid":"551c5ed8-e790-485a-bc95-a9578a29275c","created":"2024-09-17T22:40:50.189230+0000","modified":"2024-09-17T22:41:04.351092+0000","last_up_change":"2024-09-17T22:41:01.323817+0000","last_in_change":"2024-09-17T22:40:52.260526+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T22:41:02.338356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3bd5c11f-e7c2-4968-839a-2799c00d93a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2782271783},{"type":"v1","addr":"172.21.15.115:6825","nonce":2782271783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2782271783},{"type":"v1","addr":"172.21.15.115:6827","nonce":2782271783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2782271783},{"type":"v1","addr":"172.21.15.115:6831","nonce":2782271783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2782271783},{"type":"v1","addr":"172.21.15.115:6829","nonce":2782271783}]},"public_addr":"172.21.15.115:6825/2782271783","cluster_addr":"172.21.15.115:6827/2782271783","heartbeat_back_addr":"172.21.15.115:6831/2782271783","heartbeat_front_addr":"172.21.15.115:6829/2782271783","state":["exists","up"]},{"osd":1,"uuid":"02918203-cf2e-49fc-b994-743b5d81b4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6817","nonce":1941780390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6819","nonce":1941780390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6823","nonce":1941780390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6821","nonce":1941780390}]},"public_addr":"172.21.15.115:6817/1941780390","cluster_addr":"172.21.15.115:6819/1941780390","heartbeat_back_addr":"172.21.15.115:6823/1941780390","heartbeat_front_addr":"172.21.15.115:6821/1941780390","state":["exists","up"]},{"osd":2,"uuid":"22ee34ef-522e-4121-85b7-38bbafeca5de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6809","nonce":689014698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6811","nonce":689014698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6815","nonce":689014698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6813","nonce":689014698}]},"public_addr":"172.21.15.115:6809/689014698","cluster_addr":"172.21.15.115:6811/689014698","heartbeat_back_addr":"172.21.15.115:6815/689014698","heartbeat_front_addr":"172.21.15.115:6813/689014698","state":["exists","up"]},{"osd":3,"uuid":"5d464921-5696-43c5-b89d-430a54ddefb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6801","nonce":453192890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6803","nonce":453192890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6807","nonce":453192890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6805","nonce":453192890}]},"public_addr":"172.21.15.115:6801/453192890","cluster_addr":"172.21.15.115:6803/453192890","heartbeat_back_addr":"172.21.15.115:6807/453192890","heartbeat_front_addr":"172.21.15.115:6805/453192890","state":["exists","up"]},{"osd":4,"uuid":"4cb6973a-63ae-4f48-af94-37610aee3c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6801","nonce":380388860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6803","nonce":380388860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6807","nonce":380388860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6805","nonce":380388860}]},"public_addr":"172.21.15.129:6801/380388860","cluster_addr":"172.21.15.129:6803/380388860","heartbeat_back_addr":"172.21.15.129:6807/380388860","heartbeat_front_addr":"172.21.15.129:6805/380388860","state":["exists","up"]},{"osd":5,"uuid":"ef9fe11b-b719-48f4-9c6c-e1b1b355acdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6809","nonce":943488949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6811","nonce":943488949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6815","nonce":943488949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6813","nonce":943488949}]},"public_addr":"172.21.15.129:6809/943488949","cluster_addr":"172.21.15.129:6811/943488949","heartbeat_back_addr":"172.21.15.129:6815/943488949","heartbeat_front_addr":"172.21.15.129:6813/943488949","state":["exists","up"]},{"osd":6,"uuid":"e4045aec-22c1-401b-bfbf-64f115bc32af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6817","nonce":1873501077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6819","nonce":1873501077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6823","nonce":1873501077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6821","nonce":1873501077}]},"public_addr":"172.21.15.129:6817/1873501077","cluster_addr":"172.21.15.129:6819/1873501077","heartbeat_back_addr":"172.21.15.129:6823/1873501077","heartbeat_front_addr":"172.21.15.129:6821/1873501077","state":["exists","up"]},{"osd":7,"uuid":"4a05a5d7-8281-4572-a9d0-6095ee5e168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6825","nonce":3609357666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6827","nonce":3609357666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6831","nonce":3609357666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6829","nonce":3609357666}]},"public_addr":"172.21.15.129:6825/3609357666","cluster_addr":"172.21.15.129:6827/3609357666","heartbeat_back_addr":"172.21.15.129:6831/3609357666","heartbeat_front_addr":"172.21.15.129:6829/3609357666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:59.351758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:59.058915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.922383+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.802379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.881681+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.616649+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.589555+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:59.047783+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T22:41:07.807 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-17T22:41:07.807 INFO:tasks.ceph:Creating RBD pool 2024-09-17T22:41:07.807 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-17T22:41:08.384 INFO:teuthology.orchestra.run.smithi115.stderr:pool 'rbd' created 2024-09-17T22:41:08.397 DEBUG:teuthology.orchestra.run.smithi115:> rbd --cluster ceph pool init rbd 2024-09-17T22:41:11.408 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-17T22:41:11.408 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T22:41:11.409 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-17T22:41:11.409 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-17T22:41:11.409 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-17T22:41:11.747 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:41:11.758 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":5,"flags":0,"active_gid":4111,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2419928297},{"type":"v1","addr":"172.21.15.129:6833","nonce":2419928297}]},"active_addr":"172.21.15.129:6833/2419928297","active_change":"2024-09-17T22:40:54.299263+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":1808267799}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":2150849834}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":2220141390}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":7146198}]}]} 2024-09-17T22:41:11.763 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-17T22:41:11.763 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T22:41:11.763 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T22:41:12.040 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:41:12.040 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":21,"fsid":"551c5ed8-e790-485a-bc95-a9578a29275c","created":"2024-09-17T22:40:50.189230+0000","modified":"2024-09-17T22:41:11.391786+0000","last_up_change":"2024-09-17T22:41:01.323817+0000","last_in_change":"2024-09-17T22:40:52.260526+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T22:41:02.338356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T22:41:08.097861+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3bd5c11f-e7c2-4968-839a-2799c00d93a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2782271783},{"type":"v1","addr":"172.21.15.115:6825","nonce":2782271783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2782271783},{"type":"v1","addr":"172.21.15.115:6827","nonce":2782271783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2782271783},{"type":"v1","addr":"172.21.15.115:6831","nonce":2782271783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2782271783},{"type":"v1","addr":"172.21.15.115:6829","nonce":2782271783}]},"public_addr":"172.21.15.115:6825/2782271783","cluster_addr":"172.21.15.115:6827/2782271783","heartbeat_back_addr":"172.21.15.115:6831/2782271783","heartbeat_front_addr":"172.21.15.115:6829/2782271783","state":["exists","up"]},{"osd":1,"uuid":"02918203-cf2e-49fc-b994-743b5d81b4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6817","nonce":1941780390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6819","nonce":1941780390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6823","nonce":1941780390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6821","nonce":1941780390}]},"public_addr":"172.21.15.115:6817/1941780390","cluster_addr":"172.21.15.115:6819/1941780390","heartbeat_back_addr":"172.21.15.115:6823/1941780390","heartbeat_front_addr":"172.21.15.115:6821/1941780390","state":["exists","up"]},{"osd":2,"uuid":"22ee34ef-522e-4121-85b7-38bbafeca5de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6809","nonce":689014698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6811","nonce":689014698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6815","nonce":689014698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6813","nonce":689014698}]},"public_addr":"172.21.15.115:6809/689014698","cluster_addr":"172.21.15.115:6811/689014698","heartbeat_back_addr":"172.21.15.115:6815/689014698","heartbeat_front_addr":"172.21.15.115:6813/689014698","state":["exists","up"]},{"osd":3,"uuid":"5d464921-5696-43c5-b89d-430a54ddefb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6801","nonce":453192890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6803","nonce":453192890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6807","nonce":453192890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6805","nonce":453192890}]},"public_addr":"172.21.15.115:6801/453192890","cluster_addr":"172.21.15.115:6803/453192890","heartbeat_back_addr":"172.21.15.115:6807/453192890","heartbeat_front_addr":"172.21.15.115:6805/453192890","state":["exists","up"]},{"osd":4,"uuid":"4cb6973a-63ae-4f48-af94-37610aee3c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6801","nonce":380388860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6803","nonce":380388860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6807","nonce":380388860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6805","nonce":380388860}]},"public_addr":"172.21.15.129:6801/380388860","cluster_addr":"172.21.15.129:6803/380388860","heartbeat_back_addr":"172.21.15.129:6807/380388860","heartbeat_front_addr":"172.21.15.129:6805/380388860","state":["exists","up"]},{"osd":5,"uuid":"ef9fe11b-b719-48f4-9c6c-e1b1b355acdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6809","nonce":943488949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6811","nonce":943488949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6815","nonce":943488949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6813","nonce":943488949}]},"public_addr":"172.21.15.129:6809/943488949","cluster_addr":"172.21.15.129:6811/943488949","heartbeat_back_addr":"172.21.15.129:6815/943488949","heartbeat_front_addr":"172.21.15.129:6813/943488949","state":["exists","up"]},{"osd":6,"uuid":"e4045aec-22c1-401b-bfbf-64f115bc32af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6817","nonce":1873501077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6819","nonce":1873501077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6823","nonce":1873501077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6821","nonce":1873501077}]},"public_addr":"172.21.15.129:6817/1873501077","cluster_addr":"172.21.15.129:6819/1873501077","heartbeat_back_addr":"172.21.15.129:6823/1873501077","heartbeat_front_addr":"172.21.15.129:6821/1873501077","state":["exists","up"]},{"osd":7,"uuid":"4a05a5d7-8281-4572-a9d0-6095ee5e168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6825","nonce":3609357666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6827","nonce":3609357666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6831","nonce":3609357666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6829","nonce":3609357666}]},"public_addr":"172.21.15.129:6825/3609357666","cluster_addr":"172.21.15.129:6827/3609357666","heartbeat_back_addr":"172.21.15.129:6831/3609357666","heartbeat_front_addr":"172.21.15.129:6829/3609357666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:59.351758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:59.058915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.922383+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.802379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.881681+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.616649+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.589555+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:59.047783+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T22:41:12.053 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T22:41:12.053 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T22:41:12.332 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:41:12.332 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":21,"fsid":"551c5ed8-e790-485a-bc95-a9578a29275c","created":"2024-09-17T22:40:50.189230+0000","modified":"2024-09-17T22:41:11.391786+0000","last_up_change":"2024-09-17T22:41:01.323817+0000","last_in_change":"2024-09-17T22:40:52.260526+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T22:41:02.338356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T22:41:08.097861+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3bd5c11f-e7c2-4968-839a-2799c00d93a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2782271783},{"type":"v1","addr":"172.21.15.115:6825","nonce":2782271783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2782271783},{"type":"v1","addr":"172.21.15.115:6827","nonce":2782271783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2782271783},{"type":"v1","addr":"172.21.15.115:6831","nonce":2782271783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2782271783},{"type":"v1","addr":"172.21.15.115:6829","nonce":2782271783}]},"public_addr":"172.21.15.115:6825/2782271783","cluster_addr":"172.21.15.115:6827/2782271783","heartbeat_back_addr":"172.21.15.115:6831/2782271783","heartbeat_front_addr":"172.21.15.115:6829/2782271783","state":["exists","up"]},{"osd":1,"uuid":"02918203-cf2e-49fc-b994-743b5d81b4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6817","nonce":1941780390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6819","nonce":1941780390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6823","nonce":1941780390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6821","nonce":1941780390}]},"public_addr":"172.21.15.115:6817/1941780390","cluster_addr":"172.21.15.115:6819/1941780390","heartbeat_back_addr":"172.21.15.115:6823/1941780390","heartbeat_front_addr":"172.21.15.115:6821/1941780390","state":["exists","up"]},{"osd":2,"uuid":"22ee34ef-522e-4121-85b7-38bbafeca5de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6809","nonce":689014698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6811","nonce":689014698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6815","nonce":689014698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6813","nonce":689014698}]},"public_addr":"172.21.15.115:6809/689014698","cluster_addr":"172.21.15.115:6811/689014698","heartbeat_back_addr":"172.21.15.115:6815/689014698","heartbeat_front_addr":"172.21.15.115:6813/689014698","state":["exists","up"]},{"osd":3,"uuid":"5d464921-5696-43c5-b89d-430a54ddefb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6801","nonce":453192890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6803","nonce":453192890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6807","nonce":453192890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6805","nonce":453192890}]},"public_addr":"172.21.15.115:6801/453192890","cluster_addr":"172.21.15.115:6803/453192890","heartbeat_back_addr":"172.21.15.115:6807/453192890","heartbeat_front_addr":"172.21.15.115:6805/453192890","state":["exists","up"]},{"osd":4,"uuid":"4cb6973a-63ae-4f48-af94-37610aee3c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6801","nonce":380388860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6803","nonce":380388860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6807","nonce":380388860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6805","nonce":380388860}]},"public_addr":"172.21.15.129:6801/380388860","cluster_addr":"172.21.15.129:6803/380388860","heartbeat_back_addr":"172.21.15.129:6807/380388860","heartbeat_front_addr":"172.21.15.129:6805/380388860","state":["exists","up"]},{"osd":5,"uuid":"ef9fe11b-b719-48f4-9c6c-e1b1b355acdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6809","nonce":943488949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6811","nonce":943488949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6815","nonce":943488949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6813","nonce":943488949}]},"public_addr":"172.21.15.129:6809/943488949","cluster_addr":"172.21.15.129:6811/943488949","heartbeat_back_addr":"172.21.15.129:6815/943488949","heartbeat_front_addr":"172.21.15.129:6813/943488949","state":["exists","up"]},{"osd":6,"uuid":"e4045aec-22c1-401b-bfbf-64f115bc32af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6817","nonce":1873501077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6819","nonce":1873501077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6823","nonce":1873501077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6821","nonce":1873501077}]},"public_addr":"172.21.15.129:6817/1873501077","cluster_addr":"172.21.15.129:6819/1873501077","heartbeat_back_addr":"172.21.15.129:6823/1873501077","heartbeat_front_addr":"172.21.15.129:6821/1873501077","state":["exists","up"]},{"osd":7,"uuid":"4a05a5d7-8281-4572-a9d0-6095ee5e168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6825","nonce":3609357666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6827","nonce":3609357666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6831","nonce":3609357666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6829","nonce":3609357666}]},"public_addr":"172.21.15.129:6825/3609357666","cluster_addr":"172.21.15.129:6827/3609357666","heartbeat_back_addr":"172.21.15.129:6831/3609357666","heartbeat_front_addr":"172.21.15.129:6829/3609357666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:59.351758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:59.058915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.922383+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.802379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.881681+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.616649+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.589555+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:59.047783+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T22:41:12.346 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T22:41:12.346 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T22:41:12.347 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T22:41:12.347 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T22:41:12.347 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T22:41:12.348 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T22:41:12.348 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-17T22:41:12.349 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-17T22:41:12.578 INFO:teuthology.orchestra.run.smithi115.stdout:60129542150 2024-09-17T22:41:12.578 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T22:41:12.586 INFO:teuthology.orchestra.run.smithi115.stdout:55834574853 2024-09-17T22:41:12.586 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T22:41:12.593 INFO:teuthology.orchestra.run.smithi115.stdout:60129542148 2024-09-17T22:41:12.594 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T22:41:12.595 INFO:teuthology.orchestra.run.smithi115.stdout:60129542148 2024-09-17T22:41:12.595 INFO:teuthology.orchestra.run.smithi115.stdout:60129542148 2024-09-17T22:41:12.596 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T22:41:12.596 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T22:41:12.597 INFO:teuthology.orchestra.run.smithi115.stdout:55834574853 2024-09-17T22:41:12.597 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T22:41:12.598 INFO:teuthology.orchestra.run.smithi115.stdout:60129542148 2024-09-17T22:41:12.598 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T22:41:12.599 INFO:teuthology.orchestra.run.smithi115.stdout:60129542148 2024-09-17T22:41:12.600 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T22:41:13.117 INFO:teuthology.orchestra.run.smithi115.stdout:60129542148 2024-09-17T22:41:13.137 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.1 2024-09-17T22:41:13.143 INFO:teuthology.orchestra.run.smithi115.stdout:55834574852 2024-09-17T22:41:13.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2024-09-17T22:41:13.170 INFO:teuthology.orchestra.run.smithi115.stdout:60129542147 2024-09-17T22:41:13.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.4 2024-09-17T22:41:13.188 INFO:teuthology.orchestra.run.smithi115.stdout:60129542147 2024-09-17T22:41:13.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.7 2024-09-17T22:41:13.226 INFO:teuthology.orchestra.run.smithi115.stdout:60129542147 2024-09-17T22:41:13.230 INFO:teuthology.orchestra.run.smithi115.stdout:60129542147 2024-09-17T22:41:13.241 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.6 2024-09-17T22:41:13.241 INFO:teuthology.orchestra.run.smithi115.stdout:55834574852 2024-09-17T22:41:13.242 INFO:teuthology.orchestra.run.smithi115.stdout:60129542146 2024-09-17T22:41:13.245 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.5 2024-09-17T22:41:13.257 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2024-09-17T22:41:13.259 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542146 for osd.0 2024-09-17T22:41:14.138 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T22:41:14.162 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T22:41:14.185 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T22:41:14.202 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T22:41:14.242 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T22:41:14.246 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T22:41:14.258 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T22:41:14.259 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T22:41:14.638 INFO:teuthology.orchestra.run.smithi115.stdout:60129542150 2024-09-17T22:41:14.655 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.1 2024-09-17T22:41:14.656 DEBUG:teuthology.parallel:result is None 2024-09-17T22:41:14.674 INFO:teuthology.orchestra.run.smithi115.stdout:55834574853 2024-09-17T22:41:14.687 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2024-09-17T22:41:14.688 DEBUG:teuthology.parallel:result is None 2024-09-17T22:41:14.725 INFO:teuthology.orchestra.run.smithi115.stdout:60129542148 2024-09-17T22:41:14.738 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.4 2024-09-17T22:41:14.738 DEBUG:teuthology.parallel:result is None 2024-09-17T22:41:14.782 INFO:teuthology.orchestra.run.smithi115.stdout:60129542148 2024-09-17T22:41:14.793 INFO:teuthology.orchestra.run.smithi115.stdout:60129542148 2024-09-17T22:41:14.794 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.0 2024-09-17T22:41:14.795 DEBUG:teuthology.parallel:result is None 2024-09-17T22:41:14.810 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.7 2024-09-17T22:41:14.810 DEBUG:teuthology.parallel:result is None 2024-09-17T22:41:14.831 INFO:teuthology.orchestra.run.smithi115.stdout:60129542148 2024-09-17T22:41:14.837 INFO:teuthology.orchestra.run.smithi115.stdout:55834574853 2024-09-17T22:41:14.845 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.6 2024-09-17T22:41:14.845 DEBUG:teuthology.parallel:result is None 2024-09-17T22:41:14.846 INFO:teuthology.orchestra.run.smithi115.stdout:60129542148 2024-09-17T22:41:14.852 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-09-17T22:41:14.852 DEBUG:teuthology.parallel:result is None 2024-09-17T22:41:14.861 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.5 2024-09-17T22:41:14.861 DEBUG:teuthology.parallel:result is None 2024-09-17T22:41:14.861 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T22:41:14.861 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:41:15.176 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:41:15.176 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:41:15.188 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-17T22:41:14.972519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218056,"kb_used_data":1784,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756664,"statfs":{"total":773094113280,"available":772870823936,"internally_reserved":0,"allocated":1826816,"data_stored":1762782,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.589485"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.397570+0000","last_change":"2024-09-17T22:41:11.397814+0000","last_active":"2024-09-17T22:41:11.397570+0000","last_peered":"2024-09-17T22:41:11.397570+0000","last_clean":"2024-09-17T22:41:11.397570+0000","last_became_active":"2024-09-17T22:41:09.506775+0000","last_became_peered":"2024-09-17T22:41:09.506775+0000","last_unstale":"2024-09-17T22:41:11.397570+0000","last_undegraded":"2024-09-17T22:41:11.397570+0000","last_fullsized":"2024-09-17T22:41:11.397570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:47:43.867775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.395545+0000","last_change":"2024-09-17T22:41:11.395783+0000","last_active":"2024-09-17T22:41:11.395545+0000","last_peered":"2024-09-17T22:41:11.395545+0000","last_clean":"2024-09-17T22:41:11.395545+0000","last_became_active":"2024-09-17T22:41:09.507056+0000","last_became_peered":"2024-09-17T22:41:09.507056+0000","last_unstale":"2024-09-17T22:41:11.395545+0000","last_undegraded":"2024-09-17T22:41:11.395545+0000","last_fullsized":"2024-09-17T22:41:11.395545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:13.438783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.397506+0000","last_change":"2024-09-17T22:41:11.397753+0000","last_active":"2024-09-17T22:41:11.397506+0000","last_peered":"2024-09-17T22:41:11.397506+0000","last_clean":"2024-09-17T22:41:11.397506+0000","last_became_active":"2024-09-17T22:41:09.387651+0000","last_became_peered":"2024-09-17T22:41:09.387651+0000","last_unstale":"2024-09-17T22:41:11.397506+0000","last_undegraded":"2024-09-17T22:41:11.397506+0000","last_fullsized":"2024-09-17T22:41:11.397506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:14:30.915047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.510307+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:41:11.510307+0000","last_peered":"2024-09-17T22:41:11.510307+0000","last_clean":"2024-09-17T22:41:11.510307+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:41:11.510307+0000","last_undegraded":"2024-09-17T22:41:11.510307+0000","last_fullsized":"2024-09-17T22:41:11.510307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.397488+0000","last_change":"2024-09-17T22:41:11.397488+0000","last_active":"2024-09-17T22:41:11.397488+0000","last_peered":"2024-09-17T22:41:11.397488+0000","last_clean":"2024-09-17T22:41:11.397488+0000","last_became_active":"2024-09-17T22:41:09.651390+0000","last_became_peered":"2024-09-17T22:41:09.651390+0000","last_unstale":"2024-09-17T22:41:11.397488+0000","last_undegraded":"2024-09-17T22:41:11.397488+0000","last_fullsized":"2024-09-17T22:41:11.397488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:00.598921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.619512+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:41:11.619512+0000","last_peered":"2024-09-17T22:41:11.619512+0000","last_clean":"2024-09-17T22:41:11.619512+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:41:11.619512+0000","last_undegraded":"2024-09-17T22:41:11.619512+0000","last_fullsized":"2024-09-17T22:41:11.619512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.397171+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:41:11.397171+0000","last_peered":"2024-09-17T22:41:11.397171+0000","last_clean":"2024-09-17T22:41:11.397171+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:41:11.397171+0000","last_undegraded":"2024-09-17T22:41:11.397171+0000","last_fullsized":"2024-09-17T22:41:11.397171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.397821+0000","last_change":"2024-09-17T22:41:11.398108+0000","last_active":"2024-09-17T22:41:11.397821+0000","last_peered":"2024-09-17T22:41:11.397821+0000","last_clean":"2024-09-17T22:41:11.397821+0000","last_became_active":"2024-09-17T22:41:09.388184+0000","last_became_peered":"2024-09-17T22:41:09.388184+0000","last_unstale":"2024-09-17T22:41:11.397821+0000","last_undegraded":"2024-09-17T22:41:11.397821+0000","last_fullsized":"2024-09-17T22:41:11.397821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:43.506329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.398069+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:41:11.398069+0000","last_peered":"2024-09-17T22:41:11.398069+0000","last_clean":"2024-09-17T22:41:11.398069+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:41:11.398069+0000","last_undegraded":"2024-09-17T22:41:11.398069+0000","last_fullsized":"2024-09-17T22:41:11.398069+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":663119,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":72751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":72770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":663119,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":72751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":72770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":72751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:41:15.189 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:41:15.463 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:41:15.463 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:41:15.475 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-17T22:41:14.972519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218056,"kb_used_data":1784,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756664,"statfs":{"total":773094113280,"available":772870823936,"internally_reserved":0,"allocated":1826816,"data_stored":1762782,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.589485"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.397570+0000","last_change":"2024-09-17T22:41:11.397814+0000","last_active":"2024-09-17T22:41:11.397570+0000","last_peered":"2024-09-17T22:41:11.397570+0000","last_clean":"2024-09-17T22:41:11.397570+0000","last_became_active":"2024-09-17T22:41:09.506775+0000","last_became_peered":"2024-09-17T22:41:09.506775+0000","last_unstale":"2024-09-17T22:41:11.397570+0000","last_undegraded":"2024-09-17T22:41:11.397570+0000","last_fullsized":"2024-09-17T22:41:11.397570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:47:43.867775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.395545+0000","last_change":"2024-09-17T22:41:11.395783+0000","last_active":"2024-09-17T22:41:11.395545+0000","last_peered":"2024-09-17T22:41:11.395545+0000","last_clean":"2024-09-17T22:41:11.395545+0000","last_became_active":"2024-09-17T22:41:09.507056+0000","last_became_peered":"2024-09-17T22:41:09.507056+0000","last_unstale":"2024-09-17T22:41:11.395545+0000","last_undegraded":"2024-09-17T22:41:11.395545+0000","last_fullsized":"2024-09-17T22:41:11.395545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:13.438783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.397506+0000","last_change":"2024-09-17T22:41:11.397753+0000","last_active":"2024-09-17T22:41:11.397506+0000","last_peered":"2024-09-17T22:41:11.397506+0000","last_clean":"2024-09-17T22:41:11.397506+0000","last_became_active":"2024-09-17T22:41:09.387651+0000","last_became_peered":"2024-09-17T22:41:09.387651+0000","last_unstale":"2024-09-17T22:41:11.397506+0000","last_undegraded":"2024-09-17T22:41:11.397506+0000","last_fullsized":"2024-09-17T22:41:11.397506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:14:30.915047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.510307+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:41:11.510307+0000","last_peered":"2024-09-17T22:41:11.510307+0000","last_clean":"2024-09-17T22:41:11.510307+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:41:11.510307+0000","last_undegraded":"2024-09-17T22:41:11.510307+0000","last_fullsized":"2024-09-17T22:41:11.510307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.397488+0000","last_change":"2024-09-17T22:41:11.397488+0000","last_active":"2024-09-17T22:41:11.397488+0000","last_peered":"2024-09-17T22:41:11.397488+0000","last_clean":"2024-09-17T22:41:11.397488+0000","last_became_active":"2024-09-17T22:41:09.651390+0000","last_became_peered":"2024-09-17T22:41:09.651390+0000","last_unstale":"2024-09-17T22:41:11.397488+0000","last_undegraded":"2024-09-17T22:41:11.397488+0000","last_fullsized":"2024-09-17T22:41:11.397488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:00.598921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.619512+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:41:11.619512+0000","last_peered":"2024-09-17T22:41:11.619512+0000","last_clean":"2024-09-17T22:41:11.619512+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:41:11.619512+0000","last_undegraded":"2024-09-17T22:41:11.619512+0000","last_fullsized":"2024-09-17T22:41:11.619512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.397171+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:41:11.397171+0000","last_peered":"2024-09-17T22:41:11.397171+0000","last_clean":"2024-09-17T22:41:11.397171+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:41:11.397171+0000","last_undegraded":"2024-09-17T22:41:11.397171+0000","last_fullsized":"2024-09-17T22:41:11.397171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.397821+0000","last_change":"2024-09-17T22:41:11.398108+0000","last_active":"2024-09-17T22:41:11.397821+0000","last_peered":"2024-09-17T22:41:11.397821+0000","last_clean":"2024-09-17T22:41:11.397821+0000","last_became_active":"2024-09-17T22:41:09.388184+0000","last_became_peered":"2024-09-17T22:41:09.388184+0000","last_unstale":"2024-09-17T22:41:11.397821+0000","last_undegraded":"2024-09-17T22:41:11.397821+0000","last_fullsized":"2024-09-17T22:41:11.397821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:43.506329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.398069+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:41:11.398069+0000","last_peered":"2024-09-17T22:41:11.398069+0000","last_clean":"2024-09-17T22:41:11.398069+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:41:11.398069+0000","last_undegraded":"2024-09-17T22:41:11.398069+0000","last_fullsized":"2024-09-17T22:41:11.398069+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":663119,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":72751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":72770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":663119,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":72751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":72770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":72751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:41:15.476 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T22:41:15.476 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T22:41:15.476 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-17T22:41:15.476 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T22:41:15.784 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:41:15.784 INFO:teuthology.orchestra.run.smithi115.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T22:41:15.797 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-17T22:41:15.797 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-17T22:41:15.806 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-17T22:41:15.807 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7'}} 2024-09-17T22:41:15.807 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-17T22:41:15.807 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-17T22:41:16.101 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-17T22:41:16.101 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T22:41:16.101 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:41:16.376 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:41:16.377 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:41:16.389 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-17T22:41:16.308363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218060,"kb_used_data":1796,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756660,"statfs":{"total":773094113280,"available":772870819840,"internally_reserved":0,"allocated":1839104,"data_stored":1768248,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001223"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T22:41:14.973407+0000","last_change":"2024-09-17T22:41:11.397814+0000","last_active":"2024-09-17T22:41:14.973407+0000","last_peered":"2024-09-17T22:41:14.973407+0000","last_clean":"2024-09-17T22:41:14.973407+0000","last_became_active":"2024-09-17T22:41:09.506775+0000","last_became_peered":"2024-09-17T22:41:09.506775+0000","last_unstale":"2024-09-17T22:41:14.973407+0000","last_undegraded":"2024-09-17T22:41:14.973407+0000","last_fullsized":"2024-09-17T22:41:14.973407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:47:43.867775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.395545+0000","last_change":"2024-09-17T22:41:11.395783+0000","last_active":"2024-09-17T22:41:11.395545+0000","last_peered":"2024-09-17T22:41:11.395545+0000","last_clean":"2024-09-17T22:41:11.395545+0000","last_became_active":"2024-09-17T22:41:09.507056+0000","last_became_peered":"2024-09-17T22:41:09.507056+0000","last_unstale":"2024-09-17T22:41:11.395545+0000","last_undegraded":"2024-09-17T22:41:11.395545+0000","last_fullsized":"2024-09-17T22:41:11.395545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:13.438783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T22:41:14.973567+0000","last_change":"2024-09-17T22:41:11.397753+0000","last_active":"2024-09-17T22:41:14.973567+0000","last_peered":"2024-09-17T22:41:14.973567+0000","last_clean":"2024-09-17T22:41:14.973567+0000","last_became_active":"2024-09-17T22:41:09.387651+0000","last_became_peered":"2024-09-17T22:41:09.387651+0000","last_unstale":"2024-09-17T22:41:14.973567+0000","last_undegraded":"2024-09-17T22:41:14.973567+0000","last_fullsized":"2024-09-17T22:41:14.973567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:14:30.915047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.510307+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:41:11.510307+0000","last_peered":"2024-09-17T22:41:11.510307+0000","last_clean":"2024-09-17T22:41:11.510307+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:41:11.510307+0000","last_undegraded":"2024-09-17T22:41:11.510307+0000","last_fullsized":"2024-09-17T22:41:11.510307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T22:41:14.973397+0000","last_change":"2024-09-17T22:41:11.397488+0000","last_active":"2024-09-17T22:41:14.973397+0000","last_peered":"2024-09-17T22:41:14.973397+0000","last_clean":"2024-09-17T22:41:14.973397+0000","last_became_active":"2024-09-17T22:41:09.651390+0000","last_became_peered":"2024-09-17T22:41:09.651390+0000","last_unstale":"2024-09-17T22:41:14.973397+0000","last_undegraded":"2024-09-17T22:41:14.973397+0000","last_fullsized":"2024-09-17T22:41:14.973397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:00.598921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.619512+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:41:11.619512+0000","last_peered":"2024-09-17T22:41:11.619512+0000","last_clean":"2024-09-17T22:41:11.619512+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:41:11.619512+0000","last_undegraded":"2024-09-17T22:41:11.619512+0000","last_fullsized":"2024-09-17T22:41:11.619512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T22:41:14.973392+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:41:14.973392+0000","last_peered":"2024-09-17T22:41:14.973392+0000","last_clean":"2024-09-17T22:41:14.973392+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:41:14.973392+0000","last_undegraded":"2024-09-17T22:41:14.973392+0000","last_fullsized":"2024-09-17T22:41:14.973392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T22:41:14.973392+0000","last_change":"2024-09-17T22:41:11.398108+0000","last_active":"2024-09-17T22:41:14.973392+0000","last_peered":"2024-09-17T22:41:14.973392+0000","last_clean":"2024-09-17T22:41:14.973392+0000","last_became_active":"2024-09-17T22:41:09.388184+0000","last_became_peered":"2024-09-17T22:41:09.388184+0000","last_unstale":"2024-09-17T22:41:14.973392+0000","last_undegraded":"2024-09-17T22:41:14.973392+0000","last_fullsized":"2024-09-17T22:41:14.973392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:43.506329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.398069+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:41:11.398069+0000","last_peered":"2024-09-17T22:41:11.398069+0000","last_clean":"2024-09-17T22:41:11.398069+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:41:11.398069+0000","last_undegraded":"2024-09-17T22:41:11.398069+0000","last_fullsized":"2024-09-17T22:41:11.398069+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":663119,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":72751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":72770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":663119,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":72751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":78236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":72751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:41:16.389 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:41:16.665 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:41:16.665 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:41:16.677 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-17T22:41:16.308363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218060,"kb_used_data":1796,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756660,"statfs":{"total":773094113280,"available":772870819840,"internally_reserved":0,"allocated":1839104,"data_stored":1768248,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001223"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T22:41:14.973407+0000","last_change":"2024-09-17T22:41:11.397814+0000","last_active":"2024-09-17T22:41:14.973407+0000","last_peered":"2024-09-17T22:41:14.973407+0000","last_clean":"2024-09-17T22:41:14.973407+0000","last_became_active":"2024-09-17T22:41:09.506775+0000","last_became_peered":"2024-09-17T22:41:09.506775+0000","last_unstale":"2024-09-17T22:41:14.973407+0000","last_undegraded":"2024-09-17T22:41:14.973407+0000","last_fullsized":"2024-09-17T22:41:14.973407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:47:43.867775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.395545+0000","last_change":"2024-09-17T22:41:11.395783+0000","last_active":"2024-09-17T22:41:11.395545+0000","last_peered":"2024-09-17T22:41:11.395545+0000","last_clean":"2024-09-17T22:41:11.395545+0000","last_became_active":"2024-09-17T22:41:09.507056+0000","last_became_peered":"2024-09-17T22:41:09.507056+0000","last_unstale":"2024-09-17T22:41:11.395545+0000","last_undegraded":"2024-09-17T22:41:11.395545+0000","last_fullsized":"2024-09-17T22:41:11.395545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:13.438783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T22:41:14.973567+0000","last_change":"2024-09-17T22:41:11.397753+0000","last_active":"2024-09-17T22:41:14.973567+0000","last_peered":"2024-09-17T22:41:14.973567+0000","last_clean":"2024-09-17T22:41:14.973567+0000","last_became_active":"2024-09-17T22:41:09.387651+0000","last_became_peered":"2024-09-17T22:41:09.387651+0000","last_unstale":"2024-09-17T22:41:14.973567+0000","last_undegraded":"2024-09-17T22:41:14.973567+0000","last_fullsized":"2024-09-17T22:41:14.973567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:14:30.915047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.510307+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:41:11.510307+0000","last_peered":"2024-09-17T22:41:11.510307+0000","last_clean":"2024-09-17T22:41:11.510307+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:41:11.510307+0000","last_undegraded":"2024-09-17T22:41:11.510307+0000","last_fullsized":"2024-09-17T22:41:11.510307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T22:41:14.973397+0000","last_change":"2024-09-17T22:41:11.397488+0000","last_active":"2024-09-17T22:41:14.973397+0000","last_peered":"2024-09-17T22:41:14.973397+0000","last_clean":"2024-09-17T22:41:14.973397+0000","last_became_active":"2024-09-17T22:41:09.651390+0000","last_became_peered":"2024-09-17T22:41:09.651390+0000","last_unstale":"2024-09-17T22:41:14.973397+0000","last_undegraded":"2024-09-17T22:41:14.973397+0000","last_fullsized":"2024-09-17T22:41:14.973397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:00.598921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.619512+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:41:11.619512+0000","last_peered":"2024-09-17T22:41:11.619512+0000","last_clean":"2024-09-17T22:41:11.619512+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:41:11.619512+0000","last_undegraded":"2024-09-17T22:41:11.619512+0000","last_fullsized":"2024-09-17T22:41:11.619512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T22:41:14.973392+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:41:14.973392+0000","last_peered":"2024-09-17T22:41:14.973392+0000","last_clean":"2024-09-17T22:41:14.973392+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:41:14.973392+0000","last_undegraded":"2024-09-17T22:41:14.973392+0000","last_fullsized":"2024-09-17T22:41:14.973392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T22:41:14.973392+0000","last_change":"2024-09-17T22:41:11.398108+0000","last_active":"2024-09-17T22:41:14.973392+0000","last_peered":"2024-09-17T22:41:14.973392+0000","last_clean":"2024-09-17T22:41:14.973392+0000","last_became_active":"2024-09-17T22:41:09.388184+0000","last_became_peered":"2024-09-17T22:41:09.388184+0000","last_unstale":"2024-09-17T22:41:14.973392+0000","last_undegraded":"2024-09-17T22:41:14.973392+0000","last_fullsized":"2024-09-17T22:41:14.973392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:43.506329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-17T22:41:11.398069+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:41:11.398069+0000","last_peered":"2024-09-17T22:41:11.398069+0000","last_clean":"2024-09-17T22:41:11.398069+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:41:11.398069+0000","last_undegraded":"2024-09-17T22:41:11.398069+0000","last_fullsized":"2024-09-17T22:41:11.398069+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":663119,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":72751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":72770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":663119,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":72751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":78236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":212992,"data_stored":72751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:41:16.678 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T22:41:16.678 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-17T22:41:16.951 INFO:teuthology.orchestra.run.smithi115.stdout:epoch 22 2024-09-17T22:41:16.952 INFO:teuthology.orchestra.run.smithi115.stdout:fsid 551c5ed8-e790-485a-bc95-a9578a29275c 2024-09-17T22:41:16.952 INFO:teuthology.orchestra.run.smithi115.stdout:created 2024-09-17T22:40:50.189230+0000 2024-09-17T22:41:16.952 INFO:teuthology.orchestra.run.smithi115.stdout:modified 2024-09-17T22:41:14.970673+0000 2024-09-17T22:41:16.952 INFO:teuthology.orchestra.run.smithi115.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-17T22:41:16.952 INFO:teuthology.orchestra.run.smithi115.stdout:crush_version 5 2024-09-17T22:41:16.952 INFO:teuthology.orchestra.run.smithi115.stdout:full_ratio 0.95 2024-09-17T22:41:16.952 INFO:teuthology.orchestra.run.smithi115.stdout:backfillfull_ratio 0.9 2024-09-17T22:41:16.952 INFO:teuthology.orchestra.run.smithi115.stdout:nearfull_ratio 0.85 2024-09-17T22:41:16.952 INFO:teuthology.orchestra.run.smithi115.stdout:require_min_compat_client luminous 2024-09-17T22:41:16.952 INFO:teuthology.orchestra.run.smithi115.stdout:min_compat_client jewel 2024-09-17T22:41:16.952 INFO:teuthology.orchestra.run.smithi115.stdout:require_osd_release squid 2024-09-17T22:41:16.952 INFO:teuthology.orchestra.run.smithi115.stdout:stretch_mode_enabled false 2024-09-17T22:41:16.952 INFO:teuthology.orchestra.run.smithi115.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-17T22:41:16.953 INFO:teuthology.orchestra.run.smithi115.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 21 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 4.98 2024-09-17T22:41:16.953 INFO:teuthology.orchestra.run.smithi115.stdout:max_osd 8 2024-09-17T22:41:16.953 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6824/2782271783,v1:172.21.15.115:6825/2782271783] [v2:172.21.15.115:6826/2782271783,v1:172.21.15.115:6827/2782271783] exists,up 3bd5c11f-e7c2-4968-839a-2799c00d93a6 2024-09-17T22:41:16.953 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1 up in weight 1 up_from 14 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6816/1941780390,v1:172.21.15.115:6817/1941780390] [v2:172.21.15.115:6818/1941780390,v1:172.21.15.115:6819/1941780390] exists,up 02918203-cf2e-49fc-b994-743b5d81b4a1 2024-09-17T22:41:16.953 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6808/689014698,v1:172.21.15.115:6809/689014698] [v2:172.21.15.115:6810/689014698,v1:172.21.15.115:6811/689014698] exists,up 22ee34ef-522e-4121-85b7-38bbafeca5de 2024-09-17T22:41:16.953 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6800/453192890,v1:172.21.15.115:6801/453192890] [v2:172.21.15.115:6802/453192890,v1:172.21.15.115:6803/453192890] exists,up 5d464921-5696-43c5-b89d-430a54ddefb5 2024-09-17T22:41:16.953 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6800/380388860,v1:172.21.15.129:6801/380388860] [v2:172.21.15.129:6802/380388860,v1:172.21.15.129:6803/380388860] exists,up 4cb6973a-63ae-4f48-af94-37610aee3c06 2024-09-17T22:41:16.953 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6808/943488949,v1:172.21.15.129:6809/943488949] [v2:172.21.15.129:6810/943488949,v1:172.21.15.129:6811/943488949] exists,up ef9fe11b-b719-48f4-9c6c-e1b1b355acdc 2024-09-17T22:41:16.953 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6816/1873501077,v1:172.21.15.129:6817/1873501077] [v2:172.21.15.129:6818/1873501077,v1:172.21.15.129:6819/1873501077] exists,up e4045aec-22c1-401b-bfbf-64f115bc32af 2024-09-17T22:41:16.953 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6824/3609357666,v1:172.21.15.129:6825/3609357666] [v2:172.21.15.129:6826/3609357666,v1:172.21.15.129:6827/3609357666] exists,up 4a05a5d7-8281-4572-a9d0-6095ee5e168b 2024-09-17T22:41:16.964 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6824/2782271783,v1:172.21.15.115:6825/2782271783] [v2:172.21.15.115:6826/2782271783,v1:172.21.15.115:6827/2782271783] exists,up 3bd5c11f-e7c2-4968-839a-2799c00d93a6', 'osd.1 up in weight 1 up_from 14 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6816/1941780390,v1:172.21.15.115:6817/1941780390] [v2:172.21.15.115:6818/1941780390,v1:172.21.15.115:6819/1941780390] exists,up 02918203-cf2e-49fc-b994-743b5d81b4a1', 'osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6808/689014698,v1:172.21.15.115:6809/689014698] [v2:172.21.15.115:6810/689014698,v1:172.21.15.115:6811/689014698] exists,up 22ee34ef-522e-4121-85b7-38bbafeca5de', 'osd.3 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6800/453192890,v1:172.21.15.115:6801/453192890] [v2:172.21.15.115:6802/453192890,v1:172.21.15.115:6803/453192890] exists,up 5d464921-5696-43c5-b89d-430a54ddefb5', 'osd.4 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6800/380388860,v1:172.21.15.129:6801/380388860] [v2:172.21.15.129:6802/380388860,v1:172.21.15.129:6803/380388860] exists,up 4cb6973a-63ae-4f48-af94-37610aee3c06', 'osd.5 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6808/943488949,v1:172.21.15.129:6809/943488949] [v2:172.21.15.129:6810/943488949,v1:172.21.15.129:6811/943488949] exists,up ef9fe11b-b719-48f4-9c6c-e1b1b355acdc', 'osd.6 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6816/1873501077,v1:172.21.15.129:6817/1873501077] [v2:172.21.15.129:6818/1873501077,v1:172.21.15.129:6819/1873501077] exists,up e4045aec-22c1-401b-bfbf-64f115bc32af', 'osd.7 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6824/3609357666,v1:172.21.15.129:6825/3609357666] [v2:172.21.15.129:6826/3609357666,v1:172.21.15.129:6827/3609357666] exists,up 4a05a5d7-8281-4572-a9d0-6095ee5e168b'] 2024-09-17T22:41:16.964 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-17T22:41:17.063 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:17.063 INFO:teuthology.orchestra.run.smithi115.stdout: "name": "mon.a", 2024-09-17T22:41:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster": "ceph", 2024-09-17T22:41:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-17T22:41:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "admin_socket_mode": "", 2024-09-17T22:41:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-17T22:41:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_client_required": "cephx, none", 2024-09-17T22:41:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_cluster_required": "cephx", 2024-09-17T22:41:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_debug": "true", 2024-09-17T22:41:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-17T22:41:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-17T22:41:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_service_required": "cephx", 2024-09-17T22:41:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-17T22:41:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "auth_supported": "cephx", 2024-09-17T22:41:17.064 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_aio": "true", 2024-09-17T22:41:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-17T22:41:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_aio_poll_ms": "250", 2024-09-17T22:41:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_aio_reap_max": "16", 2024-09-17T22:41:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_async_discard": "false", 2024-09-17T22:41:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_block_size": "4096", 2024-09-17T22:41:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_debug_aio": "false", 2024-09-17T22:41:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-17T22:41:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-17T22:41:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_debug_inflight_ios": "false", 2024-09-17T22:41:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_enable_discard": "false", 2024-09-17T22:41:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_flock_retry": "3", 2024-09-17T22:41:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-17T22:41:17.065 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_inject_crash": "0", 2024-09-17T22:41:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-17T22:41:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_ioring": "false", 2024-09-17T22:41:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_ioring_hipri": "false", 2024-09-17T22:41:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-17T22:41:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-17T22:41:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-17T22:41:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-17T22:41:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-17T22:41:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-17T22:41:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "bdev_type": "", 2024-09-17T22:41:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_alloc_size": "1048576", 2024-09-17T22:41:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_allocator": "hybrid", 2024-09-17T22:41:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_buffered_io": "true", 2024-09-17T22:41:17.066 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_check_for_zeros": "false", 2024-09-17T22:41:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-17T22:41:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-17T22:41:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_compact_log_sync": "false", 2024-09-17T22:41:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-17T22:41:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-17T22:41:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-17T22:41:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-17T22:41:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_max_log_runway": "4194304", 2024-09-17T22:41:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_max_prefetch": "1048576", 2024-09-17T22:41:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_min_flush_size": "524288", 2024-09-17T22:41:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_min_log_runway": "1048576", 2024-09-17T22:41:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_replay_recovery": "false", 2024-09-17T22:41:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-17T22:41:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-17T22:41:17.067 INFO:teuthology.orchestra.run.smithi115.stdout: "bluefs_sync_write": "false", 2024-09-17T22:41:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-17T22:41:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-17T22:41:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-17T22:41:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_allocation_from_file": "true", 2024-09-17T22:41:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_allocator": "hybrid", 2024-09-17T22:41:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-17T22:41:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-17T22:41:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-17T22:41:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-17T22:41:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_bdev_label_multi": "true", 2024-09-17T22:41:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-17T22:41:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-17T22:41:17.068 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-17T22:41:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-17T22:41:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-17T22:41:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_block_create": "true", 2024-09-17T22:41:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_block_db_create": "false", 2024-09-17T22:41:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_block_db_path": "", 2024-09-17T22:41:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_block_db_size": "0", 2024-09-17T22:41:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_block_path": "", 2024-09-17T22:41:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_block_preallocate_file": "false", 2024-09-17T22:41:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_block_size": "107374182400", 2024-09-17T22:41:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_block_wal_create": "false", 2024-09-17T22:41:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_block_wal_path": "", 2024-09-17T22:41:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_block_wal_size": "100663296", 2024-09-17T22:41:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_bluefs": "true", 2024-09-17T22:41:17.069 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-17T22:41:17.070 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-17T22:41:17.070 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-17T22:41:17.070 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-17T22:41:17.070 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T22:41:17.070 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T22:41:17.070 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-17T22:41:17.070 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T22:41:17.070 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_autotune": "true", 2024-09-17T22:41:17.070 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-17T22:41:17.070 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-17T22:41:17.070 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-17T22:41:17.070 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-17T22:41:17.070 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_size": "0", 2024-09-17T22:41:17.071 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-17T22:41:17.071 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-17T22:41:17.071 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-17T22:41:17.071 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-17T22:41:17.071 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cache_type": "2q", 2024-09-17T22:41:17.071 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-17T22:41:17.071 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_clone_cow": "true", 2024-09-17T22:41:17.071 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-17T22:41:17.071 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-17T22:41:17.071 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-17T22:41:17.071 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-17T22:41:17.071 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-17T22:41:17.071 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-17T22:41:17.072 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-17T22:41:17.072 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_compression_mode": "none", 2024-09-17T22:41:17.072 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-17T22:41:17.072 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_csum_type": "crc32c", 2024-09-17T22:41:17.072 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-17T22:41:17.072 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_debug_freelist": "false", 2024-09-17T22:41:17.072 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-17T22:41:17.072 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-17T22:41:17.072 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-17T22:41:17.072 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-17T22:41:17.072 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-17T22:41:17.072 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-17T22:41:17.072 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-17T22:41:17.072 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-17T22:41:17.073 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-17T22:41:17.073 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-17T22:41:17.073 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-17T22:41:17.073 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-17T22:41:17.073 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_debug_small_allocations": "0", 2024-09-17T22:41:17.073 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-17T22:41:17.073 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_default_buffered_read": "true", 2024-09-17T22:41:17.073 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_default_buffered_write": "false", 2024-09-17T22:41:17.073 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-17T22:41:17.073 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-17T22:41:17.074 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-17T22:41:17.074 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-17T22:41:17.074 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-17T22:41:17.074 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-17T22:41:17.074 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-17T22:41:17.074 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-17T22:41:17.074 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-17T22:41:17.074 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fail_eio": "false", 2024-09-17T22:41:17.074 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-17T22:41:17.074 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-17T22:41:17.074 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-17T22:41:17.074 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-17T22:41:17.075 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-17T22:41:17.075 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-17T22:41:17.075 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_on_mount": "false", 2024-09-17T22:41:17.075 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-17T22:41:17.075 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_on_umount": "false", 2024-09-17T22:41:17.075 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-17T22:41:17.075 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-17T22:41:17.075 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-17T22:41:17.075 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-17T22:41:17.075 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-17T22:41:17.075 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-17T22:41:17.075 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-17T22:41:17.075 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-17T22:41:17.076 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_ignore_data_csum": "false", 2024-09-17T22:41:17.076 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-17T22:41:17.076 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-17T22:41:17.076 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-17T22:41:17.076 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-17T22:41:17.076 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_log_op_age": "5.000000", 2024-09-17T22:41:17.076 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_max_alloc_size": "0", 2024-09-17T22:41:17.076 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_max_blob_size": "0", 2024-09-17T22:41:17.076 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-17T22:41:17.076 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-17T22:41:17.076 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-17T22:41:17.076 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_max_deferred_txc": "32", 2024-09-17T22:41:17.076 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_min_alloc_size": "0", 2024-09-17T22:41:17.076 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-17T22:41:17.077 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-17T22:41:17.077 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_nid_prealloc": "1024", 2024-09-17T22:41:17.077 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-17T22:41:17.077 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-17T22:41:17.077 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-17T22:41:17.077 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-17T22:41:17.077 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_retry_disk_reads": "3", 2024-09-17T22:41:17.077 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_rocksdb_cf": "true", 2024-09-17T22:41:17.077 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-17T22:41:17.077 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-17T22:41:17.077 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-17T22:41:17.077 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-17T22:41:17.078 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-17T22:41:17.078 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-17T22:41:17.078 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-17T22:41:17.078 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-17T22:41:17.078 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_spdk_mem": "512", 2024-09-17T22:41:17.078 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-17T22:41:17.078 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-17T22:41:17.078 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-17T22:41:17.078 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-17T22:41:17.078 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-17T22:41:17.078 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-17T22:41:17.078 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-17T22:41:17.079 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_tracing": "false", 2024-09-17T22:41:17.079 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-17T22:41:17.079 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-17T22:41:17.079 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-17T22:41:17.079 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-17T22:41:17.079 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-17T22:41:17.079 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-17T22:41:17.079 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-17T22:41:17.079 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-17T22:41:17.079 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-17T22:41:17.079 INFO:teuthology.orchestra.run.smithi115.stdout: "bluestore_zero_block_detection": "false", 2024-09-17T22:41:17.079 INFO:teuthology.orchestra.run.smithi115.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-17T22:41:17.079 INFO:teuthology.orchestra.run.smithi115.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-17T22:41:17.079 INFO:teuthology.orchestra.run.smithi115.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-17T22:41:17.080 INFO:teuthology.orchestra.run.smithi115.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-17T22:41:17.080 INFO:teuthology.orchestra.run.smithi115.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-17T22:41:17.080 INFO:teuthology.orchestra.run.smithi115.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-17T22:41:17.080 INFO:teuthology.orchestra.run.smithi115.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-17T22:41:17.080 INFO:teuthology.orchestra.run.smithi115.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-17T22:41:17.080 INFO:teuthology.orchestra.run.smithi115.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-17T22:41:17.080 INFO:teuthology.orchestra.run.smithi115.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-17T22:41:17.080 INFO:teuthology.orchestra.run.smithi115.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-17T22:41:17.080 INFO:teuthology.orchestra.run.smithi115.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-17T22:41:17.080 INFO:teuthology.orchestra.run.smithi115.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-17T22:41:17.080 INFO:teuthology.orchestra.run.smithi115.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-17T22:41:17.080 INFO:teuthology.orchestra.run.smithi115.stdout: "cephx_cluster_require_signatures": "false", 2024-09-17T22:41:17.080 INFO:teuthology.orchestra.run.smithi115.stdout: "cephx_cluster_require_version": "2", 2024-09-17T22:41:17.080 INFO:teuthology.orchestra.run.smithi115.stdout: "cephx_require_signatures": "false", 2024-09-17T22:41:17.081 INFO:teuthology.orchestra.run.smithi115.stdout: "cephx_require_version": "2", 2024-09-17T22:41:17.081 INFO:teuthology.orchestra.run.smithi115.stdout: "cephx_service_require_signatures": "false", 2024-09-17T22:41:17.081 INFO:teuthology.orchestra.run.smithi115.stdout: "cephx_service_require_version": "2", 2024-09-17T22:41:17.081 INFO:teuthology.orchestra.run.smithi115.stdout: "cephx_sign_messages": "true", 2024-09-17T22:41:17.081 INFO:teuthology.orchestra.run.smithi115.stdout: "chdir": "", 2024-09-17T22:41:17.081 INFO:teuthology.orchestra.run.smithi115.stdout: "client_acl_type": "", 2024-09-17T22:41:17.081 INFO:teuthology.orchestra.run.smithi115.stdout: "client_asio_thread_count": "2", 2024-09-17T22:41:17.081 INFO:teuthology.orchestra.run.smithi115.stdout: "client_cache_mid": "0.750000", 2024-09-17T22:41:17.081 INFO:teuthology.orchestra.run.smithi115.stdout: "client_cache_size": "16384", 2024-09-17T22:41:17.081 INFO:teuthology.orchestra.run.smithi115.stdout: "client_caps_release_delay": "5", 2024-09-17T22:41:17.081 INFO:teuthology.orchestra.run.smithi115.stdout: "client_check_pool_perm": "true", 2024-09-17T22:41:17.081 INFO:teuthology.orchestra.run.smithi115.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-17T22:41:17.081 INFO:teuthology.orchestra.run.smithi115.stdout: "client_debug_force_sync_read": "false", 2024-09-17T22:41:17.081 INFO:teuthology.orchestra.run.smithi115.stdout: "client_debug_getattr_caps": "false", 2024-09-17T22:41:17.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_debug_inject_features": "", 2024-09-17T22:41:17.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_debug_inject_tick_delay": "0", 2024-09-17T22:41:17.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-17T22:41:17.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_die_on_failed_remount": "false", 2024-09-17T22:41:17.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_dirsize_rbytes": "true", 2024-09-17T22:41:17.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_force_lazyio": "false", 2024-09-17T22:41:17.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_fs": "", 2024-09-17T22:41:17.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-17T22:41:17.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_inject_release_failure": "false", 2024-09-17T22:41:17.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_max_inline_size": "4096", 2024-09-17T22:41:17.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-17T22:41:17.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_mds_namespace": "", 2024-09-17T22:41:17.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_metadata": "", 2024-09-17T22:41:17.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_mount_gid": "-1", 2024-09-17T22:41:17.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client_mount_timeout": "300", 2024-09-17T22:41:17.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client_mount_uid": "-1", 2024-09-17T22:41:17.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client_mountpoint": "/", 2024-09-17T22:41:17.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client_notify_timeout": "10", 2024-09-17T22:41:17.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client_oc": "true", 2024-09-17T22:41:17.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client_oc_max_dirty": "104857600", 2024-09-17T22:41:17.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-17T22:41:17.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client_oc_max_objects": "1000", 2024-09-17T22:41:17.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client_oc_size": "209715200", 2024-09-17T22:41:17.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client_oc_target_dirty": "8388608", 2024-09-17T22:41:17.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client_permissions": "true", 2024-09-17T22:41:17.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client_quota": "true", 2024-09-17T22:41:17.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client_quota_df": "true", 2024-09-17T22:41:17.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client_readahead_max_bytes": "0", 2024-09-17T22:41:17.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client_readahead_max_periods": "4", 2024-09-17T22:41:17.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_readahead_min": "131072", 2024-09-17T22:41:17.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_reconnect_stale": "false", 2024-09-17T22:41:17.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_shutdown_timeout": "30", 2024-09-17T22:41:17.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_snapdir": ".snap", 2024-09-17T22:41:17.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_tick_interval": "1", 2024-09-17T22:41:17.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_trace": "", 2024-09-17T22:41:17.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_try_dentry_invalidate": "false", 2024-09-17T22:41:17.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_use_faked_inos": "false", 2024-09-17T22:41:17.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_use_random_mds": "false", 2024-09-17T22:41:17.084 INFO:teuthology.orchestra.run.smithi115.stdout: "clog_to_graylog": "false", 2024-09-17T22:41:17.084 INFO:teuthology.orchestra.run.smithi115.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-17T22:41:17.084 INFO:teuthology.orchestra.run.smithi115.stdout: "clog_to_graylog_port": "12201", 2024-09-17T22:41:17.084 INFO:teuthology.orchestra.run.smithi115.stdout: "clog_to_monitors": "default=true", 2024-09-17T22:41:17.085 INFO:teuthology.orchestra.run.smithi115.stdout: "clog_to_syslog": "false", 2024-09-17T22:41:17.085 INFO:teuthology.orchestra.run.smithi115.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-17T22:41:17.085 INFO:teuthology.orchestra.run.smithi115.stdout: "clog_to_syslog_level": "info", 2024-09-17T22:41:17.085 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "-", 2024-09-17T22:41:17.085 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_network": "", 2024-09-17T22:41:17.085 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_network_interface": "", 2024-09-17T22:41:17.085 INFO:teuthology.orchestra.run.smithi115.stdout: "compressor_zlib_isal": "false", 2024-09-17T22:41:17.085 INFO:teuthology.orchestra.run.smithi115.stdout: "compressor_zlib_level": "5", 2024-09-17T22:41:17.085 INFO:teuthology.orchestra.run.smithi115.stdout: "compressor_zlib_winsize": "-15", 2024-09-17T22:41:17.085 INFO:teuthology.orchestra.run.smithi115.stdout: "compressor_zstd_level": "1", 2024-09-17T22:41:17.085 INFO:teuthology.orchestra.run.smithi115.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-17T22:41:17.085 INFO:teuthology.orchestra.run.smithi115.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-17T22:41:17.085 INFO:teuthology.orchestra.run.smithi115.stdout: "crimson_alien_op_num_threads": "6", 2024-09-17T22:41:17.086 INFO:teuthology.orchestra.run.smithi115.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-17T22:41:17.086 INFO:teuthology.orchestra.run.smithi115.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-17T22:41:17.086 INFO:teuthology.orchestra.run.smithi115.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-17T22:41:17.086 INFO:teuthology.orchestra.run.smithi115.stdout: "crimson_osd_stat_interval": "0", 2024-09-17T22:41:17.086 INFO:teuthology.orchestra.run.smithi115.stdout: "crimson_seastar_cpu_cores": "", 2024-09-17T22:41:17.086 INFO:teuthology.orchestra.run.smithi115.stdout: "crimson_seastar_num_threads": "0", 2024-09-17T22:41:17.086 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location": "", 2024-09-17T22:41:17.086 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location_hook": "", 2024-09-17T22:41:17.086 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_location_hook_timeout": "10", 2024-09-17T22:41:17.086 INFO:teuthology.orchestra.run.smithi115.stdout: "daemonize": "false", 2024-09-17T22:41:17.086 INFO:teuthology.orchestra.run.smithi115.stdout: "daos_pool": "tank", 2024-09-17T22:41:17.087 INFO:teuthology.orchestra.run.smithi115.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-17T22:41:17.087 INFO:teuthology.orchestra.run.smithi115.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-17T22:41:17.087 INFO:teuthology.orchestra.run.smithi115.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-17T22:41:17.087 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_alienstore": "0/5", 2024-09-17T22:41:17.087 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_allow_any_pool_priority": "false", 2024-09-17T22:41:17.087 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_asok": "1/5", 2024-09-17T22:41:17.087 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_asok_assert_abort": "false", 2024-09-17T22:41:17.087 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_asserts_on_shutdown": "true", 2024-09-17T22:41:17.087 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_auth": "20/20", 2024-09-17T22:41:17.087 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_bdev": "1/3", 2024-09-17T22:41:17.087 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_bluefs": "1/5", 2024-09-17T22:41:17.087 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_bluestore": "1/5", 2024-09-17T22:41:17.087 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_buffer": "0/1", 2024-09-17T22:41:17.088 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_ceph_exporter": "1/5", 2024-09-17T22:41:17.088 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_cephfs_mirror": "0/5", 2024-09-17T22:41:17.088 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_cephsqlite": "0/5", 2024-09-17T22:41:17.088 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_client": "0/5", 2024-09-17T22:41:17.088 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_compressor": "1/5", 2024-09-17T22:41:17.088 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_context": "0/1", 2024-09-17T22:41:17.088 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_crimson_interrupt": "0/5", 2024-09-17T22:41:17.088 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_crush": "1/1", 2024-09-17T22:41:17.088 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_crypto": "1/5", 2024-09-17T22:41:17.088 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_cyanstore": "0/5", 2024-09-17T22:41:17.088 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_deliberately_leak_memory": "false", 2024-09-17T22:41:17.089 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_disable_randomized_ping": "false", 2024-09-17T22:41:17.089 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_dpdk": "1/5", 2024-09-17T22:41:17.089 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_eventtrace": "1/5", 2024-09-17T22:41:17.089 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_filer": "0/1", 2024-09-17T22:41:17.089 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_filestore": "1/3", 2024-09-17T22:41:17.089 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_finisher": "1/1", 2024-09-17T22:41:17.089 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_fuse": "1/5", 2024-09-17T22:41:17.089 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_heartbeat_testing_span": "0", 2024-09-17T22:41:17.089 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_heartbeatmap": "1/5", 2024-09-17T22:41:17.089 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-17T22:41:17.089 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_javaclient": "1/5", 2024-09-17T22:41:17.089 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_journal": "1/3", 2024-09-17T22:41:17.090 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_journaler": "0/5", 2024-09-17T22:41:17.090 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_kstore": "1/5", 2024-09-17T22:41:17.090 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_lockdep": "0/1", 2024-09-17T22:41:17.090 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_mclock": "1/5", 2024-09-17T22:41:17.090 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_mds": "1/5", 2024-09-17T22:41:17.090 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_mds_balancer": "1/5", 2024-09-17T22:41:17.090 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_mds_locker": "1/5", 2024-09-17T22:41:17.090 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_mds_log": "1/5", 2024-09-17T22:41:17.090 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_mds_log_expire": "1/5", 2024-09-17T22:41:17.090 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_mds_migrator": "1/5", 2024-09-17T22:41:17.090 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_mds_quiesce": "3/5", 2024-09-17T22:41:17.090 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_memstore": "1/5", 2024-09-17T22:41:17.090 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_mgr": "2/5", 2024-09-17T22:41:17.090 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_mgrc": "1/5", 2024-09-17T22:41:17.090 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_mon": "20/20", 2024-09-17T22:41:17.091 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_monc": "0/10", 2024-09-17T22:41:17.091 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_ms": "1/1", 2024-09-17T22:41:17.091 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_none": "0/5", 2024-09-17T22:41:17.091 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_objclass": "0/5", 2024-09-17T22:41:17.091 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_objectcacher": "0/5", 2024-09-17T22:41:17.091 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_objecter": "0/1", 2024-09-17T22:41:17.091 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_optracker": "0/5", 2024-09-17T22:41:17.091 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_osd": "1/5", 2024-09-17T22:41:17.091 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_paxos": "20/20", 2024-09-17T22:41:17.091 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_perfcounter": "1/5", 2024-09-17T22:41:17.091 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_prioritycache": "1/5", 2024-09-17T22:41:17.091 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_rados": "0/5", 2024-09-17T22:41:17.091 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_rbd": "0/5", 2024-09-17T22:41:17.091 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_rbd_mirror": "0/5", 2024-09-17T22:41:17.091 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_rbd_pwl": "0/5", 2024-09-17T22:41:17.092 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_rbd_replay": "0/5", 2024-09-17T22:41:17.092 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_refs": "0/0", 2024-09-17T22:41:17.092 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_reserver": "1/1", 2024-09-17T22:41:17.092 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_rgw": "1/5", 2024-09-17T22:41:17.092 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_rgw_access": "1/5", 2024-09-17T22:41:17.092 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_rgw_datacache": "1/5", 2024-09-17T22:41:17.092 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_rgw_dbstore": "1/5", 2024-09-17T22:41:17.092 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_rgw_flight": "1/5", 2024-09-17T22:41:17.092 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-17T22:41:17.092 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_rgw_sync": "1/5", 2024-09-17T22:41:17.092 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_rocksdb": "4/5", 2024-09-17T22:41:17.092 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_seastore": "0/5", 2024-09-17T22:41:17.092 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_seastore_backref": "0/5", 2024-09-17T22:41:17.092 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_seastore_cache": "0/5", 2024-09-17T22:41:17.092 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_seastore_cleaner": "0/5", 2024-09-17T22:41:17.093 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_seastore_device": "0/5", 2024-09-17T22:41:17.093 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_seastore_epm": "0/5", 2024-09-17T22:41:17.093 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-17T22:41:17.093 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_seastore_journal": "0/5", 2024-09-17T22:41:17.093 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_seastore_lba": "0/5", 2024-09-17T22:41:17.093 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_seastore_odata": "0/5", 2024-09-17T22:41:17.093 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_seastore_omap": "0/5", 2024-09-17T22:41:17.093 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_seastore_onode": "0/5", 2024-09-17T22:41:17.093 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_seastore_t": "0/5", 2024-09-17T22:41:17.093 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_seastore_tm": "0/5", 2024-09-17T22:41:17.093 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_striper": "0/1", 2024-09-17T22:41:17.093 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_test": "0/5", 2024-09-17T22:41:17.093 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_throttle": "1/1", 2024-09-17T22:41:17.094 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_timer": "0/1", 2024-09-17T22:41:17.094 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_tp": "0/5", 2024-09-17T22:41:17.094 INFO:teuthology.orchestra.run.smithi115.stdout: "debug_trace": "1/5", 2024-09-17T22:41:17.094 INFO:teuthology.orchestra.run.smithi115.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-17T22:41:17.094 INFO:teuthology.orchestra.run.smithi115.stdout: "device_failure_prediction_mode": "none", 2024-09-17T22:41:17.094 INFO:teuthology.orchestra.run.smithi115.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-17T22:41:17.094 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-17T22:41:17.094 INFO:teuthology.orchestra.run.smithi115.stdout: "err_to_graylog": "false", 2024-09-17T22:41:17.094 INFO:teuthology.orchestra.run.smithi115.stdout: "err_to_journald": "false", 2024-09-17T22:41:17.094 INFO:teuthology.orchestra.run.smithi115.stdout: "err_to_stderr": "true", 2024-09-17T22:41:17.094 INFO:teuthology.orchestra.run.smithi115.stdout: "err_to_syslog": "false", 2024-09-17T22:41:17.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event_tracing": "false", 2024-09-17T22:41:17.094 INFO:teuthology.orchestra.run.smithi115.stdout: "exporter_addr": "0.0.0.0", 2024-09-17T22:41:17.094 INFO:teuthology.orchestra.run.smithi115.stdout: "exporter_http_port": "9926", 2024-09-17T22:41:17.094 INFO:teuthology.orchestra.run.smithi115.stdout: "exporter_prio_limit": "5", 2024-09-17T22:41:17.095 INFO:teuthology.orchestra.run.smithi115.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-17T22:41:17.095 INFO:teuthology.orchestra.run.smithi115.stdout: "exporter_sort_metrics": "true", 2024-09-17T22:41:17.095 INFO:teuthology.orchestra.run.smithi115.stdout: "exporter_stats_period": "5", 2024-09-17T22:41:17.095 INFO:teuthology.orchestra.run.smithi115.stdout: "fake_statfs_for_testing": "0", 2024-09-17T22:41:17.095 INFO:teuthology.orchestra.run.smithi115.stdout: "fatal_signal_handlers": "true", 2024-09-17T22:41:17.095 INFO:teuthology.orchestra.run.smithi115.stdout: "filer_max_purge_ops": "10", 2024-09-17T22:41:17.095 INFO:teuthology.orchestra.run.smithi115.stdout: "filer_max_truncate_ops": "128", 2024-09-17T22:41:17.095 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_apply_finisher_threads": "1", 2024-09-17T22:41:17.095 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_blackhole": "false", 2024-09-17T22:41:17.095 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_btrfs_clone_range": "true", 2024-09-17T22:41:17.095 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_btrfs_snap": "true", 2024-09-17T22:41:17.095 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_caller_concurrency": "10", 2024-09-17T22:41:17.095 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_collect_device_partition_information": "true", 2024-09-17T22:41:17.095 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_commit_timeout": "600.000000", 2024-09-17T22:41:17.096 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_debug_inject_read_err": "false", 2024-09-17T22:41:17.096 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_debug_omap_check": "false", 2024-09-17T22:41:17.096 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-17T22:41:17.096 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_debug_verify_split": "false", 2024-09-17T22:41:17.096 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_dump_file": "", 2024-09-17T22:41:17.096 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-17T22:41:17.096 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-17T22:41:17.096 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_fadvise": "true", 2024-09-17T22:41:17.096 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_fail_eio": "true", 2024-09-17T22:41:17.096 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_fd_cache_shards": "16", 2024-09-17T22:41:17.096 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_fd_cache_size": "128", 2024-09-17T22:41:17.096 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_fiemap": "false", 2024-09-17T22:41:17.096 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_fiemap_threshold": "4096", 2024-09-17T22:41:17.096 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-17T22:41:17.097 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-17T22:41:17.097 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_inject_stall": "0", 2024-09-17T22:41:17.097 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_journal_parallel": "false", 2024-09-17T22:41:17.097 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_journal_trailing": "false", 2024-09-17T22:41:17.097 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_journal_writeahead": "false", 2024-09-17T22:41:17.097 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_kill_at": "0", 2024-09-17T22:41:17.097 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-17T22:41:17.097 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-17T22:41:17.097 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-17T22:41:17.097 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-17T22:41:17.097 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-17T22:41:17.097 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_max_inline_xattrs": "0", 2024-09-17T22:41:17.097 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-17T22:41:17.097 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-17T22:41:17.097 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-17T22:41:17.098 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-17T22:41:17.098 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_max_xattr_value_size": "0", 2024-09-17T22:41:17.098 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-17T22:41:17.098 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-17T22:41:17.098 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-17T22:41:17.098 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_merge_threshold": "-10", 2024-09-17T22:41:17.098 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-17T22:41:17.098 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_odsync_write": "false", 2024-09-17T22:41:17.098 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_omap_backend": "rocksdb", 2024-09-17T22:41:17.098 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_omap_backend_path": "", 2024-09-17T22:41:17.098 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-17T22:41:17.098 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-17T22:41:17.098 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-17T22:41:17.098 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_op_thread_timeout": "60", 2024-09-17T22:41:17.099 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_op_threads": "2", 2024-09-17T22:41:17.099 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_punch_hole": "false", 2024-09-17T22:41:17.099 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-17T22:41:17.099 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-17T22:41:17.099 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-17T22:41:17.099 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-17T22:41:17.099 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-17T22:41:17.099 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-17T22:41:17.099 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-17T22:41:17.099 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-17T22:41:17.099 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-17T22:41:17.099 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_queue_max_ops": "50", 2024-09-17T22:41:17.099 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-17T22:41:17.099 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_seek_data_hole": "false", 2024-09-17T22:41:17.099 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_sloppy_crc": "false", 2024-09-17T22:41:17.100 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-17T22:41:17.100 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_splice": "false", 2024-09-17T22:41:17.100 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_split_multiple": "2", 2024-09-17T22:41:17.100 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_split_rand_factor": "20", 2024-09-17T22:41:17.100 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_update_to": "1000", 2024-09-17T22:41:17.100 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-17T22:41:17.100 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-17T22:41:17.100 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-17T22:41:17.100 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-17T22:41:17.100 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-17T22:41:17.100 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-17T22:41:17.100 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_wbthrottle_enable": "true", 2024-09-17T22:41:17.100 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-17T22:41:17.100 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-17T22:41:17.101 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-17T22:41:17.101 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-17T22:41:17.101 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-17T22:41:17.101 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-17T22:41:17.101 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_xfs_extsize": "false", 2024-09-17T22:41:17.101 INFO:teuthology.orchestra.run.smithi115.stdout: "filestore_zfs_snap": "false", 2024-09-17T22:41:17.101 INFO:teuthology.orchestra.run.smithi115.stdout: "fio_dir": "/tmp/fio", 2024-09-17T22:41:17.101 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "551c5ed8-e790-485a-bc95-a9578a29275c", 2024-09-17T22:41:17.101 INFO:teuthology.orchestra.run.smithi115.stdout: "fuse_allow_other": "true", 2024-09-17T22:41:17.101 INFO:teuthology.orchestra.run.smithi115.stdout: "fuse_atomic_o_trunc": "true", 2024-09-17T22:41:17.101 INFO:teuthology.orchestra.run.smithi115.stdout: "fuse_big_writes": "true", 2024-09-17T22:41:17.101 INFO:teuthology.orchestra.run.smithi115.stdout: "fuse_debug": "false", 2024-09-17T22:41:17.101 INFO:teuthology.orchestra.run.smithi115.stdout: "fuse_default_permissions": "false", 2024-09-17T22:41:17.101 INFO:teuthology.orchestra.run.smithi115.stdout: "fuse_disable_pagecache": "false", 2024-09-17T22:41:17.101 INFO:teuthology.orchestra.run.smithi115.stdout: "fuse_max_write": "0", 2024-09-17T22:41:17.102 INFO:teuthology.orchestra.run.smithi115.stdout: "fuse_multithreaded": "true", 2024-09-17T22:41:17.102 INFO:teuthology.orchestra.run.smithi115.stdout: "fuse_require_active_mds": "true", 2024-09-17T22:41:17.102 INFO:teuthology.orchestra.run.smithi115.stdout: "fuse_set_user_groups": "true", 2024-09-17T22:41:17.102 INFO:teuthology.orchestra.run.smithi115.stdout: "fuse_splice_move": "true", 2024-09-17T22:41:17.102 INFO:teuthology.orchestra.run.smithi115.stdout: "fuse_splice_read": "true", 2024-09-17T22:41:17.102 INFO:teuthology.orchestra.run.smithi115.stdout: "fuse_splice_write": "true", 2024-09-17T22:41:17.102 INFO:teuthology.orchestra.run.smithi115.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-17T22:41:17.102 INFO:teuthology.orchestra.run.smithi115.stdout: "fuse_use_invalidate_cb": "true", 2024-09-17T22:41:17.102 INFO:teuthology.orchestra.run.smithi115.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-17T22:41:17.102 INFO:teuthology.orchestra.run.smithi115.stdout: "gss_target_name": "ceph", 2024-09-17T22:41:17.102 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_file": "", 2024-09-17T22:41:17.102 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_inject_failure": "0", 2024-09-17T22:41:17.102 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_interval": "5", 2024-09-17T22:41:17.102 INFO:teuthology.orchestra.run.smithi115.stdout: "host": "smithi115", 2024-09-17T22:41:17.102 INFO:teuthology.orchestra.run.smithi115.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-17T22:41:17.102 INFO:teuthology.orchestra.run.smithi115.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-17T22:41:17.103 INFO:teuthology.orchestra.run.smithi115.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-17T22:41:17.103 INFO:teuthology.orchestra.run.smithi115.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-17T22:41:17.103 INFO:teuthology.orchestra.run.smithi115.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-17T22:41:17.103 INFO:teuthology.orchestra.run.smithi115.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-17T22:41:17.103 INFO:teuthology.orchestra.run.smithi115.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-17T22:41:17.103 INFO:teuthology.orchestra.run.smithi115.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-17T22:41:17.103 INFO:teuthology.orchestra.run.smithi115.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-17T22:41:17.103 INFO:teuthology.orchestra.run.smithi115.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-17T22:41:17.103 INFO:teuthology.orchestra.run.smithi115.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-17T22:41:17.103 INFO:teuthology.orchestra.run.smithi115.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-17T22:41:17.103 INFO:teuthology.orchestra.run.smithi115.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-17T22:41:17.104 INFO:teuthology.orchestra.run.smithi115.stdout: "inject_early_sigterm": "false", 2024-09-17T22:41:17.104 INFO:teuthology.orchestra.run.smithi115.stdout: "jaeger_agent_port": "6799", 2024-09-17T22:41:17.104 INFO:teuthology.orchestra.run.smithi115.stdout: "jaeger_tracing_enable": "false", 2024-09-17T22:41:17.104 INFO:teuthology.orchestra.run.smithi115.stdout: "journal_aio": "true", 2024-09-17T22:41:17.104 INFO:teuthology.orchestra.run.smithi115.stdout: "journal_align_min_size": "65536", 2024-09-17T22:41:17.104 INFO:teuthology.orchestra.run.smithi115.stdout: "journal_block_align": "true", 2024-09-17T22:41:17.104 INFO:teuthology.orchestra.run.smithi115.stdout: "journal_block_size": "4096", 2024-09-17T22:41:17.104 INFO:teuthology.orchestra.run.smithi115.stdout: "journal_dio": "true", 2024-09-17T22:41:17.104 INFO:teuthology.orchestra.run.smithi115.stdout: "journal_discard": "false", 2024-09-17T22:41:17.104 INFO:teuthology.orchestra.run.smithi115.stdout: "journal_force_aio": "false", 2024-09-17T22:41:17.104 INFO:teuthology.orchestra.run.smithi115.stdout: "journal_ignore_corruption": "false", 2024-09-17T22:41:17.104 INFO:teuthology.orchestra.run.smithi115.stdout: "journal_max_write_bytes": "10485760", 2024-09-17T22:41:17.104 INFO:teuthology.orchestra.run.smithi115.stdout: "journal_max_write_entries": "100", 2024-09-17T22:41:17.104 INFO:teuthology.orchestra.run.smithi115.stdout: "journal_replay_from": "0", 2024-09-17T22:41:17.105 INFO:teuthology.orchestra.run.smithi115.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-17T22:41:17.105 INFO:teuthology.orchestra.run.smithi115.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-17T22:41:17.105 INFO:teuthology.orchestra.run.smithi115.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-17T22:41:17.105 INFO:teuthology.orchestra.run.smithi115.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-17T22:41:17.105 INFO:teuthology.orchestra.run.smithi115.stdout: "journal_write_header_frequency": "0", 2024-09-17T22:41:17.105 INFO:teuthology.orchestra.run.smithi115.stdout: "journal_zero_on_create": "false", 2024-09-17T22:41:17.105 INFO:teuthology.orchestra.run.smithi115.stdout: "journaler_prefetch_periods": "10", 2024-09-17T22:41:17.105 INFO:teuthology.orchestra.run.smithi115.stdout: "journaler_prezero_periods": "5", 2024-09-17T22:41:17.105 INFO:teuthology.orchestra.run.smithi115.stdout: "journaler_write_head_interval": "15", 2024-09-17T22:41:17.105 INFO:teuthology.orchestra.run.smithi115.stdout: "key": "", 2024-09-17T22:41:17.105 INFO:teuthology.orchestra.run.smithi115.stdout: "keyfile": "", 2024-09-17T22:41:17.105 INFO:teuthology.orchestra.run.smithi115.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-17T22:41:17.105 INFO:teuthology.orchestra.run.smithi115.stdout: "kstore_backend": "rocksdb", 2024-09-17T22:41:17.105 INFO:teuthology.orchestra.run.smithi115.stdout: "kstore_default_stripe_size": "65536", 2024-09-17T22:41:17.105 INFO:teuthology.orchestra.run.smithi115.stdout: "kstore_fsck_on_mount": "false", 2024-09-17T22:41:17.106 INFO:teuthology.orchestra.run.smithi115.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-17T22:41:17.106 INFO:teuthology.orchestra.run.smithi115.stdout: "kstore_max_bytes": "67108864", 2024-09-17T22:41:17.106 INFO:teuthology.orchestra.run.smithi115.stdout: "kstore_max_ops": "512", 2024-09-17T22:41:17.106 INFO:teuthology.orchestra.run.smithi115.stdout: "kstore_nid_prealloc": "1024", 2024-09-17T22:41:17.106 INFO:teuthology.orchestra.run.smithi115.stdout: "kstore_onode_map_size": "1024", 2024-09-17T22:41:17.106 INFO:teuthology.orchestra.run.smithi115.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-17T22:41:17.106 INFO:teuthology.orchestra.run.smithi115.stdout: "kstore_sync_submit_transaction": "false", 2024-09-17T22:41:17.106 INFO:teuthology.orchestra.run.smithi115.stdout: "kstore_sync_transaction": "false", 2024-09-17T22:41:17.106 INFO:teuthology.orchestra.run.smithi115.stdout: "librados_thread_count": "2", 2024-09-17T22:41:17.106 INFO:teuthology.orchestra.run.smithi115.stdout: "lockdep": "false", 2024-09-17T22:41:17.106 INFO:teuthology.orchestra.run.smithi115.stdout: "lockdep_force_backtrace": "false", 2024-09-17T22:41:17.106 INFO:teuthology.orchestra.run.smithi115.stdout: "log_coarse_timestamps": "true", 2024-09-17T22:41:17.106 INFO:teuthology.orchestra.run.smithi115.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-17T22:41:17.106 INFO:teuthology.orchestra.run.smithi115.stdout: "log_flush_on_exit": "false", 2024-09-17T22:41:17.107 INFO:teuthology.orchestra.run.smithi115.stdout: "log_graylog_host": "127.0.0.1", 2024-09-17T22:41:17.107 INFO:teuthology.orchestra.run.smithi115.stdout: "log_graylog_port": "12201", 2024-09-17T22:41:17.107 INFO:teuthology.orchestra.run.smithi115.stdout: "log_max_new": "1000", 2024-09-17T22:41:17.107 INFO:teuthology.orchestra.run.smithi115.stdout: "log_max_recent": "10000", 2024-09-17T22:41:17.107 INFO:teuthology.orchestra.run.smithi115.stdout: "log_stderr_prefix": "", 2024-09-17T22:41:17.107 INFO:teuthology.orchestra.run.smithi115.stdout: "log_stop_at_utilization": "0.970000", 2024-09-17T22:41:17.107 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_file": "true", 2024-09-17T22:41:17.107 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_graylog": "false", 2024-09-17T22:41:17.107 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_journald": "false", 2024-09-17T22:41:17.107 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_stderr": "false", 2024-09-17T22:41:17.107 INFO:teuthology.orchestra.run.smithi115.stdout: "log_to_syslog": "false", 2024-09-17T22:41:17.107 INFO:teuthology.orchestra.run.smithi115.stdout: "max_rotating_auth_attempts": "10", 2024-09-17T22:41:17.107 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-17T22:41:17.107 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_action_on_write_error": "1", 2024-09-17T22:41:17.107 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_alternate_name_max": "8192", 2024-09-17T22:41:17.107 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_asio_thread_count": "2", 2024-09-17T22:41:17.108 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_export_pin": "true", 2024-09-17T22:41:17.108 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_fragment_dirs": "true", 2024-09-17T22:41:17.108 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-17T22:41:17.108 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_fragment_interval": "5", 2024-09-17T22:41:17.108 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-17T22:41:17.108 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-17T22:41:17.108 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_interval": "10", 2024-09-17T22:41:17.108 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_max": "-1", 2024-09-17T22:41:17.108 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_max_until": "-1", 2024-09-17T22:41:17.108 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_merge_size": "50", 2024-09-17T22:41:17.108 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_midchunk": "0.300000", 2024-09-17T22:41:17.108 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-17T22:41:17.108 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_min_start": "0.200000", 2024-09-17T22:41:17.109 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_minchunk": "0.001000", 2024-09-17T22:41:17.109 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_mode": "0", 2024-09-17T22:41:17.109 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_need_max": "1.200000", 2024-09-17T22:41:17.109 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_need_min": "0.800000", 2024-09-17T22:41:17.109 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_overload_epochs": "2", 2024-09-17T22:41:17.109 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-17T22:41:17.109 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-17T22:41:17.109 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_split_bits": "3", 2024-09-17T22:41:17.109 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-17T22:41:17.109 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_split_size": "10000", 2024-09-17T22:41:17.109 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-17T22:41:17.110 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_target_decay": "10.000000", 2024-09-17T22:41:17.110 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-17T22:41:17.110 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_beacon_grace": "15.000000", 2024-09-17T22:41:17.110 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_beacon_interval": "4.000000", 2024-09-17T22:41:17.110 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-17T22:41:17.110 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-17T22:41:17.110 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache_mid": "0.700000", 2024-09-17T22:41:17.110 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-17T22:41:17.110 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache_quiesce_delay": "0", 2024-09-17T22:41:17.110 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-17T22:41:17.110 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-17T22:41:17.110 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-17T22:41:17.110 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache_release_free_interval": "10", 2024-09-17T22:41:17.110 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache_reservation": "0.050000", 2024-09-17T22:41:17.110 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-17T22:41:17.110 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache_trim_interval": "1", 2024-09-17T22:41:17.111 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cache_trim_threshold": "262144", 2024-09-17T22:41:17.111 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-17T22:41:17.111 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-17T22:41:17.111 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-17T22:41:17.111 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_client_prealloc_inos": "1000", 2024-09-17T22:41:17.111 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-17T22:41:17.111 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_connect_bootstrapping": "false", 2024-09-17T22:41:17.111 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_damage_table_max_entries": "10000", 2024-09-17T22:41:17.111 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-17T22:41:17.111 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_debug_auth_pins": "false", 2024-09-17T22:41:17.111 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_debug_frag": "false", 2024-09-17T22:41:17.111 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_debug_scatterstat": "false", 2024-09-17T22:41:17.111 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_debug_subtrees": "false", 2024-09-17T22:41:17.111 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_decay_halflife": "5.000000", 2024-09-17T22:41:17.112 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_default_dir_hash": "2", 2024-09-17T22:41:17.112 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_defer_session_stale": "true", 2024-09-17T22:41:17.112 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_deny_all_reconnect": "false", 2024-09-17T22:41:17.112 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_dir_keys_per_op": "16384", 2024-09-17T22:41:17.112 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_dir_max_commit_size": "10", 2024-09-17T22:41:17.112 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_dir_max_entries": "0", 2024-09-17T22:41:17.112 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_dir_prefetch": "true", 2024-09-17T22:41:17.113 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-17T22:41:17.113 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-17T22:41:17.113 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_dump_cache_on_map": "false", 2024-09-17T22:41:17.113 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-17T22:41:17.113 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-17T22:41:17.113 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_early_reply": "true", 2024-09-17T22:41:17.113 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_enable_op_tracker": "true", 2024-09-17T22:41:17.114 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_enforce_unique_name": "true", 2024-09-17T22:41:17.114 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-17T22:41:17.114 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-17T22:41:17.114 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_export_ephemeral_random": "true", 2024-09-17T22:41:17.114 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-17T22:41:17.114 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-17T22:41:17.114 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-17T22:41:17.115 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-17T22:41:17.115 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-17T22:41:17.115 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-17T22:41:17.115 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-17T22:41:17.115 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-17T22:41:17.115 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_health_summarize_threshold": "10", 2024-09-17T22:41:17.115 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-17T22:41:17.116 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-17T22:41:17.116 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_inject_health_dummy": "false", 2024-09-17T22:41:17.116 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-17T22:41:17.116 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_inject_migrator_session_race": "false", 2024-09-17T22:41:17.116 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-17T22:41:17.116 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-17T22:41:17.117 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-17T22:41:17.117 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_join_fs": "", 2024-09-17T22:41:17.117 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_journal_format": "1", 2024-09-17T22:41:17.117 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-17T22:41:17.117 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_kill_create_at": "0", 2024-09-17T22:41:17.117 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_kill_export_at": "0", 2024-09-17T22:41:17.117 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_kill_import_at": "0", 2024-09-17T22:41:17.117 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_kill_journal_at": "0", 2024-09-17T22:41:17.118 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_kill_journal_expire_at": "0", 2024-09-17T22:41:17.118 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_kill_journal_replay_at": "0", 2024-09-17T22:41:17.118 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_kill_link_at": "0", 2024-09-17T22:41:17.118 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_kill_mdstable_at": "0", 2024-09-17T22:41:17.118 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_kill_openc_at": "0", 2024-09-17T22:41:17.118 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_kill_rename_at": "0", 2024-09-17T22:41:17.118 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_kill_shutdown_at": "0", 2024-09-17T22:41:17.118 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log_event_large_threshold": "524288", 2024-09-17T22:41:17.119 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log_events_per_segment": "1024", 2024-09-17T22:41:17.119 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-17T22:41:17.119 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log_max_events": "-1", 2024-09-17T22:41:17.119 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log_max_segments": "128", 2024-09-17T22:41:17.119 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log_pause": "false", 2024-09-17T22:41:17.119 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log_segment_size": "0", 2024-09-17T22:41:17.119 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-17T22:41:17.119 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-17T22:41:17.120 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-17T22:41:17.120 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log_trim_threshold": "128", 2024-09-17T22:41:17.120 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-17T22:41:17.120 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_log_warn_factor": "2.000000", 2024-09-17T22:41:17.120 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_max_caps_per_client": "1048576", 2024-09-17T22:41:17.120 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_max_completed_flushes": "100000", 2024-09-17T22:41:17.120 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_max_completed_requests": "100000", 2024-09-17T22:41:17.120 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_max_export_size": "20971520", 2024-09-17T22:41:17.120 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_max_file_recover": "32", 2024-09-17T22:41:17.121 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_max_purge_files": "64", 2024-09-17T22:41:17.121 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_max_purge_ops": "8192", 2024-09-17T22:41:17.121 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-17T22:41:17.121 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-17T22:41:17.121 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_max_snaps_per_dir": "100", 2024-09-17T22:41:17.121 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_metrics_update_interval": "2", 2024-09-17T22:41:17.121 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_min_caps_per_client": "100", 2024-09-17T22:41:17.121 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_min_caps_working_set": "10000", 2024-09-17T22:41:17.122 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-17T22:41:17.122 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_numa_node": "-1", 2024-09-17T22:41:17.122 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-17T22:41:17.122 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_op_complaint_time": "30.000000", 2024-09-17T22:41:17.122 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_op_history_duration": "600", 2024-09-17T22:41:17.122 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_op_history_size": "20", 2024-09-17T22:41:17.122 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_op_history_slow_op_size": "20", 2024-09-17T22:41:17.122 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-17T22:41:17.122 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_op_log_threshold": "5", 2024-09-17T22:41:17.122 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_ping_grace": "15", 2024-09-17T22:41:17.122 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_ping_interval": "5", 2024-09-17T22:41:17.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-17T22:41:17.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-17T22:41:17.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_recall_max_caps": "30000", 2024-09-17T22:41:17.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-17T22:41:17.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-17T22:41:17.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-17T22:41:17.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_recall_warning_threshold": "262144", 2024-09-17T22:41:17.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-17T22:41:17.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_replay_interval": "1.000000", 2024-09-17T22:41:17.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-17T22:41:17.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-17T22:41:17.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_root_ino_gid": "0", 2024-09-17T22:41:17.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_root_ino_uid": "0", 2024-09-17T22:41:17.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-17T22:41:17.123 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-17T22:41:17.124 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-17T22:41:17.124 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-17T22:41:17.124 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-17T22:41:17.124 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-17T22:41:17.124 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-17T22:41:17.124 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-17T22:41:17.124 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-17T22:41:17.124 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-17T22:41:17.124 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_shutdown_check": "0", 2024-09-17T22:41:17.124 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_skip_ino": "0", 2024-09-17T22:41:17.124 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-17T22:41:17.124 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_snap_max_uid": "4294967294", 2024-09-17T22:41:17.124 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_snap_min_uid": "0", 2024-09-17T22:41:17.124 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_snap_rstat": "false", 2024-09-17T22:41:17.125 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_standby_replay_damaged": "false", 2024-09-17T22:41:17.125 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_symlink_recovery": "true", 2024-09-17T22:41:17.125 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-17T22:41:17.125 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_thrash_exports": "0", 2024-09-17T22:41:17.125 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_thrash_fragments": "0", 2024-09-17T22:41:17.125 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_tick_interval": "5.000000", 2024-09-17T22:41:17.125 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_valgrind_exit": "false", 2024-09-17T22:41:17.125 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_verify_backtrace": "1", 2024-09-17T22:41:17.125 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_verify_scatter": "false", 2024-09-17T22:41:17.125 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-17T22:41:17.125 INFO:teuthology.orchestra.run.smithi115.stdout: "mds_wipe_sessions": "false", 2024-09-17T22:41:17.125 INFO:teuthology.orchestra.run.smithi115.stdout: "mempool_debug": "false", 2024-09-17T22:41:17.125 INFO:teuthology.orchestra.run.smithi115.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-17T22:41:17.125 INFO:teuthology.orchestra.run.smithi115.stdout: "memstore_device_bytes": "1073741824", 2024-09-17T22:41:17.126 INFO:teuthology.orchestra.run.smithi115.stdout: "memstore_page_set": "false", 2024-09-17T22:41:17.126 INFO:teuthology.orchestra.run.smithi115.stdout: "memstore_page_size": "65536", 2024-09-17T22:41:17.126 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_client_bytes": "134217728", 2024-09-17T22:41:17.126 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_client_messages": "512", 2024-09-17T22:41:17.126 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-17T22:41:17.126 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-17T22:41:17.126 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-17T22:41:17.126 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-17T22:41:17.126 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_disabled_modules": "", 2024-09-17T22:41:17.126 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_enable_op_tracker": "true", 2024-09-17T22:41:17.126 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-17T22:41:17.126 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_max_pg_num_change": "128", 2024-09-17T22:41:17.126 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_mds_bytes": "134217728", 2024-09-17T22:41:17.127 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_mds_messages": "128", 2024-09-17T22:41:17.127 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-17T22:41:17.127 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_mon_bytes": "134217728", 2024-09-17T22:41:17.127 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_mon_messages": "128", 2024-09-17T22:41:17.127 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-17T22:41:17.127 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-17T22:41:17.127 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_op_history_duration": "600", 2024-09-17T22:41:17.127 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_op_history_size": "20", 2024-09-17T22:41:17.127 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-17T22:41:17.127 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-17T22:41:17.127 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_op_log_threshold": "5", 2024-09-17T22:41:17.127 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_osd_bytes": "536870912", 2024-09-17T22:41:17.127 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_osd_messages": "8192", 2024-09-17T22:41:17.127 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_pool": "true", 2024-09-17T22:41:17.127 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-17T22:41:17.128 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_standby_modules": "true", 2024-09-17T22:41:17.128 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_stats_period": "5", 2024-09-17T22:41:17.128 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_stats_threshold": "5", 2024-09-17T22:41:17.128 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_tick_period": "2", 2024-09-17T22:41:17.128 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-17T22:41:17.128 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-17T22:41:17.128 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_allow_pool_delete": "true", 2024-09-17T22:41:17.128 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_allow_pool_size_one": "true", 2024-09-17T22:41:17.128 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_auth_validate_all_caps": "true", 2024-09-17T22:41:17.128 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-17T22:41:17.128 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-17T22:41:17.128 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_client_bytes": "104857600", 2024-09-17T22:41:17.128 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_client_directed_command_retry": "5", 2024-09-17T22:41:17.128 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-17T22:41:17.129 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-17T22:41:17.129 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-17T22:41:17.129 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-17T22:41:17.129 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_client_hunt_parallel": "3", 2024-09-17T22:41:17.129 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_client_log_interval": "1.000000", 2024-09-17T22:41:17.129 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-17T22:41:17.129 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_client_ping_interval": "10.000000", 2024-09-17T22:41:17.129 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-17T22:41:17.129 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_client_target_rank": "-1", 2024-09-17T22:41:17.129 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-17T22:41:17.129 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-17T22:41:17.129 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-17T22:41:17.129 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_cluster_log_file_level": "debug", 2024-09-17T22:41:17.130 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_cluster_log_to_file": "true", 2024-09-17T22:41:17.130 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-17T22:41:17.130 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-17T22:41:17.130 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-17T22:41:17.130 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_cluster_log_to_journald": "false", 2024-09-17T22:41:17.130 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-17T22:41:17.130 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-17T22:41:17.130 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-17T22:41:17.130 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-17T22:41:17.130 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_compact_on_bootstrap": "false", 2024-09-17T22:41:17.130 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_compact_on_start": "false", 2024-09-17T22:41:17.130 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_compact_on_trim": "true", 2024-09-17T22:41:17.130 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-17T22:41:17.130 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-17T22:41:17.130 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-17T22:41:17.131 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_cpu_threads": "4", 2024-09-17T22:41:17.131 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_crush_min_required_version": "hammer", 2024-09-17T22:41:17.131 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_daemon_bytes": "419430400", 2024-09-17T22:41:17.131 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-17T22:41:17.131 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_data_avail_crit": "5", 2024-09-17T22:41:17.131 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_data_avail_warn": "5", 2024-09-17T22:41:17.131 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_data_size_warn": "16106127360", 2024-09-17T22:41:17.131 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-17T22:41:17.131 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-17T22:41:17.131 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_debug_dump_json": "false", 2024-09-17T22:41:17.131 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-17T22:41:17.131 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_debug_dump_transactions": "false", 2024-09-17T22:41:17.131 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_debug_extra_checks": "false", 2024-09-17T22:41:17.132 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-17T22:41:17.132 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-17T22:41:17.132 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_debug_no_require_reef": "false", 2024-09-17T22:41:17.132 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_debug_no_require_squid": "false", 2024-09-17T22:41:17.132 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-17T22:41:17.132 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-17T22:41:17.132 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-17T22:41:17.132 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_down_mkfs_grace": "120", 2024-09-17T22:41:17.132 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_down_uptime_grace": "60", 2024-09-17T22:41:17.132 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_election_default_strategy": "1", 2024-09-17T22:41:17.132 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_election_timeout": "5.000000", 2024-09-17T22:41:17.132 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-17T22:41:17.132 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_elector_ping_divisor": "2", 2024-09-17T22:41:17.132 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-17T22:41:17.133 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_enable_op_tracker": "true", 2024-09-17T22:41:17.133 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_fake_pool_delete": "false", 2024-09-17T22:41:17.133 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_force_quorum_join": "false", 2024-09-17T22:41:17.133 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-17T22:41:17.133 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_globalid_prealloc": "10000", 2024-09-17T22:41:17.133 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_health_detail_to_clog": "false", 2024-09-17T22:41:17.133 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_health_log_update_period": "5", 2024-09-17T22:41:17.133 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_health_max_detail": "50", 2024-09-17T22:41:17.133 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_health_to_clog": "true", 2024-09-17T22:41:17.133 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_health_to_clog_interval": "600", 2024-09-17T22:41:17.133 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-17T22:41:17.133 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_host": "172.21.15.115,[v2:172.21.15.115:3301,v1:172.21.15.115:6790],172.21.15.129", 2024-09-17T22:41:17.133 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_host_override": "", 2024-09-17T22:41:17.134 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_initial_members": "", 2024-09-17T22:41:17.134 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-17T22:41:17.134 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-17T22:41:17.134 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-17T22:41:17.134 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-17T22:41:17.134 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-17T22:41:17.134 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_lease": "5.000000", 2024-09-17T22:41:17.134 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-17T22:41:17.134 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-17T22:41:17.134 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_log_full_interval": "50", 2024-09-17T22:41:17.134 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_log_max": "10000", 2024-09-17T22:41:17.134 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_log_max_summary": "50", 2024-09-17T22:41:17.134 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-17T22:41:17.134 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_max_log_epochs": "500", 2024-09-17T22:41:17.134 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-17T22:41:17.135 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-17T22:41:17.135 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_max_osd": "10000", 2024-09-17T22:41:17.135 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_max_pg_per_osd": "10000", 2024-09-17T22:41:17.135 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_max_pool_pg_num": "65536", 2024-09-17T22:41:17.135 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-17T22:41:17.135 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-17T22:41:17.135 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_mds_force_trim_to": "0", 2024-09-17T22:41:17.135 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_mds_skip_sanity": "false", 2024-09-17T22:41:17.135 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_memory_autotune": "true", 2024-09-17T22:41:17.135 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_memory_target": "2147483648", 2024-09-17T22:41:17.135 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_mgr_beacon_grace": "30", 2024-09-17T22:41:17.135 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-17T22:41:17.135 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_mgr_digest_period": "5", 2024-09-17T22:41:17.135 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_mgr_inactive_grace": "60", 2024-09-17T22:41:17.135 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-17T22:41:17.136 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-17T22:41:17.136 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_min_osdmap_epochs": "500", 2024-09-17T22:41:17.136 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_op_complaint_time": "30", 2024-09-17T22:41:17.136 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_op_history_duration": "600", 2024-09-17T22:41:17.136 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_op_history_size": "20", 2024-09-17T22:41:17.136 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_op_history_slow_op_size": "20", 2024-09-17T22:41:17.136 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-17T22:41:17.136 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_op_log_threshold": "5", 2024-09-17T22:41:17.136 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-17T22:41:17.136 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-17T22:41:17.136 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-17T22:41:17.136 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_auto_mark_in": "false", 2024-09-17T22:41:17.136 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-17T22:41:17.136 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-17T22:41:17.137 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-17T22:41:17.137 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_cache_size": "500", 2024-09-17T22:41:17.137 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-17T22:41:17.137 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-17T22:41:17.137 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-17T22:41:17.137 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_down_out_interval": "600", 2024-09-17T22:41:17.137 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-17T22:41:17.137 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-17T22:41:17.137 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_force_trim_to": "0", 2024-09-17T22:41:17.137 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-17T22:41:17.137 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-17T22:41:17.137 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-17T22:41:17.137 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-17T22:41:17.137 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-17T22:41:17.138 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-17T22:41:17.138 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-17T22:41:17.138 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-17T22:41:17.138 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_min_down_reporters": "2", 2024-09-17T22:41:17.138 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-17T22:41:17.138 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-17T22:41:17.138 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-17T22:41:17.138 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-17T22:41:17.138 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-17T22:41:17.138 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-17T22:41:17.138 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_report_timeout": "900", 2024-09-17T22:41:17.138 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-17T22:41:17.138 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-17T22:41:17.138 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-17T22:41:17.138 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-17T22:41:17.139 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-17T22:41:17.139 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-17T22:41:17.139 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-17T22:41:17.139 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-17T22:41:17.139 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-17T22:41:17.139 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_pg_stuck_threshold": "60", 2024-09-17T22:41:17.139 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-17T22:41:17.139 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-17T22:41:17.139 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-17T22:41:17.139 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-17T22:41:17.139 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-17T22:41:17.139 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-17T22:41:17.139 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_probe_timeout": "2.000000", 2024-09-17T22:41:17.139 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_reweight_max_change": "0.050000", 2024-09-17T22:41:17.140 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_reweight_max_osds": "4", 2024-09-17T22:41:17.140 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-17T22:41:17.140 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-17T22:41:17.140 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-17T22:41:17.140 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-17T22:41:17.140 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-17T22:41:17.140 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_scrub_interval": "86400", 2024-09-17T22:41:17.140 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_scrub_max_keys": "100", 2024-09-17T22:41:17.140 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_scrub_timeout": "300", 2024-09-17T22:41:17.140 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_session_timeout": "300", 2024-09-17T22:41:17.140 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_smart_report_timeout": "5", 2024-09-17T22:41:17.140 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_stat_smooth_intervals": "6", 2024-09-17T22:41:17.140 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-17T22:41:17.140 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_stretch_pool_min_size": "2", 2024-09-17T22:41:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_stretch_pool_size": "4", 2024-09-17T22:41:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-17T22:41:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-17T22:41:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_sync_debug": "false", 2024-09-17T22:41:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-17T22:41:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-17T22:41:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_sync_provider_kill_at": "0", 2024-09-17T22:41:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_sync_requester_kill_at": "0", 2024-09-17T22:41:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_sync_timeout": "60.000000", 2024-09-17T22:41:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_target_pg_per_osd": "100", 2024-09-17T22:41:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_tick_interval": "5", 2024-09-17T22:41:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_timecheck_interval": "300.000000", 2024-09-17T22:41:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-17T22:41:17.141 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_use_min_delay_socket": "false", 2024-09-17T22:41:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_older_version_delay": "604800", 2024-09-17T22:41:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-17T22:41:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-17T22:41:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-17T22:41:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-17T22:41:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-17T22:41:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-17T22:41:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-17T22:41:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_on_misplaced": "false", 2024-09-17T22:41:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-17T22:41:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_on_older_version": "true", 2024-09-17T22:41:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-17T22:41:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-17T22:41:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-17T22:41:17.142 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-17T22:41:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-17T22:41:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-17T22:41:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-17T22:41:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-17T22:41:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-17T22:41:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "monmap": "", 2024-09-17T22:41:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-17T22:41:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-17T22:41:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-17T22:41:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-17T22:41:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-17T22:41:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-17T22:41:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "motr_tracing_enabled": "false", 2024-09-17T22:41:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_async_op_threads": "3", 2024-09-17T22:41:17.143 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-17T22:41:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_async_rdma_cm": "false", 2024-09-17T22:41:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_async_rdma_device_name": "", 2024-09-17T22:41:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_async_rdma_dscp": "96", 2024-09-17T22:41:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-17T22:41:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-17T22:41:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_async_rdma_local_gid": "", 2024-09-17T22:41:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-17T22:41:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_async_rdma_port_num": "1", 2024-09-17T22:41:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-17T22:41:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-17T22:41:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-17T22:41:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-17T22:41:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_async_rdma_sl": "3", 2024-09-17T22:41:17.144 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_async_rdma_support_srq": "true", 2024-09-17T22:41:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_async_rdma_type": "ib", 2024-09-17T22:41:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_async_reap_threshold": "5", 2024-09-17T22:41:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_bind_before_connect": "false", 2024-09-17T22:41:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_bind_ipv4": "true", 2024-09-17T22:41:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_bind_ipv6": "false", 2024-09-17T22:41:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_bind_msgr1": "true", 2024-09-17T22:41:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_bind_msgr2": "true", 2024-09-17T22:41:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_bind_port_max": "7568", 2024-09-17T22:41:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_bind_port_min": "6800", 2024-09-17T22:41:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-17T22:41:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_bind_retry_count": "3", 2024-09-17T22:41:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_bind_retry_delay": "5", 2024-09-17T22:41:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_blackhole_client": "false", 2024-09-17T22:41:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_blackhole_mds": "false", 2024-09-17T22:41:17.145 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_blackhole_mgr": "false", 2024-09-17T22:41:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_blackhole_mon": "false", 2024-09-17T22:41:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_blackhole_osd": "false", 2024-09-17T22:41:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_client_mode": "crc secure", 2024-09-17T22:41:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_cluster_mode": "crc secure", 2024-09-17T22:41:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_cluster_type": "", 2024-09-17T22:41:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_compress_secure": "false", 2024-09-17T22:41:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_connection_idle_timeout": "900", 2024-09-17T22:41:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_connection_ready_timeout": "10", 2024-09-17T22:41:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_crc_data": "true", 2024-09-17T22:41:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_crc_header": "true", 2024-09-17T22:41:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_die_on_bad_msg": "false", 2024-09-17T22:41:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_die_on_bug": "true", 2024-09-17T22:41:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_die_on_old_message": "true", 2024-09-17T22:41:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_die_on_skipped_message": "false", 2024-09-17T22:41:17.146 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-17T22:41:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-17T22:41:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_dpdk_coremask": "0xF", 2024-09-17T22:41:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-17T22:41:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-17T22:41:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_dpdk_enable_tso": "true", 2024-09-17T22:41:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-17T22:41:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-17T22:41:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_dpdk_hugepages": "", 2024-09-17T22:41:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-17T22:41:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-17T22:41:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_dpdk_lro": "true", 2024-09-17T22:41:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_dpdk_memory_channel": "4", 2024-09-17T22:41:17.147 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-17T22:41:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_dpdk_pmd": "", 2024-09-17T22:41:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_dpdk_port_id": "0", 2024-09-17T22:41:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-17T22:41:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-17T22:41:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_dump_on_send": "false", 2024-09-17T22:41:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_initial_backoff": "0.200000", 2024-09-17T22:41:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_inject_delay_max": "1.000000", 2024-09-17T22:41:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-17T22:41:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_inject_delay_type": "", 2024-09-17T22:41:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-17T22:41:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_inject_network_congestion": "0", 2024-09-17T22:41:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_inject_socket_failures": "5000", 2024-09-17T22:41:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_learn_addr_from_peer": "true", 2024-09-17T22:41:17.148 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_max_accept_failures": "4", 2024-09-17T22:41:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_max_backoff": "15.000000", 2024-09-17T22:41:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_mon_client_mode": "secure crc", 2024-09-17T22:41:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-17T22:41:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_mon_service_mode": "secure crc", 2024-09-17T22:41:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_osd_compress_min_size": "1024", 2024-09-17T22:41:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_osd_compress_mode": "none", 2024-09-17T22:41:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-17T22:41:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-17T22:41:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_pq_min_cost": "65536", 2024-09-17T22:41:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_public_type": "", 2024-09-17T22:41:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_service_mode": "crc secure", 2024-09-17T22:41:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_tcp_listen_backlog": "512", 2024-09-17T22:41:17.149 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_tcp_nodelay": "true", 2024-09-17T22:41:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-17T22:41:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_tcp_rcvbuf": "0", 2024-09-17T22:41:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "ms_type": "async+posix", 2024-09-17T22:41:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "no_config_file": "false", 2024-09-17T22:41:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter_completion_locks_per_session": "32", 2024-09-17T22:41:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-17T22:41:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-17T22:41:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter_inflight_ops": "1024", 2024-09-17T22:41:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-17T22:41:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-17T22:41:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter_tick_interval": "5.000000", 2024-09-17T22:41:17.150 INFO:teuthology.orchestra.run.smithi115.stdout: "objecter_timeout": "10.000000", 2024-09-17T22:41:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "objectstore_blackhole": "false", 2024-09-17T22:41:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-17T22:41:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "openssl_engine_opts": "", 2024-09-17T22:41:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_agent_delay_time": "5.000000", 2024-09-17T22:41:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_agent_hist_halflife": "1000", 2024-09-17T22:41:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_agent_max_low_ops": "2", 2024-09-17T22:41:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_agent_max_ops": "4", 2024-09-17T22:41:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-17T22:41:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-17T22:41:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_agent_slop": "0.020000", 2024-09-17T22:41:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-17T22:41:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-17T22:41:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_asio_thread_count": "2", 2024-09-17T22:41:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_async_recovery_min_cost": "100", 2024-09-17T22:41:17.151 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-17T22:41:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-17T22:41:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_backfill_scan_max": "512", 2024-09-17T22:41:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_backfill_scan_min": "64", 2024-09-17T22:41:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_backoff_on_degraded": "false", 2024-09-17T22:41:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_backoff_on_peering": "false", 2024-09-17T22:41:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_backoff_on_unfound": "true", 2024-09-17T22:41:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_beacon_report_interval": "300", 2024-09-17T22:41:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bench_duration": "30", 2024-09-17T22:41:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-17T22:41:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bench_max_block_size": "67108864", 2024-09-17T22:41:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-17T22:41:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_blkin_trace_all": "false", 2024-09-17T22:41:17.152 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-17T22:41:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-17T22:41:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-17T22:41:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-17T22:41:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_check_for_log_corruption": "false", 2024-09-17T22:41:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-17T22:41:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-17T22:41:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-17T22:41:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-17T22:41:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_class_update_on_start": "true", 2024-09-17T22:41:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_client_message_cap": "256", 2024-09-17T22:41:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_client_message_size_cap": "524288000", 2024-09-17T22:41:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_client_op_priority": "63", 2024-09-17T22:41:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_client_watch_timeout": "30", 2024-09-17T22:41:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_command_max_records": "256", 2024-09-17T22:41:17.153 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_compact_on_start": "false", 2024-09-17T22:41:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-17T22:41:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-17T22:41:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-17T22:41:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_crush_update_on_start": "true", 2024-09-17T22:41:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_crush_update_weight_set": "true", 2024-09-17T22:41:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-17T22:41:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-17T22:41:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-17T22:41:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-17T22:41:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-17T22:41:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_feed_pullee": "-1", 2024-09-17T22:41:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-17T22:41:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-17T22:41:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-17T22:41:17.154 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_misdirected_ops": "false", 2024-09-17T22:41:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_no_acting_change": "false", 2024-09-17T22:41:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_no_purge_strays": "false", 2024-09-17T22:41:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_op_order": "false", 2024-09-17T22:41:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-17T22:41:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-17T22:41:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-17T22:41:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-17T22:41:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_shutdown": "false", 2024-09-17T22:41:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-17T22:41:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-17T22:41:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_trim_objects": "false", 2024-09-17T22:41:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-17T22:41:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-17T22:41:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_verify_snaps": "false", 2024-09-17T22:41:17.155 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-17T22:41:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-17T22:41:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_deep_scrub_keys": "1024", 2024-09-17T22:41:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-17T22:41:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-17T22:41:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-17T22:41:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_deep_scrub_stride": "524288", 2024-09-17T22:41:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-17T22:41:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-17T22:41:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_default_notify_timeout": "30", 2024-09-17T22:41:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_delete_sleep": "0.000000", 2024-09-17T22:41:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-17T22:41:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-17T22:41:17.156 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-17T22:41:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_discard_disconnected_ops": "true", 2024-09-17T22:41:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_enable_op_tracker": "true", 2024-09-17T22:41:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-17T22:41:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-17T22:41:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-17T22:41:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-17T22:41:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_fast_info": "true", 2024-09-17T22:41:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_fast_shutdown": "true", 2024-09-17T22:41:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-17T22:41:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-17T22:41:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-17T22:41:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-17T22:41:17.157 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-17T22:41:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_function_tracing": "false", 2024-09-17T22:41:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_heartbeat_grace": "20", 2024-09-17T22:41:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_heartbeat_interval": "6", 2024-09-17T22:41:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-17T22:41:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_heartbeat_min_peers": "10", 2024-09-17T22:41:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_heartbeat_min_size": "2000", 2024-09-17T22:41:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_heartbeat_stale": "600", 2024-09-17T22:41:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-17T22:41:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_hit_set_max_size": "100000", 2024-09-17T22:41:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_hit_set_min_size": "1000", 2024-09-17T22:41:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-17T22:41:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-17T22:41:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-17T22:41:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-17T22:41:17.158 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-17T22:41:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-17T22:41:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_journal_size": "5120", 2024-09-17T22:41:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_kill_backfill_at": "0", 2024-09-17T22:41:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-17T22:41:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_map_cache_size": "50", 2024-09-17T22:41:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_map_dedup": "true", 2024-09-17T22:41:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_map_message_max": "40", 2024-09-17T22:41:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-17T22:41:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_map_share_max_epochs": "40", 2024-09-17T22:41:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_attr_name_len": "100", 2024-09-17T22:41:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_attr_size": "0", 2024-09-17T22:41:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_backfills": "1", 2024-09-17T22:41:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_markdown_count": "5", 2024-09-17T22:41:17.159 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_markdown_period": "600", 2024-09-17T22:41:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_object_name_len": "2048", 2024-09-17T22:41:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_object_namespace_len": "256", 2024-09-17T22:41:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_object_size": "134217728", 2024-09-17T22:41:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-17T22:41:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-17T22:41:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_pg_blocked_by": "16", 2024-09-17T22:41:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_pg_log_entries": "10000", 2024-09-17T22:41:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-17T22:41:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_pgls": "1024", 2024-09-17T22:41:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_push_cost": "8388608", 2024-09-17T22:41:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_push_objects": "10", 2024-09-17T22:41:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_scrubs": "3", 2024-09-17T22:41:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-17T22:41:17.160 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_trimming_pgs": "2", 2024-09-17T22:41:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_write_op_reply_len": "64", 2024-09-17T22:41:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_max_write_size": "90", 2024-09-17T22:41:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-17T22:41:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-17T22:41:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-17T22:41:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-17T22:41:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-17T22:41:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-17T22:41:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-17T22:41:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-17T22:41:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_profile": "balanced", 2024-09-17T22:41:17.161 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-17T22:41:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-17T22:41:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-17T22:41:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-17T22:41:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-17T22:41:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-17T22:41:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-17T22:41:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-17T22:41:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-17T22:41:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-17T22:41:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-17T22:41:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_memory_base": "805306368", 2024-09-17T22:41:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_memory_cache_min": "134217728", 2024-09-17T22:41:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-17T22:41:17.162 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-17T22:41:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_memory_target": "4294967296", 2024-09-17T22:41:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_memory_target_autotune": "false", 2024-09-17T22:41:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-17T22:41:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_min_pg_log_entries": "250", 2024-09-17T22:41:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_min_recovery_priority": "0", 2024-09-17T22:41:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-17T22:41:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-17T22:41:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mon_report_interval": "5", 2024-09-17T22:41:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-17T22:41:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-17T22:41:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_num_cache_shards": "32", 2024-09-17T22:41:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_num_op_tracker_shard": "32", 2024-09-17T22:41:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_numa_auto_affinity": "true", 2024-09-17T22:41:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_numa_node": "-1", 2024-09-17T22:41:17.163 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_numa_prefer_iface": "true", 2024-09-17T22:41:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-17T22:41:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_objecter_finishers": "1", 2024-09-17T22:41:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_objectstore": "bluestore", 2024-09-17T22:41:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_objectstore_fuse": "false", 2024-09-17T22:41:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_objectstore_tracing": "false", 2024-09-17T22:41:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_op_complaint_time": "30.000000", 2024-09-17T22:41:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_op_history_duration": "600", 2024-09-17T22:41:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_op_history_size": "20", 2024-09-17T22:41:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_op_history_slow_op_size": "20", 2024-09-17T22:41:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-17T22:41:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_op_log_threshold": "5", 2024-09-17T22:41:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_op_num_shards": "0", 2024-09-17T22:41:17.164 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_op_num_shards_hdd": "5", 2024-09-17T22:41:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_op_num_shards_ssd": "8", 2024-09-17T22:41:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-17T22:41:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-17T22:41:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-17T22:41:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-17T22:41:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_op_pq_min_cost": "65536", 2024-09-17T22:41:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-17T22:41:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_op_queue_cut_off": "high", 2024-09-17T22:41:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-17T22:41:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_op_thread_timeout": "15", 2024-09-17T22:41:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_open_classes_on_start": "true", 2024-09-17T22:41:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_os_flags": "0", 2024-09-17T22:41:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_peering_op_priority": "255", 2024-09-17T22:41:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pg_delete_cost": "1048576", 2024-09-17T22:41:17.165 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pg_delete_priority": "5", 2024-09-17T22:41:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-17T22:41:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-17T22:41:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-17T22:41:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pg_log_trim_max": "10000", 2024-09-17T22:41:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pg_log_trim_min": "100", 2024-09-17T22:41:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-17T22:41:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-17T22:41:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-17T22:41:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-17T22:41:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-17T22:41:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-17T22:41:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-17T22:41:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-17T22:41:17.166 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-17T22:41:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-17T22:41:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_crimson": "false", 2024-09-17T22:41:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-17T22:41:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-17T22:41:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-17T22:41:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-17T22:41:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-17T22:41:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-17T22:41:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-17T22:41:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-17T22:41:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_flags": "0", 2024-09-17T22:41:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-17T22:41:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_min_size": "0", 2024-09-17T22:41:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-17T22:41:17.167 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_pg_num": "32", 2024-09-17T22:41:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_pgp_num": "0", 2024-09-17T22:41:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-17T22:41:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_read_ratio": "70", 2024-09-17T22:41:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_size": "2", 2024-09-17T22:41:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_default_type": "replicated", 2024-09-17T22:41:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-17T22:41:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-17T22:41:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_push_per_object_cost": "1000", 2024-09-17T22:41:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-17T22:41:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_recover_clone_overlap": "true", 2024-09-17T22:41:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-17T22:41:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_recovery_cost": "20971520", 2024-09-17T22:41:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-17T22:41:17.168 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_recovery_max_active": "0", 2024-09-17T22:41:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-17T22:41:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-17T22:41:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-17T22:41:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-17T22:41:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_recovery_max_single_start": "1", 2024-09-17T22:41:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_recovery_op_priority": "3", 2024-09-17T22:41:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-17T22:41:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_recovery_priority": "5", 2024-09-17T22:41:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-17T22:41:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_recovery_sleep": "0.000000", 2024-09-17T22:41:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-17T22:41:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-17T22:41:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-17T22:41:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_repair_during_recovery": "false", 2024-09-17T22:41:17.169 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_requested_scrub_priority": "120", 2024-09-17T22:41:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-17T22:41:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-17T22:41:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_auto_repair": "false", 2024-09-17T22:41:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-17T22:41:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-17T22:41:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_begin_hour": "0", 2024-09-17T22:41:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_begin_week_day": "0", 2024-09-17T22:41:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_chunk_max": "25", 2024-09-17T22:41:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_chunk_min": "5", 2024-09-17T22:41:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_cost": "52428800", 2024-09-17T22:41:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-17T22:41:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_during_recovery": "false", 2024-09-17T22:41:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_end_hour": "0", 2024-09-17T22:41:17.170 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_end_week_day": "0", 2024-09-17T22:41:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_event_cost": "4096", 2024-09-17T22:41:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-17T22:41:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-17T22:41:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_invalid_stats": "true", 2024-09-17T22:41:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-17T22:41:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-17T22:41:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_max_preemptions": "5", 2024-09-17T22:41:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-17T22:41:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_priority": "5", 2024-09-17T22:41:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_scrub_sleep": "0.000000", 2024-09-17T22:41:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-17T22:41:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-17T22:41:17.171 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-17T22:41:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_skip_data_digest": "false", 2024-09-17T22:41:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_smart_report_timeout": "5", 2024-09-17T22:41:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_snap_trim_cost": "1048576", 2024-09-17T22:41:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_snap_trim_priority": "5", 2024-09-17T22:41:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-17T22:41:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-17T22:41:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-17T22:41:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-17T22:41:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-17T22:41:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-17T22:41:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-17T22:41:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_target_transaction_size": "30", 2024-09-17T22:41:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-17T22:41:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-17T22:41:17.172 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-17T22:41:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-17T22:41:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-17T22:41:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-17T22:41:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-17T22:41:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-17T22:41:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-17T22:41:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-17T22:41:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_tracing": "false", 2024-09-17T22:41:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_use_stale_snap": "false", 2024-09-17T22:41:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-17T22:41:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "osdc_blkin_trace_all": "false", 2024-09-17T22:41:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "paxos_kill_at": "0", 2024-09-17T22:41:17.173 INFO:teuthology.orchestra.run.smithi115.stdout: "paxos_max_join_drift": "10", 2024-09-17T22:41:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: "paxos_min": "500", 2024-09-17T22:41:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: "paxos_min_wait": "0.050000", 2024-09-17T22:41:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: "paxos_propose_interval": "1.000000", 2024-09-17T22:41:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: "paxos_service_trim_max": "500", 2024-09-17T22:41:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-17T22:41:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: "paxos_service_trim_min": "250", 2024-09-17T22:41:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: "paxos_stash_full_interval": "25", 2024-09-17T22:41:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: "paxos_trim_max": "500", 2024-09-17T22:41:17.174 INFO:teuthology.orchestra.run.smithi115.stdout: "paxos_trim_min": "250", 2024-09-17T22:41:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "perf": "true", 2024-09-17T22:41:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-17T22:41:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-17T22:41:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-17T22:41:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "-", 2024-09-17T22:41:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrv": "", 2024-09-17T22:41:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "public_bind_addr": "-", 2024-09-17T22:41:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "public_network": "", 2024-09-17T22:41:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "public_network_interface": "", 2024-09-17T22:41:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "qat_compressor_busy_polling": "false", 2024-09-17T22:41:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "qat_compressor_enabled": "false", 2024-09-17T22:41:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "qat_compressor_session_max_number": "256", 2024-09-17T22:41:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "rados_mon_op_timeout": "0", 2024-09-17T22:41:17.175 INFO:teuthology.orchestra.run.smithi115.stdout: "rados_osd_op_timeout": "0", 2024-09-17T22:41:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: "rados_tracing": "false", 2024-09-17T22:41:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_atime_update_interval": "60", 2024-09-17T22:41:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-17T22:41:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_balance_parent_reads": "false", 2024-09-17T22:41:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_balance_snap_reads": "false", 2024-09-17T22:41:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_blkin_trace_all": "false", 2024-09-17T22:41:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-17T22:41:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-17T22:41:17.176 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_cache": "true", 2024-09-17T22:41:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-17T22:41:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-17T22:41:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-17T22:41:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-17T22:41:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_cache_policy": "writearound", 2024-09-17T22:41:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_cache_size": "33554432", 2024-09-17T22:41:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-17T22:41:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-17T22:41:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_clone_copy_on_read": "false", 2024-09-17T22:41:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_compression_hint": "none", 2024-09-17T22:41:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_concurrent_management_ops": "10", 2024-09-17T22:41:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-17T22:41:17.177 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_default_clone_format": "auto", 2024-09-17T22:41:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_default_data_pool": "", 2024-09-17T22:41:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_default_features": "61", 2024-09-17T22:41:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_default_format": "2", 2024-09-17T22:41:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_default_map_options": "", 2024-09-17T22:41:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_default_order": "22", 2024-09-17T22:41:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_default_pool": "rbd", 2024-09-17T22:41:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-17T22:41:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_default_stripe_count": "0", 2024-09-17T22:41:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_default_stripe_unit": "0", 2024-09-17T22:41:17.178 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-17T22:41:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-17T22:41:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-17T22:41:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_enable_alloc_hint": "true", 2024-09-17T22:41:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-17T22:41:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_io_scheduler": "simple", 2024-09-17T22:41:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-17T22:41:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-17T22:41:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-17T22:41:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-17T22:41:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-17T22:41:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-17T22:41:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-17T22:41:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-17T22:41:17.179 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-17T22:41:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_journal_order": "24", 2024-09-17T22:41:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_journal_pool": "", 2024-09-17T22:41:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_journal_splay_width": "4", 2024-09-17T22:41:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_localize_parent_reads": "false", 2024-09-17T22:41:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_localize_snap_reads": "false", 2024-09-17T22:41:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-17T22:41:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-17T22:41:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-17T22:41:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-17T22:41:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-17T22:41:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-17T22:41:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-17T22:41:17.180 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-17T22:41:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-17T22:41:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-17T22:41:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-17T22:41:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-17T22:41:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-17T22:41:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-17T22:41:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-17T22:41:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-17T22:41:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-17T22:41:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-17T22:41:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-17T22:41:17.181 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-17T22:41:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-17T22:41:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-17T22:41:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-17T22:41:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-17T22:41:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-17T22:41:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-17T22:41:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-17T22:41:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-17T22:41:17.182 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-17T22:41:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-17T22:41:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-17T22:41:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_mtime_update_interval": "60", 2024-09-17T22:41:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_non_blocking_aio": "true", 2024-09-17T22:41:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_op_thread_timeout": "60", 2024-09-17T22:41:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_op_threads": "1", 2024-09-17T22:41:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_parent_cache_enabled": "false", 2024-09-17T22:41:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-17T22:41:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-17T22:41:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-17T22:41:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_plugins": "", 2024-09-17T22:41:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_bps_burst": "0", 2024-09-17T22:41:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-17T22:41:17.183 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_bps_limit": "0", 2024-09-17T22:41:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_exclude_ops": "0", 2024-09-17T22:41:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_iops_burst": "0", 2024-09-17T22:41:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-17T22:41:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_iops_limit": "0", 2024-09-17T22:41:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-17T22:41:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-17T22:41:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-17T22:41:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-17T22:41:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-17T22:41:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-17T22:41:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-17T22:41:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-17T22:41:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-17T22:41:17.184 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-17T22:41:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-17T22:41:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-17T22:41:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-17T22:41:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-17T22:41:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_read_from_replica_policy": "default", 2024-09-17T22:41:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-17T22:41:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-17T22:41:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-17T22:41:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-17T22:41:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_skip_partial_discard": "true", 2024-09-17T22:41:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-17T22:41:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_tracing": "false", 2024-09-17T22:41:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_validate_names": "true", 2024-09-17T22:41:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd_validate_pool": "true", 2024-09-17T22:41:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "restapi_base_url": "", 2024-09-17T22:41:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "restapi_log_level": "", 2024-09-17T22:41:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-17T22:41:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-17T22:41:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_acl_grants_max_num": "100", 2024-09-17T22:41:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_admin_entry": "admin", 2024-09-17T22:41:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-17T22:41:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_backend_store": "rados", 2024-09-17T22:41:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_barbican_url": "", 2024-09-17T22:41:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_beast_enable_async": "true", 2024-09-17T22:41:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_bucket_counters_cache": "false", 2024-09-17T22:41:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-17T22:41:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-17T22:41:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-17T22:41:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-17T22:41:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-17T22:41:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-17T22:41:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-17T22:41:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-17T22:41:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_cache_enabled": "true", 2024-09-17T22:41:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_cache_expiry_interval": "900", 2024-09-17T22:41:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_cache_lru_size": "25000", 2024-09-17T22:41:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_config_store": "rados", 2024-09-17T22:41:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_content_length_compat": "false", 2024-09-17T22:41:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_copy_obj_progress": "true", 2024-09-17T22:41:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-17T22:41:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_cors_rules_max_num": "100", 2024-09-17T22:41:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_cross_domain_policy": "", 2024-09-17T22:41:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-17T22:41:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_kmip_addr": "", 2024-09-17T22:41:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-17T22:41:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-17T22:41:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-17T22:41:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-17T22:41:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_kmip_password": "", 2024-09-17T22:41:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-17T22:41:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_kmip_username": "", 2024-09-17T22:41:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_require_ssl": "true", 2024-09-17T22:41:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-17T22:41:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-17T22:41:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-17T22:41:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-17T22:41:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-17T22:41:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-17T22:41:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-17T22:41:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-17T22:41:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-17T22:41:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-17T22:41:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-17T22:41:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-17T22:41:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-17T22:41:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-17T22:41:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-17T22:41:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_vault_addr": "", 2024-09-17T22:41:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_vault_auth": "token", 2024-09-17T22:41:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_vault_namespace": "", 2024-09-17T22:41:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_vault_prefix": "", 2024-09-17T22:41:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-17T22:41:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-17T22:41:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-17T22:41:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-17T22:41:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_vault_token_file": "", 2024-09-17T22:41:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-17T22:41:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_curl_buffersize": "524288", 2024-09-17T22:41:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-17T22:41:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_curl_low_speed_time": "300", 2024-09-17T22:41:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-17T22:41:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-17T22:41:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-17T22:41:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-17T22:41:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-17T22:41:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-17T22:41:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-17T22:41:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-17T22:41:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-17T22:41:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-17T22:41:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-17T22:41:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_d4n_port": "6379", 2024-09-17T22:41:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-17T22:41:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_data_log_changes_size": "1000", 2024-09-17T22:41:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_data_log_num_shards": "128", 2024-09-17T22:41:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-17T22:41:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_data_log_window": "30", 2024-09-17T22:41:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-17T22:41:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-17T22:41:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-17T22:41:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-17T22:41:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-17T22:41:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-17T22:41:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-17T22:41:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-17T22:41:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-17T22:41:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-17T22:41:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-17T22:41:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-17T22:41:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-17T22:41:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-17T22:41:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-17T22:41:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-17T22:41:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-17T22:41:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-17T22:41:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-17T22:41:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-17T22:41:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-17T22:41:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-17T22:41:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-17T22:41:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-17T22:41:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-17T22:41:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_dns_name": "", 2024-09-17T22:41:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_dns_s3website_name": "", 2024-09-17T22:41:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_dynamic_resharding": "true", 2024-09-17T22:41:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-17T22:41:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_enable_gc_threads": "true", 2024-09-17T22:41:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_enable_lc_threads": "true", 2024-09-17T22:41:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_enable_ops_log": "false", 2024-09-17T22:41:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_enable_quota_threads": "true", 2024-09-17T22:41:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_enable_static_website": "false", 2024-09-17T22:41:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_enable_usage_log": "false", 2024-09-17T22:41:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_enforce_swift_acls": "true", 2024-09-17T22:41:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_exit_timeout_secs": "120", 2024-09-17T22:41:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_expose_bucket": "false", 2024-09-17T22:41:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_extended_http_attrs": "", 2024-09-17T22:41:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_filter": "none", 2024-09-17T22:41:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-17T22:41:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_frontends": "beast port=7480", 2024-09-17T22:41:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-17T22:41:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_gc_max_deferred": "50", 2024-09-17T22:41:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-17T22:41:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_gc_max_objs": "32", 2024-09-17T22:41:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-17T22:41:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-17T22:41:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-17T22:41:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-17T22:41:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_gc_processor_period": "3600", 2024-09-17T22:41:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-17T22:41:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-17T22:41:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-17T22:41:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-17T22:41:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_init_timeout": "300", 2024-09-17T22:41:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-17T22:41:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-17T22:41:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_kafka_connection_idle": "300", 2024-09-17T22:41:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-17T22:41:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-17T22:41:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-17T22:41:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-17T22:41:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-17T22:41:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_admin_domain": "", 2024-09-17T22:41:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_admin_password": "", 2024-09-17T22:41:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_admin_password_path": "", 2024-09-17T22:41:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_admin_project": "", 2024-09-17T22:41:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_admin_tenant": "", 2024-09-17T22:41:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_admin_token": "", 2024-09-17T22:41:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_admin_token_path": "", 2024-09-17T22:41:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_admin_user": "", 2024-09-17T22:41:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_api_version": "2", 2024-09-17T22:41:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_barbican_domain": "", 2024-09-17T22:41:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_barbican_password": "", 2024-09-17T22:41:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_barbican_project": "", 2024-09-17T22:41:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-17T22:41:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_barbican_user": "", 2024-09-17T22:41:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-17T22:41:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-17T22:41:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-17T22:41:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-17T22:41:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-17T22:41:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_url": "", 2024-09-17T22:41:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-17T22:41:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_lc_debug_interval": "-1", 2024-09-17T22:41:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_lc_lock_max_time": "90", 2024-09-17T22:41:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_lc_max_objs": "32", 2024-09-17T22:41:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_lc_max_rules": "1000", 2024-09-17T22:41:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_lc_max_worker": "3", 2024-09-17T22:41:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-17T22:41:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_lc_thread_delay": "0", 2024-09-17T22:41:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-17T22:41:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_ldap_dnattr": "uid", 2024-09-17T22:41:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-17T22:41:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_ldap_searchfilter": "", 2024-09-17T22:41:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-17T22:41:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_ldap_uri": "", 2024-09-17T22:41:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-17T22:41:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-17T22:41:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-17T22:41:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_log_http_headers": "", 2024-09-17T22:41:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-17T22:41:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-17T22:41:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_log_object_name_utc": "false", 2024-09-17T22:41:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-17T22:41:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-17T22:41:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_max_attr_name_len": "0", 2024-09-17T22:41:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_max_attr_size": "0", 2024-09-17T22:41:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-17T22:41:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_max_chunk_size": "4194304", 2024-09-17T22:41:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-17T22:41:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-17T22:41:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-17T22:41:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_max_listing_results": "1000", 2024-09-17T22:41:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_max_notify_retries": "10", 2024-09-17T22:41:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-17T22:41:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_max_put_param_size": "1048576", 2024-09-17T22:41:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_max_put_size": "5368709120", 2024-09-17T22:41:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_max_slo_entries": "1000", 2024-09-17T22:41:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_md_log_max_shards": "64", 2024-09-17T22:41:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-17T22:41:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-17T22:41:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-17T22:41:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-17T22:41:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_mp_lock_max_time": "600", 2024-09-17T22:41:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-17T22:41:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-17T22:41:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-17T22:41:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-17T22:41:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-17T22:41:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-17T22:41:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-17T22:41:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-17T22:41:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_nfs_max_gc": "300", 2024-09-17T22:41:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-17T22:41:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-17T22:41:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-17T22:41:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-17T22:41:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-17T22:41:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-17T22:41:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-17T22:41:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_num_async_rados_threads": "32", 2024-09-17T22:41:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_num_control_oids": "8", 2024-09-17T22:41:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_numa_node": "-1", 2024-09-17T22:41:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-17T22:41:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-17T22:41:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_objexp_chunk_size": "100", 2024-09-17T22:41:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_objexp_gc_interval": "600", 2024-09-17T22:41:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-17T22:41:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-17T22:41:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-17T22:41:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_op_thread_timeout": "600", 2024-09-17T22:41:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_op_tracing": "false", 2024-09-17T22:41:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_opa_token": "", 2024-09-17T22:41:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_opa_url": "", 2024-09-17T22:41:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_opa_verify_ssl": "true", 2024-09-17T22:41:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-17T22:41:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-17T22:41:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_ops_log_rados": "false", 2024-09-17T22:41:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_ops_log_socket_path": "", 2024-09-17T22:41:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-17T22:41:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-17T22:41:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-17T22:41:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_period_push_interval": "2.000000", 2024-09-17T22:41:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-17T22:41:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-17T22:41:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-17T22:41:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-17T22:41:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_posix_cache_lanes": "3", 2024-09-17T22:41:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-17T22:41:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-17T22:41:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_posix_cache_partitions": "3", 2024-09-17T22:41:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-17T22:41:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_print_continue": "true", 2024-09-17T22:41:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-17T22:41:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-17T22:41:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-17T22:41:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-17T22:41:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-17T22:41:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_rados_tracing": "false", 2024-09-17T22:41:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_realm": "", 2024-09-17T22:41:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_realm_id": "", 2024-09-17T22:41:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-17T22:41:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_region": "", 2024-09-17T22:41:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-17T22:41:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-17T22:41:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-17T22:41:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-17T22:41:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_request_uri": "", 2024-09-17T22:41:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_reshard_batch_size": "64", 2024-09-17T22:41:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-17T22:41:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_reshard_max_aio": "128", 2024-09-17T22:41:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_reshard_num_logs": "16", 2024-09-17T22:41:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_reshard_thread_interval": "600", 2024-09-17T22:41:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_resolve_cname": "false", 2024-09-17T22:41:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-17T22:41:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_run_sync_thread": "true", 2024-09-17T22:41:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-17T22:41:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-17T22:41:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-17T22:41:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-17T22:41:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-17T22:41:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-17T22:41:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-17T22:41:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-17T22:41:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-17T22:41:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_scheduler_type": "throttler", 2024-09-17T22:41:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_script_uri": "", 2024-09-17T22:41:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_service_provider_name": "", 2024-09-17T22:41:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-17T22:41:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_sts_client_id": "", 2024-09-17T22:41:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_sts_client_secret": "", 2024-09-17T22:41:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_sts_entry": "sts", 2024-09-17T22:41:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_sts_key": "sts", 2024-09-17T22:41:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-17T22:41:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_sts_min_session_duration": "900", 2024-09-17T22:41:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_sts_token_introspection_url": "", 2024-09-17T22:41:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_swift_account_in_url": "false", 2024-09-17T22:41:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_swift_auth_entry": "auth", 2024-09-17T22:41:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_swift_auth_url": "", 2024-09-17T22:41:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_swift_custom_header": "", 2024-09-17T22:41:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-17T22:41:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_swift_need_stats": "true", 2024-09-17T22:41:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_swift_tenant_name": "", 2024-09-17T22:41:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_swift_token_expiration": "86400", 2024-09-17T22:41:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_swift_url": "", 2024-09-17T22:41:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_swift_url_prefix": "swift", 2024-09-17T22:41:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-17T22:41:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-17T22:41:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-17T22:41:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_sync_lease_period": "120", 2024-09-17T22:41:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-17T22:41:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-17T22:41:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-17T22:41:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-17T22:41:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-17T22:41:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-17T22:41:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-17T22:41:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-17T22:41:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-17T22:41:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_thread_pool_size": "512", 2024-09-17T22:41:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-17T22:41:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-17T22:41:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-17T22:41:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-17T22:41:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_torrent_comment": "", 2024-09-17T22:41:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_torrent_createby": "", 2024-09-17T22:41:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_torrent_encoding": "", 2024-09-17T22:41:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_torrent_flag": "false", 2024-09-17T22:41:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-17T22:41:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_torrent_origin": "", 2024-09-17T22:41:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-17T22:41:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_torrent_tracker": "", 2024-09-17T22:41:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_trust_forwarded_https": "false", 2024-09-17T22:41:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-17T22:41:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-17T22:41:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_usage_max_shards": "32", 2024-09-17T22:41:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_usage_max_user_shards": "1", 2024-09-17T22:41:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_use_opa_authz": "false", 2024-09-17T22:41:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_user_counters_cache": "false", 2024-09-17T22:41:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-17T22:41:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-17T22:41:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-17T22:41:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_user_max_buckets": "1000", 2024-09-17T22:41:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_user_policies_max_num": "100", 2024-09-17T22:41:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-17T22:41:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-17T22:41:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-17T22:41:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-17T22:41:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_user_unique_email": "true", 2024-09-17T22:41:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_verify_ssl": "true", 2024-09-17T22:41:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-17T22:41:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_zone": "", 2024-09-17T22:41:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_zone_id": "", 2024-09-17T22:41:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-17T22:41:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_zonegroup": "", 2024-09-17T22:41:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_zonegroup_id": "", 2024-09-17T22:41:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-17T22:41:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "rgwlc_auto_session_clear": "true", 2024-09-17T22:41:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-17T22:41:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_block_size": "4096", 2024-09-17T22:41:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-17T22:41:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-17T22:41:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-17T22:41:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-17T22:41:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-17T22:41:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_cache_size": "536870912", 2024-09-17T22:41:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-17T22:41:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-17T22:41:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-17T22:41:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-17T22:41:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-17T22:41:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-17T22:41:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-17T22:41:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-17T22:41:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_index_type": "binary_search", 2024-09-17T22:41:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-17T22:41:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-17T22:41:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_partition_filters": "false", 2024-09-17T22:41:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_perf": "false", 2024-09-17T22:41:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-17T22:41:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-17T22:41:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-17T22:41:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "run_dir": "/var/run/ceph", 2024-09-17T22:41:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_block_create": "true", 2024-09-17T22:41:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_cache_lru_size": "67108864", 2024-09-17T22:41:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-17T22:41:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-17T22:41:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_default_max_object_size": "16777216", 2024-09-17T22:41:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-17T22:41:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_device_size": "53687091200", 2024-09-17T22:41:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-17T22:41:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_full_integrity_check": "false", 2024-09-17T22:41:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_journal_batch_capacity": "16", 2024-09-17T22:41:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-17T22:41:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-17T22:41:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-17T22:41:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_main_device_type": "SSD", 2024-09-17T22:41:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-17T22:41:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-17T22:41:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-17T22:41:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-17T22:41:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-17T22:41:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-17T22:41:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "seastore_segment_size": "67108864", 2024-09-17T22:41:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "set_keepcaps": "false", 2024-09-17T22:41:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "setgroup": "", 2024-09-17T22:41:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "setuser": "", 2024-09-17T22:41:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "setuser_match_path": "", 2024-09-17T22:41:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-17T22:41:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "thp": "false", 2024-09-17T22:41:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "threadpool_default_timeout": "60", 2024-09-17T22:41:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-17T22:41:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "throttler_perf_counter": "true" 2024-09-17T22:41:17.217 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:17.219 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-17T22:41:17.331 INFO:teuthology.orchestra.run.smithi115.stdout:mon.a: {} 2024-09-17T22:41:17.331 INFO:teuthology.orchestra.run.smithi115.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T22:41:17.341 INFO:teuthology.orchestra.run.smithi115.stdout:mon.b: {} 2024-09-17T22:41:17.341 INFO:teuthology.orchestra.run.smithi115.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T22:41:17.349 INFO:teuthology.orchestra.run.smithi115.stdout:mon.c: {} 2024-09-17T22:41:17.350 INFO:teuthology.orchestra.run.smithi115.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T22:41:17.361 DEBUG:teuthology.orchestra.run.smithi115:> type ceph-objectstore-tool 2024-09-17T22:41:17.375 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T22:41:17.375 DEBUG:teuthology.orchestra.run.smithi129:> type ceph-objectstore-tool 2024-09-17T22:41:17.393 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T22:41:17.393 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-17T22:41:17.403 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-17T22:41:17.403 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-17T22:41:17.403 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:41:17.403 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-17T22:41:17.404 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-17T22:41:17.404 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-17T22:41:17.404 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:41:17.405 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-17T22:41:17.405 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:41:17.405 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-17T22:41:17.405 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:41:17.406 DEBUG:teuthology.orchestra.run.smithi115:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-17T22:41:17.417 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:17.414+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:17.457 INFO:teuthology.orchestra.run.smithi115.stdout:mapped writes DISABLED 2024-09-17T22:41:17.457 INFO:teuthology.orchestra.run.smithi115.stdout:mapped reads DISABLED 2024-09-17T22:41:17.458 INFO:teuthology.orchestra.run.smithi115.stdout:Seed set to 9361 2024-09-17T22:41:17.506 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:17.504+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:17.542 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:17.543 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:17.543 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:41:17.543 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:17.552 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0: {} 2024-09-17T22:41:17.552 INFO:teuthology.orchestra.run.smithi115.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:41:17.554 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:41:17.571 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:41:17.571 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:41:17.588 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:41:17.588 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:41:17.606 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:41:17.606 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:41:17.607 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:17.604+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:17.623 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:41:17.624 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:41:17.641 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:41:17.641 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:41:17.659 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:41:17.659 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:41:17.660 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:17.660 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:17.669 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:41:17.676 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:41:17.677 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:41:17.708 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:17.705+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:17.779 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:17.780 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:17.788 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:41:17.832 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:17.829+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:17.891 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:17.891 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:17.899 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:41:17.940 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:17.938+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:18.011 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:18.009+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:18.042 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:18.043 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:18.051 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:41:18.112 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:18.109+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:18.154 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:18.154 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:18.163 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:41:18.247 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:18.245+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:18.307 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:18.307 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:18.314 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:18.312+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:18.316 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:41:18.415 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:18.412+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:18.431 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T22:41:18.431 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is set 2024-09-17T22:41:18.451 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-17T22:41:18.467 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:18.467 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:18.477 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:41:18.520 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:18.517+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:18.617 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:18.614+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:18.624 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:18.624 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:18.633 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:41:18.718 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:18.715+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:18.777 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:18.777 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:18.786 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:41:18.826 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:18.824+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:18.919 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:18.917+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:18.928 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:18.928 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:18.937 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:41:19.020 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:19.017+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:19.038 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:19.039 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:19.047 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:41:19.120 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:19.118+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:19.161 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:19.162 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:19.171 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:41:19.221 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:19.219+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:19.274 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:19.274 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:19.283 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:41:19.322 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:19.320+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:19.388 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:19.388 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:19.397 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:41:19.423 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:19.421+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:19.439 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T22:41:19.462 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-17T22:41:19.507 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:19.507 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:19.515 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:41:19.546 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:19.544+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:19.618 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:19.618 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:19.624 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:19.622+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:19.627 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:41:19.689 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:41:19.726 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:19.723+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:19.730 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:19.730 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:19.739 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:41:19.808 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0: {} 2024-09-17T22:41:19.809 INFO:teuthology.orchestra.run.smithi115.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:41:19.826 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:41:19.826 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:41:19.826 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:19.824+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:19.841 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:41:19.841 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:41:19.857 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:41:19.857 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:41:19.873 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:41:19.873 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:41:19.883 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:19.884 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:19.890 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:41:19.890 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:41:19.893 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:41:19.906 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:41:19.906 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:41:19.923 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:41:19.923 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:41:19.934 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:19.932+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:20.028 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:20.025+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:20.037 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:20.037 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:20.037 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:20.037 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:20.037 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:20.046 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:41:20.130 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:20.128+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:20.191 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:20.191 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:20.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:20.191 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:20.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.191 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4229.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-17T22:41:20.191 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:10.399997+0000", 2024-09-17T22:41:20.191 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.7889561470000004, 2024-09-17T22:41:20.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00093779499999999995, 2024-09-17T22:41:20.191 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:20.191 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:20.192 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:20.192 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:20.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:20.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:10.399997+0000", 2024-09-17T22:41:20.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:20.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:20.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:20.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:10.399997+0000", 2024-09-17T22:41:20.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:20.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:20.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:20.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:10.399997+0000", 2024-09-17T22:41:20.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.85e-07 2024-09-17T22:41:20.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:20.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:20.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:10.400003+0000", 2024-09-17T22:41:20.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6359999999999999e-06 2024-09-17T22:41:20.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:20.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:20.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:10.400017+0000", 2024-09-17T22:41:20.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.4752e-05 2024-09-17T22:41:20.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:20.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:20.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:10.400029+0000", 2024-09-17T22:41:20.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1122e-05 2024-09-17T22:41:20.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:20.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:20.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:10.400111+0000", 2024-09-17T22:41:20.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2917999999999995e-05 2024-09-17T22:41:20.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:20.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:20.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:10.400141+0000", 2024-09-17T22:41:20.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9113999999999999e-05 2024-09-17T22:41:20.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:20.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:20.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:10.400895+0000", 2024-09-17T22:41:20.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00075445899999999997 2024-09-17T22:41:20.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:20.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:20.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:10.400935+0000", 2024-09-17T22:41:20.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9609e-05 2024-09-17T22:41:20.195 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:20.195 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:20.195 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:20.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:20.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.195 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4229.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-17T22:41:20.196 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:11.399948+0000", 2024-09-17T22:41:20.196 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.7890050290000001, 2024-09-17T22:41:20.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00094461999999999996, 2024-09-17T22:41:20.196 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:20.196 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:20.196 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:20.196 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:20.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:20.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:11.399948+0000", 2024-09-17T22:41:20.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:20.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:20.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:20.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:11.399948+0000", 2024-09-17T22:41:20.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:20.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:20.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:20.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:11.399948+0000", 2024-09-17T22:41:20.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.92e-07 2024-09-17T22:41:20.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:20.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:20.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:11.399960+0000", 2024-09-17T22:41:20.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.17e-05 2024-09-17T22:41:20.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:20.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:20.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:11.399988+0000", 2024-09-17T22:41:20.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8e-05 2024-09-17T22:41:20.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:20.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:20.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:11.399999+0000", 2024-09-17T22:41:20.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1551e-05 2024-09-17T22:41:20.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:20.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:20.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:11.400065+0000", 2024-09-17T22:41:20.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5518999999999994e-05 2024-09-17T22:41:20.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:20.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:20.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:11.400097+0000", 2024-09-17T22:41:20.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2468000000000002e-05 2024-09-17T22:41:20.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:20.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:20.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:11.400851+0000", 2024-09-17T22:41:20.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00075310600000000002 2024-09-17T22:41:20.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:20.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:20.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:20.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:11.400893+0000", 2024-09-17T22:41:20.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2083999999999999e-05 2024-09-17T22:41:20.200 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:20.200 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:20.200 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:20.200 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:20.200 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:20.200 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:20.201 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:41:20.230 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:20.228+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:20.331 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:20.329+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:20.345 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:20.345 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:20.345 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:41:20.345 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:20.354 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:41:20.432 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:20.430+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:20.443 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T22:41:20.454 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:41:20.458 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-17T22:41:20.498 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:20.498 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:20.498 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:20.499 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:20.499 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:20.507 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:41:20.554 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:20.552+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:20.611 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:20.611 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:20.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:20.611 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [] 2024-09-17T22:41:20.611 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:20.619 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:41:20.663 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:20.661+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:20.724 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:20.724 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:20.724 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:41:20.724 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:20.733 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:41:20.775 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:20.772+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:20.835 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:20.833+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:20.876 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:20.876 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:20.876 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:41:20.876 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:41:20.876 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:20.885 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:41:20.936 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:20.934+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:20.989 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:20.989 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:41:20.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:41:20.989 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [] 2024-09-17T22:41:20.989 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:20.998 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:41:21.037 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:21.034+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:21.100 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:21.100 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:21.100 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:41:21.100 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:21.108 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:41:21.153 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:21.151+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:21.212 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:21.212 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:21.212 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:41:21.212 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:41:21.212 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:21.221 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:41:21.238 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:21.236+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:21.339 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:21.336+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:21.367 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:41:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:41:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:41:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:41:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.004444328999998, 2024-09-17T22:41:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.249158453, 2024-09-17T22:41:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.367 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-17T22:41:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:41:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:41:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358680+0000", 2024-09-17T22:41:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.592e-06 2024-09-17T22:41:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358688+0000", 2024-09-17T22:41:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1610000000000003e-06 2024-09-17T22:41:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358697+0000", 2024-09-17T22:41:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2580000000000001e-06 2024-09-17T22:41:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358706+0000", 2024-09-17T22:41:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1640000000000007e-06 2024-09-17T22:41:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358766+0000", 2024-09-17T22:41:21.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9893999999999999e-05 2024-09-17T22:41:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:41:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358778+0000", 2024-09-17T22:41:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1148999999999999e-05 2024-09-17T22:41:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606347+0000", 2024-09-17T22:41:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.247569971 2024-09-17T22:41:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606533+0000", 2024-09-17T22:41:21.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018599400000000001 2024-09-17T22:41:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606752+0000", 2024-09-17T22:41:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000218373 2024-09-17T22:41:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607455+0000", 2024-09-17T22:41:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070311799999999997 2024-09-17T22:41:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607735+0000", 2024-09-17T22:41:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027972300000000001 2024-09-17T22:41:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607793+0000", 2024-09-17T22:41:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8529999999999997e-05 2024-09-17T22:41:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607837+0000", 2024-09-17T22:41:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3526000000000003e-05 2024-09-17T22:41:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.638594+0000", 2024-09-17T22:41:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.724528788000001, 2024-09-17T22:41:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011505949, 2024-09-17T22:41:21.374 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 36 2024-09-17T22:41:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638594+0000", 2024-09-17T22:41:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638594+0000", 2024-09-17T22:41:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638594+0000", 2024-09-17T22:41:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3299999999999998e-07 2024-09-17T22:41:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638597+0000", 2024-09-17T22:41:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4099999999999998e-06 2024-09-17T22:41:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638601+0000", 2024-09-17T22:41:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3710000000000002e-06 2024-09-17T22:41:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638607+0000", 2024-09-17T22:41:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5060000000000001e-06 2024-09-17T22:41:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.641700+0000", 2024-09-17T22:41:21.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030928309999999999 2024-09-17T22:41:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.641778+0000", 2024-09-17T22:41:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7735e-05 2024-09-17T22:41:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.641910+0000", 2024-09-17T22:41:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013224100000000001 2024-09-17T22:41:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.644227+0000", 2024-09-17T22:41:21.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002316987 2024-09-17T22:41:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.650014+0000", 2024-09-17T22:41:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0057866289999999997 2024-09-17T22:41:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.650063+0000", 2024-09-17T22:41:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8980999999999997e-05 2024-09-17T22:41:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.650100+0000", 2024-09-17T22:41:21.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6925000000000003e-05 2024-09-17T22:41:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.638612+0000", 2024-09-17T22:41:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.724510311, 2024-09-17T22:41:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011569195000000001, 2024-09-17T22:41:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 37 2024-09-17T22:41:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638612+0000", 2024-09-17T22:41:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638612+0000", 2024-09-17T22:41:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638613+0000", 2024-09-17T22:41:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5899999999999998e-07 2024-09-17T22:41:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638615+0000", 2024-09-17T22:41:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2560000000000001e-06 2024-09-17T22:41:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638619+0000", 2024-09-17T22:41:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9509999999999999e-06 2024-09-17T22:41:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638622+0000", 2024-09-17T22:41:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8200000000000001e-06 2024-09-17T22:41:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.642217+0000", 2024-09-17T22:41:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035953180000000001 2024-09-17T22:41:21.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.642272+0000", 2024-09-17T22:41:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5096999999999998e-05 2024-09-17T22:41:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.642361+0000", 2024-09-17T22:41:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8518999999999997e-05 2024-09-17T22:41:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.644493+0000", 2024-09-17T22:41:21.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021328190000000002 2024-09-17T22:41:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.650108+0000", 2024-09-17T22:41:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056144259999999996 2024-09-17T22:41:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.650149+0000", 2024-09-17T22:41:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1511999999999998e-05 2024-09-17T22:41:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.650182+0000", 2024-09-17T22:41:21.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2218000000000002e-05 2024-09-17T22:41:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.691247+0000", 2024-09-17T22:41:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.671876001000001, 2024-09-17T22:41:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0075837589999999998, 2024-09-17T22:41:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 70 2024-09-17T22:41:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.691247+0000", 2024-09-17T22:41:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.691247+0000", 2024-09-17T22:41:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.691247+0000", 2024-09-17T22:41:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5699999999999998e-07 2024-09-17T22:41:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.691250+0000", 2024-09-17T22:41:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.525e-06 2024-09-17T22:41:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.691253+0000", 2024-09-17T22:41:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9060000000000004e-06 2024-09-17T22:41:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.691256+0000", 2024-09-17T22:41:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8289999999999999e-06 2024-09-17T22:41:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.692763+0000", 2024-09-17T22:41:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015062599999999999 2024-09-17T22:41:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.692810+0000", 2024-09-17T22:41:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7194000000000002e-05 2024-09-17T22:41:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.692887+0000", 2024-09-17T22:41:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7310000000000004e-05 2024-09-17T22:41:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.696280+0000", 2024-09-17T22:41:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033933420000000001 2024-09-17T22:41:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.698742+0000", 2024-09-17T22:41:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024613650000000001 2024-09-17T22:41:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.698790+0000", 2024-09-17T22:41:21.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8531e-05 2024-09-17T22:41:21.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.698830+0000", 2024-09-17T22:41:21.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0139999999999999e-05 2024-09-17T22:41:21.392 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.392 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.392 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.902335+0000", 2024-09-17T22:41:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.460787632999999, 2024-09-17T22:41:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011889003, 2024-09-17T22:41:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 112 2024-09-17T22:41:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902335+0000", 2024-09-17T22:41:21.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902335+0000", 2024-09-17T22:41:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902335+0000", 2024-09-17T22:41:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0699999999999999e-07 2024-09-17T22:41:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902413+0000", 2024-09-17T22:41:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8279e-05 2024-09-17T22:41:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902419+0000", 2024-09-17T22:41:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.677e-06 2024-09-17T22:41:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902422+0000", 2024-09-17T22:41:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2760000000000001e-06 2024-09-17T22:41:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.905774+0000", 2024-09-17T22:41:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033517809999999999 2024-09-17T22:41:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.905816+0000", 2024-09-17T22:41:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2265999999999998e-05 2024-09-17T22:41:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.905891+0000", 2024-09-17T22:41:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.47e-05 2024-09-17T22:41:21.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.911676+0000", 2024-09-17T22:41:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0057843529999999999 2024-09-17T22:41:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914066+0000", 2024-09-17T22:41:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023904439999999998 2024-09-17T22:41:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914191+0000", 2024-09-17T22:41:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012519000000000001 2024-09-17T22:41:21.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914224+0000", 2024-09-17T22:41:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2830000000000002e-05 2024-09-17T22:41:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.902440+0000", 2024-09-17T22:41:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.460682619, 2024-09-17T22:41:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011895823999999999, 2024-09-17T22:41:21.399 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113 2024-09-17T22:41:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902440+0000", 2024-09-17T22:41:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902440+0000", 2024-09-17T22:41:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902440+0000", 2024-09-17T22:41:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-09-17T22:41:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902443+0000", 2024-09-17T22:41:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1229999999999998e-06 2024-09-17T22:41:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902453+0000", 2024-09-17T22:41:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0380999999999999e-05 2024-09-17T22:41:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902456+0000", 2024-09-17T22:41:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0479999999999999e-06 2024-09-17T22:41:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.906256+0000", 2024-09-17T22:41:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037996760000000001 2024-09-17T22:41:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.906328+0000", 2024-09-17T22:41:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2637000000000003e-05 2024-09-17T22:41:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.906448+0000", 2024-09-17T22:41:21.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119826 2024-09-17T22:41:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.911685+0000", 2024-09-17T22:41:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052368359999999999 2024-09-17T22:41:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914268+0000", 2024-09-17T22:41:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0025828700000000001 2024-09-17T22:41:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914299+0000", 2024-09-17T22:41:21.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0880999999999998e-05 2024-09-17T22:41:21.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914336+0000", 2024-09-17T22:41:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7156999999999997e-05 2024-09-17T22:41:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.902461+0000", 2024-09-17T22:41:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.460661734999999, 2024-09-17T22:41:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011999490999999999, 2024-09-17T22:41:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 114 2024-09-17T22:41:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.409 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902461+0000", 2024-09-17T22:41:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902461+0000", 2024-09-17T22:41:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902461+0000", 2024-09-17T22:41:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4999999999999999e-07 2024-09-17T22:41:21.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902463+0000", 2024-09-17T22:41:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.858e-06 2024-09-17T22:41:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902468+0000", 2024-09-17T22:41:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.101e-06 2024-09-17T22:41:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902471+0000", 2024-09-17T22:41:21.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7410000000000001e-06 2024-09-17T22:41:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.906768+0000", 2024-09-17T22:41:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042968069999999997 2024-09-17T22:41:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.906843+0000", 2024-09-17T22:41:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5053000000000001e-05 2024-09-17T22:41:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.906961+0000", 2024-09-17T22:41:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118513 2024-09-17T22:41:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.911689+0000", 2024-09-17T22:41:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004727373 2024-09-17T22:41:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914379+0000", 2024-09-17T22:41:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026900579999999999 2024-09-17T22:41:21.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914424+0000", 2024-09-17T22:41:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5404000000000001e-05 2024-09-17T22:41:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914460+0000", 2024-09-17T22:41:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6332999999999999e-05 2024-09-17T22:41:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.414 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.902488+0000", 2024-09-17T22:41:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.460634241000001, 2024-09-17T22:41:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012064375, 2024-09-17T22:41:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 115 2024-09-17T22:41:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902488+0000", 2024-09-17T22:41:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902488+0000", 2024-09-17T22:41:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902489+0000", 2024-09-17T22:41:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-09-17T22:41:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902537+0000", 2024-09-17T22:41:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8665999999999998e-05 2024-09-17T22:41:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902543+0000", 2024-09-17T22:41:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3040000000000004e-06 2024-09-17T22:41:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902546+0000", 2024-09-17T22:41:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1580000000000001e-06 2024-09-17T22:41:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.907325+0000", 2024-09-17T22:41:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047793779999999999 2024-09-17T22:41:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.907372+0000", 2024-09-17T22:41:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7017000000000001e-05 2024-09-17T22:41:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.907445+0000", 2024-09-17T22:41:21.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2909000000000003e-05 2024-09-17T22:41:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.911693+0000", 2024-09-17T22:41:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042477360000000002 2024-09-17T22:41:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914495+0000", 2024-09-17T22:41:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028017630000000001 2024-09-17T22:41:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914522+0000", 2024-09-17T22:41:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7231999999999999e-05 2024-09-17T22:41:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914553+0000", 2024-09-17T22:41:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0849000000000002e-05 2024-09-17T22:41:21.420 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.902552+0000", 2024-09-17T22:41:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.460571014999999, 2024-09-17T22:41:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012123254999999999, 2024-09-17T22:41:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.421 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116 2024-09-17T22:41:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902552+0000", 2024-09-17T22:41:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902552+0000", 2024-09-17T22:41:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902552+0000", 2024-09-17T22:41:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4600000000000001e-07 2024-09-17T22:41:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902554+0000", 2024-09-17T22:41:21.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9230000000000001e-06 2024-09-17T22:41:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902558+0000", 2024-09-17T22:41:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7309999999999999e-06 2024-09-17T22:41:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902560+0000", 2024-09-17T22:41:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.757e-06 2024-09-17T22:41:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.907809+0000", 2024-09-17T22:41:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052484610000000003 2024-09-17T22:41:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.907877+0000", 2024-09-17T22:41:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7947000000000005e-05 2024-09-17T22:41:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.907995+0000", 2024-09-17T22:41:21.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011783 2024-09-17T22:41:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.911697+0000", 2024-09-17T22:41:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037020629999999998 2024-09-17T22:41:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914595+0000", 2024-09-17T22:41:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002898454 2024-09-17T22:41:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914640+0000", 2024-09-17T22:41:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4527999999999998e-05 2024-09-17T22:41:21.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914675+0000", 2024-09-17T22:41:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5315000000000003e-05 2024-09-17T22:41:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.902564+0000", 2024-09-17T22:41:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.460558978000002, 2024-09-17T22:41:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012193107, 2024-09-17T22:41:21.427 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117 2024-09-17T22:41:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902564+0000", 2024-09-17T22:41:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902564+0000", 2024-09-17T22:41:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902564+0000", 2024-09-17T22:41:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:41:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902566+0000", 2024-09-17T22:41:21.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.694e-06 2024-09-17T22:41:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902570+0000", 2024-09-17T22:41:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000003e-06 2024-09-17T22:41:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902573+0000", 2024-09-17T22:41:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7659999999999999e-06 2024-09-17T22:41:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.908269+0000", 2024-09-17T22:41:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056967500000000004 2024-09-17T22:41:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.908316+0000", 2024-09-17T22:41:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7135000000000002e-05 2024-09-17T22:41:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.908387+0000", 2024-09-17T22:41:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0625999999999998e-05 2024-09-17T22:41:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.911701+0000", 2024-09-17T22:41:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033137560000000002 2024-09-17T22:41:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914700+0000", 2024-09-17T22:41:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029988889999999998 2024-09-17T22:41:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914725+0000", 2024-09-17T22:41:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5746e-05 2024-09-17T22:41:21.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914757+0000", 2024-09-17T22:41:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1327e-05 2024-09-17T22:41:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.331914+0000", 2024-09-17T22:41:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.031208680999999, 2024-09-17T22:41:21.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00765082, 2024-09-17T22:41:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 134 2024-09-17T22:41:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.331914+0000", 2024-09-17T22:41:21.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.331914+0000", 2024-09-17T22:41:21.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.331915+0000", 2024-09-17T22:41:21.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4700000000000001e-07 2024-09-17T22:41:21.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332040+0000", 2024-09-17T22:41:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012565300000000001 2024-09-17T22:41:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332071+0000", 2024-09-17T22:41:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0437e-05 2024-09-17T22:41:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332080+0000", 2024-09-17T22:41:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1880000000000008e-06 2024-09-17T22:41:21.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.333900+0000", 2024-09-17T22:41:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001820161 2024-09-17T22:41:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.333943+0000", 2024-09-17T22:41:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2898999999999997e-05 2024-09-17T22:41:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.334019+0000", 2024-09-17T22:41:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5834999999999995e-05 2024-09-17T22:41:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.337825+0000", 2024-09-17T22:41:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038066060000000001 2024-09-17T22:41:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.339494+0000", 2024-09-17T22:41:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001668601 2024-09-17T22:41:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.339527+0000", 2024-09-17T22:41:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2712000000000001e-05 2024-09-17T22:41:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.339565+0000", 2024-09-17T22:41:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8080999999999997e-05 2024-09-17T22:41:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.332133+0000", 2024-09-17T22:41:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.030989623, 2024-09-17T22:41:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010216339, 2024-09-17T22:41:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 136 2024-09-17T22:41:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332133+0000", 2024-09-17T22:41:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332133+0000", 2024-09-17T22:41:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332134+0000", 2024-09-17T22:41:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1799999999999995e-07 2024-09-17T22:41:21.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332139+0000", 2024-09-17T22:41:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3399999999999997e-06 2024-09-17T22:41:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332156+0000", 2024-09-17T22:41:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7473e-05 2024-09-17T22:41:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332161+0000", 2024-09-17T22:41:21.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9840000000000001e-06 2024-09-17T22:41:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.334756+0000", 2024-09-17T22:41:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0025947800000000001 2024-09-17T22:41:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.334800+0000", 2024-09-17T22:41:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4375000000000001e-05 2024-09-17T22:41:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.334868+0000", 2024-09-17T22:41:21.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7705000000000003e-05 2024-09-17T22:41:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.337846+0000", 2024-09-17T22:41:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029778460000000001 2024-09-17T22:41:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342302+0000", 2024-09-17T22:41:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044555510000000003 2024-09-17T22:41:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342328+0000", 2024-09-17T22:41:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6233000000000001e-05 2024-09-17T22:41:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342349+0000", 2024-09-17T22:41:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1534000000000001e-05 2024-09-17T22:41:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.445 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.332171+0000", 2024-09-17T22:41:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.030951367, 2024-09-17T22:41:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010295187000000001, 2024-09-17T22:41:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 137 2024-09-17T22:41:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332171+0000", 2024-09-17T22:41:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332171+0000", 2024-09-17T22:41:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332172+0000", 2024-09-17T22:41:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0399999999999996e-07 2024-09-17T22:41:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332264+0000", 2024-09-17T22:41:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1873000000000006e-05 2024-09-17T22:41:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332272+0000", 2024-09-17T22:41:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3310000000000004e-06 2024-09-17T22:41:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332289+0000", 2024-09-17T22:41:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7501000000000001e-05 2024-09-17T22:41:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.335120+0000", 2024-09-17T22:41:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028308780000000002 2024-09-17T22:41:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.335164+0000", 2024-09-17T22:41:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4056e-05 2024-09-17T22:41:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.335230+0000", 2024-09-17T22:41:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6018000000000004e-05 2024-09-17T22:41:21.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.337857+0000", 2024-09-17T22:41:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026263689999999999 2024-09-17T22:41:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342395+0000", 2024-09-17T22:41:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045378730000000004 2024-09-17T22:41:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342430+0000", 2024-09-17T22:41:21.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5474e-05 2024-09-17T22:41:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342466+0000", 2024-09-17T22:41:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6310000000000003e-05 2024-09-17T22:41:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.332302+0000", 2024-09-17T22:41:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.030820159000001, 2024-09-17T22:41:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010241083, 2024-09-17T22:41:21.451 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 138 2024-09-17T22:41:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332302+0000", 2024-09-17T22:41:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332302+0000", 2024-09-17T22:41:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332303+0000", 2024-09-17T22:41:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8899999999999999e-07 2024-09-17T22:41:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332308+0000", 2024-09-17T22:41:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6530000000000002e-06 2024-09-17T22:41:21.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332316+0000", 2024-09-17T22:41:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.807e-06 2024-09-17T22:41:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332323+0000", 2024-09-17T22:41:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9149999999999996e-06 2024-09-17T22:41:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.335574+0000", 2024-09-17T22:41:21.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032509599999999998 2024-09-17T22:41:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.335618+0000", 2024-09-17T22:41:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3213000000000001e-05 2024-09-17T22:41:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.335685+0000", 2024-09-17T22:41:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7127999999999998e-05 2024-09-17T22:41:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.339686+0000", 2024-09-17T22:41:21.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040013160000000004 2024-09-17T22:41:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342498+0000", 2024-09-17T22:41:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028124309999999998 2024-09-17T22:41:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342525+0000", 2024-09-17T22:41:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6021999999999999e-05 2024-09-17T22:41:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342544+0000", 2024-09-17T22:41:21.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9049e-05 2024-09-17T22:41:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.332335+0000", 2024-09-17T22:41:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.030787804999999, 2024-09-17T22:41:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010302891, 2024-09-17T22:41:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.457 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 139 2024-09-17T22:41:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332335+0000", 2024-09-17T22:41:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332335+0000", 2024-09-17T22:41:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332335+0000", 2024-09-17T22:41:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7399999999999998e-07 2024-09-17T22:41:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332340+0000", 2024-09-17T22:41:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5360000000000003e-06 2024-09-17T22:41:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332347+0000", 2024-09-17T22:41:21.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4760000000000001e-06 2024-09-17T22:41:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332353+0000", 2024-09-17T22:41:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.203e-06 2024-09-17T22:41:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.335958+0000", 2024-09-17T22:41:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036055589999999999 2024-09-17T22:41:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.336004+0000", 2024-09-17T22:41:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6139e-05 2024-09-17T22:41:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.336085+0000", 2024-09-17T22:41:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1007000000000006e-05 2024-09-17T22:41:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.339702+0000", 2024-09-17T22:41:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036171490000000001 2024-09-17T22:41:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342578+0000", 2024-09-17T22:41:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028758999999999998 2024-09-17T22:41:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342605+0000", 2024-09-17T22:41:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6313999999999999e-05 2024-09-17T22:41:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342638+0000", 2024-09-17T22:41:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3133999999999998e-05 2024-09-17T22:41:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.462 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.332362+0000", 2024-09-17T22:41:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.030760584999999, 2024-09-17T22:41:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010346518000000001, 2024-09-17T22:41:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 140 2024-09-17T22:41:21.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.464 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332362+0000", 2024-09-17T22:41:21.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332362+0000", 2024-09-17T22:41:21.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332363+0000", 2024-09-17T22:41:21.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6800000000000001e-07 2024-09-17T22:41:21.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332425+0000", 2024-09-17T22:41:21.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2601000000000006e-05 2024-09-17T22:41:21.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332431+0000", 2024-09-17T22:41:21.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.297e-06 2024-09-17T22:41:21.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332435+0000", 2024-09-17T22:41:21.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4800000000000001e-06 2024-09-17T22:41:21.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.336306+0000", 2024-09-17T22:41:21.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038710989999999998 2024-09-17T22:41:21.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.336351+0000", 2024-09-17T22:41:21.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5411000000000003e-05 2024-09-17T22:41:21.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.336421+0000", 2024-09-17T22:41:21.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9550999999999999e-05 2024-09-17T22:41:21.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.339708+0000", 2024-09-17T22:41:21.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032866560000000002 2024-09-17T22:41:21.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342667+0000", 2024-09-17T22:41:21.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029590189999999998 2024-09-17T22:41:21.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342691+0000", 2024-09-17T22:41:21.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4556e-05 2024-09-17T22:41:21.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342709+0000", 2024-09-17T22:41:21.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.738e-05 2024-09-17T22:41:21.468 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.468 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.468 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.468 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.468 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.332444+0000", 2024-09-17T22:41:21.468 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.030678409, 2024-09-17T22:41:21.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010350679, 2024-09-17T22:41:21.468 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.469 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.469 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.469 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.469 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.469 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.469 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 141 2024-09-17T22:41:21.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.469 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332444+0000", 2024-09-17T22:41:21.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332444+0000", 2024-09-17T22:41:21.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332445+0000", 2024-09-17T22:41:21.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:41:21.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332451+0000", 2024-09-17T22:41:21.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.268e-06 2024-09-17T22:41:21.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332455+0000", 2024-09-17T22:41:21.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5809999999999998e-06 2024-09-17T22:41:21.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332458+0000", 2024-09-17T22:41:21.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8090000000000001e-06 2024-09-17T22:41:21.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.336766+0000", 2024-09-17T22:41:21.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0043076240000000003 2024-09-17T22:41:21.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.336809+0000", 2024-09-17T22:41:21.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2821000000000002e-05 2024-09-17T22:41:21.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.336879+0000", 2024-09-17T22:41:21.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9876999999999995e-05 2024-09-17T22:41:21.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.339722+0000", 2024-09-17T22:41:21.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028431300000000001 2024-09-17T22:41:21.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342736+0000", 2024-09-17T22:41:21.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003014671 2024-09-17T22:41:21.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342764+0000", 2024-09-17T22:41:21.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7545000000000001e-05 2024-09-17T22:41:21.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342795+0000", 2024-09-17T22:41:21.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0930999999999999e-05 2024-09-17T22:41:21.474 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.474 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.474 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.474 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.474 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.811897+0000", 2024-09-17T22:41:21.475 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.551225221999999, 2024-09-17T22:41:21.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0098504270000000001, 2024-09-17T22:41:21.475 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.475 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.475 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.475 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.475 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.475 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.475 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 157 2024-09-17T22:41:21.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.475 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811897+0000", 2024-09-17T22:41:21.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811897+0000", 2024-09-17T22:41:21.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811898+0000", 2024-09-17T22:41:21.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.46e-07 2024-09-17T22:41:21.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811900+0000", 2024-09-17T22:41:21.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3329999999999999e-06 2024-09-17T22:41:21.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811905+0000", 2024-09-17T22:41:21.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0359999999999997e-06 2024-09-17T22:41:21.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811908+0000", 2024-09-17T22:41:21.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9579999999999999e-06 2024-09-17T22:41:21.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.813587+0000", 2024-09-17T22:41:21.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001678761 2024-09-17T22:41:21.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.813630+0000", 2024-09-17T22:41:21.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3590000000000001e-05 2024-09-17T22:41:21.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.813697+0000", 2024-09-17T22:41:21.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6973999999999999e-05 2024-09-17T22:41:21.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.817999+0000", 2024-09-17T22:41:21.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0043015960000000004 2024-09-17T22:41:21.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.821677+0000", 2024-09-17T22:41:21.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036780129999999999 2024-09-17T22:41:21.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.821715+0000", 2024-09-17T22:41:21.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8059000000000003e-05 2024-09-17T22:41:21.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.821748+0000", 2024-09-17T22:41:21.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2761e-05 2024-09-17T22:41:21.480 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.480 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.480 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.480 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.480 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.811914+0000", 2024-09-17T22:41:21.481 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.551208139, 2024-09-17T22:41:21.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0099239910000000001, 2024-09-17T22:41:21.481 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.481 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.481 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.481 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.481 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.481 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.481 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 158 2024-09-17T22:41:21.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.481 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811914+0000", 2024-09-17T22:41:21.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811914+0000", 2024-09-17T22:41:21.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811915+0000", 2024-09-17T22:41:21.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.35e-07 2024-09-17T22:41:21.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811943+0000", 2024-09-17T22:41:21.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8696000000000001e-05 2024-09-17T22:41:21.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811951+0000", 2024-09-17T22:41:21.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4109999999999999e-06 2024-09-17T22:41:21.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811955+0000", 2024-09-17T22:41:21.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2089999999999999e-06 2024-09-17T22:41:21.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.813955+0000", 2024-09-17T22:41:21.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020004189999999998 2024-09-17T22:41:21.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.814008+0000", 2024-09-17T22:41:21.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2258e-05 2024-09-17T22:41:21.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.814082+0000", 2024-09-17T22:41:21.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4080999999999998e-05 2024-09-17T22:41:21.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.818002+0000", 2024-09-17T22:41:21.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039201690000000003 2024-09-17T22:41:21.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.821785+0000", 2024-09-17T22:41:21.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003782756 2024-09-17T22:41:21.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.821816+0000", 2024-09-17T22:41:21.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0781000000000002e-05 2024-09-17T22:41:21.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.821838+0000", 2024-09-17T22:41:21.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2976000000000002e-05 2024-09-17T22:41:21.486 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.486 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.486 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.486 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:21.486 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.811966+0000", 2024-09-17T22:41:21.486 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.551156975000001, 2024-09-17T22:41:21.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0099609099999999999, 2024-09-17T22:41:21.487 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.487 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.487 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.487 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:21.487 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:21.487 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:21.487 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 159 2024-09-17T22:41:21.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.487 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811966+0000", 2024-09-17T22:41:21.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811966+0000", 2024-09-17T22:41:21.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811966+0000", 2024-09-17T22:41:21.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8900000000000001e-07 2024-09-17T22:41:21.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811969+0000", 2024-09-17T22:41:21.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1930000000000002e-06 2024-09-17T22:41:21.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811974+0000", 2024-09-17T22:41:21.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1560000000000003e-06 2024-09-17T22:41:21.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811977+0000", 2024-09-17T22:41:21.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0230000000000001e-06 2024-09-17T22:41:21.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.814540+0000", 2024-09-17T22:41:21.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002562272 2024-09-17T22:41:21.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.814587+0000", 2024-09-17T22:41:21.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7029e-05 2024-09-17T22:41:21.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:21.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.814664+0000", 2024-09-17T22:41:21.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7198000000000002e-05 2024-09-17T22:41:21.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:21.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.818006+0000", 2024-09-17T22:41:21.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033418670000000001 2024-09-17T22:41:21.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:21.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.821870+0000", 2024-09-17T22:41:21.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038639019999999998 2024-09-17T22:41:21.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.821895+0000", 2024-09-17T22:41:21.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5449999999999999e-05 2024-09-17T22:41:21.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.821927+0000", 2024-09-17T22:41:21.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1531e-05 2024-09-17T22:41:21.491 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.492 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.492 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.492 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.492 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.492 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:21.494 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:41:21.494 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is set 2024-09-17T22:41:21.494 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T22:41:21.495 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T22:41:21.496 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:21.491+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:21.594 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:21.591+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:21.640 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:21.640 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:21.640 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:41:21.640 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:21.650 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:41:21.695 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:21.692+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:21.754 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:21.755 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:21.755 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:41:21.755 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:41:21.755 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:21.763 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:41:21.796 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:21.794+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:21.897 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:21.895+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:21.908 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:41:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:41:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:41:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4229.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-17T22:41:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:09.389090+0000", 2024-09-17T22:41:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.517728254, 2024-09-17T22:41:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028010729999999998, 2024-09-17T22:41:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.389090+0000", 2024-09-17T22:41:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.389090+0000", 2024-09-17T22:41:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.389090+0000", 2024-09-17T22:41:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4999999999999999e-07 2024-09-17T22:41:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.389100+0000", 2024-09-17T22:41:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7680000000000003e-06 2024-09-17T22:41:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.389108+0000", 2024-09-17T22:41:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6500000000000002e-06 2024-09-17T22:41:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.389117+0000", 2024-09-17T22:41:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5809999999999997e-06 2024-09-17T22:41:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.389163+0000", 2024-09-17T22:41:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5818000000000001e-05 2024-09-17T22:41:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.389185+0000", 2024-09-17T22:41:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2422999999999999e-05 2024-09-17T22:41:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.391851+0000", 2024-09-17T22:41:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026662410000000002 2024-09-17T22:41:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.391891+0000", 2024-09-17T22:41:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9341999999999997e-05 2024-09-17T22:41:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:21.913 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:21.917 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:41:21.935 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:41:21.998 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:21.996+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:22.024 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:22.024 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:22.025 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:41:22.025 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:22.033 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:41:22.083 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0: {} 2024-09-17T22:41:22.083 INFO:teuthology.orchestra.run.smithi115.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:41:22.099 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:22.097+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:22.100 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:41:22.100 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:41:22.116 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:41:22.116 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:41:22.133 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:41:22.133 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:41:22.139 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:22.139 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:22.149 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:41:22.150 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:41:22.150 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:41:22.169 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:41:22.169 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:41:22.186 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:41:22.187 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:41:22.200 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:22.197+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:22.203 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:41:22.204 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:41:22.254 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:22.254 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:22.263 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:41:22.304 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:22.302+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:22.380 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:22.380 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:22.389 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:41:22.402 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:22.399+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:22.455 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T22:41:22.468 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-17T22:41:22.493 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:22.493 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:22.502 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:41:22.509 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:22.507+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:22.604 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:22.601+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:22.605 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:22.605 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:22.613 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:41:22.705 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:22.702+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:22.716 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:22.716 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:22.724 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:41:22.805 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:22.803+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:22.827 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:22.827 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:22.836 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:41:22.906 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:22.904+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:22.939 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:22.939 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:22.948 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:41:23.008 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:23.005+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:23.050 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:23.051 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:23.059 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:41:23.109 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:23.106+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:23.164 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:23.165 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:23.170 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:23.170 INFO:teuthology.orchestra.run.smithi115.stdout:skipping zero size read 2024-09-17T22:41:23.174 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:41:23.177 INFO:teuthology.orchestra.run.smithi115.stdout:4 write 0x19ecc5f thru 0x19fba34 (0xedd6 bytes) 2024-09-17T22:41:23.216 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:23.213+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:23.312 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:23.309+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:23.319 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:23.319 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:23.328 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:41:23.392 INFO:teuthology.orchestra.run.smithi115.stdout:5 write 0x577bee3 thru 0x578138d (0x54ab bytes) 2024-09-17T22:41:23.395 INFO:teuthology.orchestra.run.smithi115.stdout:truncating to largest ever: 0xace63e8 2024-09-17T22:41:23.402 INFO:teuthology.orchestra.run.smithi115.stdout:7 trunc from 0x578138e to 0xace63e8 2024-09-17T22:41:23.408 INFO:teuthology.orchestra.run.smithi115.stdout:9 punch from 0x75ac3f7 to 0x75acaf4, (0x6fd bytes) 2024-09-17T22:41:23.408 INFO:teuthology.orchestra.run.smithi115.stdout:10 write 0x6d1b5f9 thru 0x6d2aad0 (0xf4d8 bytes) 2024-09-17T22:41:23.412 INFO:teuthology.orchestra.run.smithi115.stdout:11 punch from 0x14d696f to 0x14e337b, (0xca0c bytes) 2024-09-17T22:41:23.412 INFO:teuthology.orchestra.run.smithi115.stdout:12 read 0x6efd640 thru 0x6f0b709 (0xe0ca bytes) 2024-09-17T22:41:23.413 INFO:teuthology.orchestra.run.smithi115.stdout:13 writesame 0x8371b74 thru 0x8374ee6 data_size 0x1(0x3373 bytes) 2024-09-17T22:41:23.413 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:23.410+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:23.423 INFO:teuthology.orchestra.run.smithi115.stdout:14 write 0x6e12416 thru 0x6e182c0 (0x5eab bytes) 2024-09-17T22:41:23.434 INFO:teuthology.orchestra.run.smithi115.stdout:15 trunc from 0xace63e8 to 0x284e8b8 2024-09-17T22:41:23.445 INFO:teuthology.orchestra.run.smithi115.stdout:16 read 0x1acf5c8 thru 0x1ad80cf (0x8b08 bytes) 2024-09-17T22:41:23.448 INFO:teuthology.orchestra.run.smithi115.stdout:17 trunc from 0x284e8b8 to 0x4db869b 2024-09-17T22:41:23.453 INFO:teuthology.orchestra.run.smithi115.stdout:18 read 0x32befc1 thru 0x32c947f (0xa4bf bytes) 2024-09-17T22:41:23.457 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T22:41:23.471 INFO:teuthology.orchestra.run.smithi115.stdout:19 write 0xd04743a thru 0xd055f7c (0xeb43 bytes) 2024-09-17T22:41:23.474 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-17T22:41:23.475 INFO:teuthology.orchestra.run.smithi115.stdout:21 read 0x3d3a075 thru 0x3d47205 (0xd191 bytes) 2024-09-17T22:41:23.475 INFO:teuthology.orchestra.run.smithi115.stdout:22 read 0x7aab264 thru 0x7aabeba (0xc57 bytes) 2024-09-17T22:41:23.479 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:23.479 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:23.480 INFO:teuthology.orchestra.run.smithi115.stdout:23 writesame 0xe00a7dd thru 0xe0125fb data_size 0x1(0x7e1f bytes) 2024-09-17T22:41:23.480 INFO:teuthology.orchestra.run.smithi115.stdout:24 punch from 0x430dc4e to 0x431a96d, (0xcd1f bytes) 2024-09-17T22:41:23.481 INFO:teuthology.orchestra.run.smithi115.stdout:25 write 0x25f8be9 thru 0x25fdf98 (0x53b0 bytes) 2024-09-17T22:41:23.484 INFO:teuthology.orchestra.run.smithi115.stdout:27 read 0xb361d0b thru 0xb36eb49 (0xce3f bytes) 2024-09-17T22:41:23.484 INFO:teuthology.orchestra.run.smithi115.stdout:29 punch from 0x22ef0ba to 0x22fc4c9, (0xd40f bytes) 2024-09-17T22:41:23.489 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:41:23.490 INFO:teuthology.orchestra.run.smithi115.stdout:30 write 0xee21ac2 thru 0xee25ac5 (0x4004 bytes) 2024-09-17T22:41:23.495 INFO:teuthology.orchestra.run.smithi115.stdout:truncating to largest ever: 0xb4ddbee 2024-09-17T22:41:23.496 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:41:23.499 INFO:teuthology.orchestra.run.smithi115.stdout:32 trunc from 0xee25ac6 to 0xb4ddbee 2024-09-17T22:41:23.509 INFO:teuthology.orchestra.run.smithi115.stdout:33 write 0x6350417 thru 0x6359abd (0x96a7 bytes) 2024-09-17T22:41:23.512 INFO:teuthology.orchestra.run.smithi115.stdout:34 write 0x42475e8 thru 0x4249bc4 (0x25dd bytes) 2024-09-17T22:41:23.513 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:23.510+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:23.524 INFO:teuthology.orchestra.run.smithi115.stdout:35 trunc from 0xb4ddbee to 0x465cfa8 2024-09-17T22:41:23.547 INFO:teuthology.orchestra.run.smithi115.stdout:37 write 0xae2fc6e thru 0xae3494b (0x4cde bytes) 2024-09-17T22:41:23.557 INFO:teuthology.orchestra.run.smithi115.stdout:40 writesame 0xb5a7a50 thru 0xb5b40e6 data_size 0x1(0xc697 bytes) 2024-09-17T22:41:23.567 INFO:teuthology.orchestra.run.smithi115.stdout:41 trunc from 0xb5b40e7 to 0x2c95ea8 2024-09-17T22:41:23.582 INFO:teuthology.orchestra.run.smithi115.stdout:42 trunc from 0x2c95ea8 to 0x13f28d3 2024-09-17T22:41:23.594 INFO:teuthology.orchestra.run.smithi115.stdout:43 read 0xa3ac9e thru 0xa469ba (0xbd1d bytes) 2024-09-17T22:41:23.601 INFO:teuthology.orchestra.run.smithi115.stdout:44 write 0x2cb7fb7 thru 0x2cc0674 (0x86be bytes) 2024-09-17T22:41:23.604 INFO:teuthology.orchestra.run.smithi115.stdout:46 write 0x23ddd6d thru 0x23e8b78 (0xae0c bytes) 2024-09-17T22:41:23.608 INFO:teuthology.orchestra.run.smithi115.stdout:47 punch from 0x1a86acd to 0x1a95f2d, (0xf460 bytes) 2024-09-17T22:41:23.614 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:23.611+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:23.621 INFO:teuthology.orchestra.run.smithi115.stdout:48 write 0x905f5e3 thru 0x906d8d6 (0xe2f4 bytes) 2024-09-17T22:41:23.631 INFO:teuthology.orchestra.run.smithi115.stdout:51 trunc from 0x906d8d7 to 0x3f51e06 2024-09-17T22:41:23.639 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:23.640 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:23.649 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:41:23.659 INFO:teuthology.orchestra.run.smithi115.stdout:52 write 0xe2f2216 thru 0xe2fe54b (0xc336 bytes) 2024-09-17T22:41:23.662 INFO:teuthology.orchestra.run.smithi115.stdout:53 write 0x325e3d8 thru 0x3268dc2 (0xa9eb bytes) 2024-09-17T22:41:23.672 INFO:teuthology.orchestra.run.smithi115.stdout:54 write 0xec3b697 thru 0xec3eb42 (0x34ac bytes) 2024-09-17T22:41:23.675 INFO:teuthology.orchestra.run.smithi115.stdout:55 writesame 0xe129012 thru 0xe12f02a data_size 0x1(0x6019 bytes) 2024-09-17T22:41:23.688 INFO:teuthology.orchestra.run.smithi115.stdout:56 writesame 0x451788 thru 0x4566af data_size 0x44(0x4f28 bytes) 2024-09-17T22:41:23.705 INFO:teuthology.orchestra.run.smithi115.stdout:57 trunc from 0xec3eb43 to 0x3e9ec65 2024-09-17T22:41:23.715 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:23.712+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:23.718 INFO:teuthology.orchestra.run.smithi115.stdout:58 read 0x36a1b thru 0x40153 (0x9739 bytes) 2024-09-17T22:41:23.718 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:23.718 INFO:teuthology.orchestra.run.smithi115.stdout:60 read 0x471063 thru 0x47f764 (0xe702 bytes) 2024-09-17T22:41:23.718 INFO:teuthology.orchestra.run.smithi115.stdout:61 trunc from 0x3e9ec65 to 0x3b710bc 2024-09-17T22:41:23.723 INFO:teuthology.orchestra.run.smithi115.stdout:truncating to largest ever: 0xe3edfbb 2024-09-17T22:41:23.735 INFO:teuthology.orchestra.run.smithi115.stdout:62 trunc from 0x3b710bc to 0xe3edfbb 2024-09-17T22:41:23.741 INFO:teuthology.orchestra.run.smithi115.stdout:63 write 0x78e52fd thru 0x78ec0ab (0x6daf bytes) 2024-09-17T22:41:23.751 INFO:teuthology.orchestra.run.smithi115.stdout:64 write 0xeca596d thru 0xecaee76 (0x950a bytes) 2024-09-17T22:41:23.755 INFO:teuthology.orchestra.run.smithi115.stdout:65 punch from 0x45bc525 to 0x45c058d, (0x4068 bytes) 2024-09-17T22:41:23.755 INFO:teuthology.orchestra.run.smithi115.stdout:66 read 0x4067410 thru 0x4070881 (0x9472 bytes) 2024-09-17T22:41:23.755 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:23.755 INFO:teuthology.orchestra.run.smithi115.stdout:68 punch from 0x189effa to 0x18a6c17, (0x7c1d bytes) 2024-09-17T22:41:23.755 INFO:teuthology.orchestra.run.smithi115.stdout:69 write 0xa07557d thru 0xa07a6ab (0x512f bytes) 2024-09-17T22:41:23.759 INFO:teuthology.orchestra.run.smithi115.stdout:71 write 0xe8984a5 thru 0xe89ddc9 (0x5925 bytes) 2024-09-17T22:41:23.763 INFO:teuthology.orchestra.run.smithi115.stdout:72 read 0x1f0f98c thru 0x1f102f1 (0x966 bytes) 2024-09-17T22:41:23.763 INFO:teuthology.orchestra.run.smithi115.stdout:75 write 0x41d8f49 thru 0x41e8ea6 (0xff5e bytes) 2024-09-17T22:41:23.769 INFO:teuthology.orchestra.run.smithi115.stdout:76 read 0x44f669f thru 0x44fdcad (0x760f bytes) 2024-09-17T22:41:23.769 INFO:teuthology.orchestra.run.smithi115.stdout:79 writesame 0x3045944 thru 0x3054783 data_size 0x20(0xee40 bytes) 2024-09-17T22:41:23.771 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:23.771 INFO:teuthology.orchestra.run.smithi115.stdout:82 punch from 0x62c4385 to 0x62ce2a5, (0x9f20 bytes) 2024-09-17T22:41:23.771 INFO:teuthology.orchestra.run.smithi115.stdout:83 read 0xdfdbeac thru 0xdfe528c (0x93e1 bytes) 2024-09-17T22:41:23.771 INFO:teuthology.orchestra.run.smithi115.stdout:84 write 0xbd3dd67 thru 0xbd4a46d (0xc707 bytes) 2024-09-17T22:41:23.775 INFO:teuthology.orchestra.run.smithi115.stdout:86 read 0x8498446 thru 0x84a1973 (0x952e bytes) 2024-09-17T22:41:23.775 INFO:teuthology.orchestra.run.smithi115.stdout:88 read 0x3f45c30 thru 0x3f46127 (0x4f8 bytes) 2024-09-17T22:41:23.775 INFO:teuthology.orchestra.run.smithi115.stdout:89 write 0xe843418 thru 0xe852a19 (0xf602 bytes) 2024-09-17T22:41:23.777 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:23.791 INFO:teuthology.orchestra.run.smithi115.stdout:91 trunc from 0xecaee77 to 0x2d389bd 2024-09-17T22:41:23.797 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:23.797 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:23.807 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:41:23.826 INFO:teuthology.orchestra.run.smithi115.stdout:93 write 0xd0aac4b thru 0xd0b7c64 (0xd01a bytes) 2024-09-17T22:41:23.827 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:23.824+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:23.833 INFO:teuthology.orchestra.run.smithi115.stdout:94 write 0x28c4d12 thru 0x28ce13a (0x9429 bytes) 2024-09-17T22:41:23.836 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:23.837 INFO:teuthology.orchestra.run.smithi115.stdout:96 punch from 0x67371f3 to 0x6746c3c, (0xfa49 bytes) 2024-09-17T22:41:23.837 INFO:teuthology.orchestra.run.smithi115.stdout:97 read 0x492f85e thru 0x49376d3 (0x7e76 bytes) 2024-09-17T22:41:23.837 INFO:teuthology.orchestra.run.smithi115.stdout:98 read 0xf8451a thru 0xf8b47d (0x6f64 bytes) 2024-09-17T22:41:23.837 INFO:teuthology.orchestra.run.smithi115.stdout:99 write 0xb99f9ae thru 0xb9aac24 (0xb277 bytes) 2024-09-17T22:41:23.839 INFO:teuthology.orchestra.run.smithi115.stdout:100 read 0x495ad thru 0x4beef (0x2943 bytes) 2024-09-17T22:41:23.839 INFO:teuthology.orchestra.run.smithi115.stdout:101 write 0x1bba333 thru 0x1bc4681 (0xa34f bytes) 2024-09-17T22:41:23.842 INFO:teuthology.orchestra.run.smithi115.stdout:103 write 0xaf677e6 thru 0xaf715f3 (0x9e0e bytes) 2024-09-17T22:41:23.845 INFO:teuthology.orchestra.run.smithi115.stdout:104 writesame 0x691f5f6 thru 0x692d5ca data_size 0x1(0xdfd5 bytes) 2024-09-17T22:41:23.871 INFO:teuthology.orchestra.run.smithi115.stdout:105 writesame 0x91b81f3 thru 0x91b831c data_size 0x1(0x12a bytes) 2024-09-17T22:41:23.889 INFO:teuthology.orchestra.run.smithi115.stdout:106 trunc from 0xd0b7c65 to 0x12969cf 2024-09-17T22:41:23.902 INFO:teuthology.orchestra.run.smithi115.stdout:107 read 0xaaa557 thru 0xab8c7f (0xe729 bytes) 2024-09-17T22:41:23.902 INFO:teuthology.orchestra.run.smithi115.stdout:108 read 0x26dc6c thru 0x278416 (0xa7ab bytes) 2024-09-17T22:41:23.909 INFO:teuthology.orchestra.run.smithi115.stdout:109 write 0x310c880 thru 0x310e48f (0x1c10 bytes) 2024-09-17T22:41:23.912 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:23.917 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:23.914+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:23.917 INFO:teuthology.orchestra.run.smithi115.stdout:112 writesame 0x3aaaba4 thru 0x3ab3387 data_size 0x4e(0x87e4 bytes) 2024-09-17T22:41:23.921 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:23.922 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:23.931 INFO:teuthology.orchestra.run.smithi115.stdout:113 write 0xa7fb19c thru 0xa80914b (0xdfb0 bytes) 2024-09-17T22:41:23.933 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:41:23.942 INFO:teuthology.orchestra.run.smithi115.stdout:116 trunc from 0xa80914c to 0x4ce5f99 2024-09-17T22:41:23.951 INFO:teuthology.orchestra.run.smithi115.stdout:117 read 0x3e71a8d thru 0x3e7c053 (0xa5c7 bytes) 2024-09-17T22:41:23.959 INFO:teuthology.orchestra.run.smithi115.stdout:118 writesame 0x699bcb3 thru 0x69ab976 data_size 0x4(0xfcc4 bytes) 2024-09-17T22:41:23.959 INFO:teuthology.orchestra.run.smithi115.stdout:119 writesame 0x3f5adca thru 0x3f61b0e data_size 0x1(0x6d45 bytes) 2024-09-17T22:41:23.976 INFO:teuthology.orchestra.run.smithi115.stdout:120 read 0x12a0278 thru 0x12a1bbb (0x1944 bytes) 2024-09-17T22:41:23.976 INFO:teuthology.orchestra.run.smithi115.stdout:121 read 0x90cf8 thru 0x9c9e1 (0xbcea bytes) 2024-09-17T22:41:23.977 INFO:teuthology.orchestra.run.smithi115.stdout:124 writesame 0x42ae90d thru 0x42b0597 data_size 0x1(0x1c8b bytes) 2024-09-17T22:41:23.982 INFO:teuthology.orchestra.run.smithi115.stdout:126 read 0x55333d1 thru 0x5535afb (0x272b bytes) 2024-09-17T22:41:23.982 INFO:teuthology.orchestra.run.smithi115.stdout:127 read 0xd54e7b thru 0xd64d61 (0xfee7 bytes) 2024-09-17T22:41:23.982 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:23.982 INFO:teuthology.orchestra.run.smithi115.stdout:130 write 0x52bc06d thru 0x52bd956 (0x18ea bytes) 2024-09-17T22:41:23.986 INFO:teuthology.orchestra.run.smithi115.stdout:131 read 0x326c4b0 thru 0x32733c2 (0x6f13 bytes) 2024-09-17T22:41:23.997 INFO:teuthology.orchestra.run.smithi115.stdout:132 writesame 0xa6ebb1f thru 0xa6ef928 data_size 0x3(0x3e0a bytes) 2024-09-17T22:41:23.997 INFO:teuthology.orchestra.run.smithi115.stdout:133 write 0x1143d97 thru 0x11445b9 (0x823 bytes) 2024-09-17T22:41:24.000 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:24.000 INFO:teuthology.orchestra.run.smithi115.stdout:135 write 0x4e293a0 thru 0x4e31c8b (0x88ec bytes) 2024-09-17T22:41:24.003 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:24.003 INFO:teuthology.orchestra.run.smithi115.stdout:138 writesame 0x5bf8590 thru 0x5bfa00b data_size 0xf(0x1a7c bytes) 2024-09-17T22:41:24.006 INFO:teuthology.orchestra.run.smithi115.stdout:140 punch from 0x2c30f7a to 0x2c3643f, (0x54c5 bytes) 2024-09-17T22:41:24.006 INFO:teuthology.orchestra.run.smithi115.stdout:141 read 0x925d001 thru 0x9268a2f (0xba2f bytes) 2024-09-17T22:41:24.006 INFO:teuthology.orchestra.run.smithi115.stdout:142 read 0x8575541 thru 0x857fba3 (0xa663 bytes) 2024-09-17T22:41:24.007 INFO:teuthology.orchestra.run.smithi115.stdout:143 read 0x83bf86c thru 0x83c3559 (0x3cee bytes) 2024-09-17T22:41:24.014 INFO:teuthology.orchestra.run.smithi115.stdout:145 write 0xd0671a2 thru 0xd071bb4 (0xaa13 bytes) 2024-09-17T22:41:24.017 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:24.015+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:24.018 INFO:teuthology.orchestra.run.smithi115.stdout:146 read 0x4a42d54 thru 0x4a45502 (0x27af bytes) 2024-09-17T22:41:24.018 INFO:teuthology.orchestra.run.smithi115.stdout:147 clone 1 order 25 su 131072 sc 8 2024-09-17T22:41:24.051 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:24.052 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:24.061 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:41:24.144 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:24.142+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:24.204 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:24.204 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:24.213 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:41:24.215 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:41:24.219 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:24.217+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:24.320 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:24.317+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:24.359 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:24.359 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:24.368 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:41:24.375 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0: {} 2024-09-17T22:41:24.375 INFO:teuthology.orchestra.run.smithi115.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:41:24.393 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:41:24.393 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:41:24.409 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:41:24.410 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:41:24.421 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:24.419+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:24.425 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:41:24.425 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:41:24.442 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:41:24.442 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:41:24.459 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:41:24.459 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:41:24.466 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is unset 2024-09-17T22:41:24.467 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.6 to 1 (10000) 2024-09-17T22:41:24.476 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:24.477 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:24.477 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:41:24.477 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:41:24.477 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:24.481 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:41:24.481 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:41:24.488 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-17T22:41:24.490 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:41:24.500 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:41:24.501 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:41:24.553 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:24.551+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:24.615 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:24.615 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:41:24.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:41:24.615 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:41:24.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.616 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.616 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:41:24.616 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 21.251483743000001, 2024-09-17T22:41:24.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.249158453, 2024-09-17T22:41:24.616 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.616 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.616 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.616 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-17T22:41:24.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.617 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:41:24.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:41:24.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358680+0000", 2024-09-17T22:41:24.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.592e-06 2024-09-17T22:41:24.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358688+0000", 2024-09-17T22:41:24.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1610000000000003e-06 2024-09-17T22:41:24.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358697+0000", 2024-09-17T22:41:24.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2580000000000001e-06 2024-09-17T22:41:24.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358706+0000", 2024-09-17T22:41:24.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1640000000000007e-06 2024-09-17T22:41:24.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358766+0000", 2024-09-17T22:41:24.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9893999999999999e-05 2024-09-17T22:41:24.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:41:24.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358778+0000", 2024-09-17T22:41:24.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1148999999999999e-05 2024-09-17T22:41:24.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606347+0000", 2024-09-17T22:41:24.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.247569971 2024-09-17T22:41:24.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606533+0000", 2024-09-17T22:41:24.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018599400000000001 2024-09-17T22:41:24.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606752+0000", 2024-09-17T22:41:24.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000218373 2024-09-17T22:41:24.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607455+0000", 2024-09-17T22:41:24.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070311799999999997 2024-09-17T22:41:24.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607735+0000", 2024-09-17T22:41:24.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027972300000000001 2024-09-17T22:41:24.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607793+0000", 2024-09-17T22:41:24.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8529999999999997e-05 2024-09-17T22:41:24.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607837+0000", 2024-09-17T22:41:24.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3526000000000003e-05 2024-09-17T22:41:24.622 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.622 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.622 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.622 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.622 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.638594+0000", 2024-09-17T22:41:24.622 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.971568202, 2024-09-17T22:41:24.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011505949, 2024-09-17T22:41:24.623 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.623 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.623 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.623 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.623 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.623 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.623 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 36 2024-09-17T22:41:24.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.623 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638594+0000", 2024-09-17T22:41:24.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638594+0000", 2024-09-17T22:41:24.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638594+0000", 2024-09-17T22:41:24.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3299999999999998e-07 2024-09-17T22:41:24.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638597+0000", 2024-09-17T22:41:24.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4099999999999998e-06 2024-09-17T22:41:24.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638601+0000", 2024-09-17T22:41:24.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3710000000000002e-06 2024-09-17T22:41:24.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638607+0000", 2024-09-17T22:41:24.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5060000000000001e-06 2024-09-17T22:41:24.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.641700+0000", 2024-09-17T22:41:24.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030928309999999999 2024-09-17T22:41:24.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.641778+0000", 2024-09-17T22:41:24.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7735e-05 2024-09-17T22:41:24.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.641910+0000", 2024-09-17T22:41:24.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013224100000000001 2024-09-17T22:41:24.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.644227+0000", 2024-09-17T22:41:24.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002316987 2024-09-17T22:41:24.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.650014+0000", 2024-09-17T22:41:24.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0057866289999999997 2024-09-17T22:41:24.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.650063+0000", 2024-09-17T22:41:24.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8980999999999997e-05 2024-09-17T22:41:24.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.650100+0000", 2024-09-17T22:41:24.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6925000000000003e-05 2024-09-17T22:41:24.628 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.628 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.628 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.628 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.629 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.638612+0000", 2024-09-17T22:41:24.629 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.971549724999999, 2024-09-17T22:41:24.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011569195000000001, 2024-09-17T22:41:24.629 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.629 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.629 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.629 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.629 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.629 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.629 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 37 2024-09-17T22:41:24.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.629 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638612+0000", 2024-09-17T22:41:24.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638612+0000", 2024-09-17T22:41:24.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638613+0000", 2024-09-17T22:41:24.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5899999999999998e-07 2024-09-17T22:41:24.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638615+0000", 2024-09-17T22:41:24.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2560000000000001e-06 2024-09-17T22:41:24.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638619+0000", 2024-09-17T22:41:24.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9509999999999999e-06 2024-09-17T22:41:24.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.638622+0000", 2024-09-17T22:41:24.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8200000000000001e-06 2024-09-17T22:41:24.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.642217+0000", 2024-09-17T22:41:24.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035953180000000001 2024-09-17T22:41:24.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.642272+0000", 2024-09-17T22:41:24.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5096999999999998e-05 2024-09-17T22:41:24.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.642361+0000", 2024-09-17T22:41:24.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8518999999999997e-05 2024-09-17T22:41:24.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.644493+0000", 2024-09-17T22:41:24.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021328190000000002 2024-09-17T22:41:24.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.650108+0000", 2024-09-17T22:41:24.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056144259999999996 2024-09-17T22:41:24.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.650149+0000", 2024-09-17T22:41:24.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1511999999999998e-05 2024-09-17T22:41:24.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.650182+0000", 2024-09-17T22:41:24.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2218000000000002e-05 2024-09-17T22:41:24.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.635 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.635 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.635 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.635 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.691247+0000", 2024-09-17T22:41:24.635 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.918915415000001, 2024-09-17T22:41:24.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0075837589999999998, 2024-09-17T22:41:24.635 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.635 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.635 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.635 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 70 2024-09-17T22:41:24.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.636 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.691247+0000", 2024-09-17T22:41:24.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.691247+0000", 2024-09-17T22:41:24.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.691247+0000", 2024-09-17T22:41:24.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5699999999999998e-07 2024-09-17T22:41:24.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.691250+0000", 2024-09-17T22:41:24.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.525e-06 2024-09-17T22:41:24.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.691253+0000", 2024-09-17T22:41:24.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9060000000000004e-06 2024-09-17T22:41:24.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.691256+0000", 2024-09-17T22:41:24.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8289999999999999e-06 2024-09-17T22:41:24.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.692763+0000", 2024-09-17T22:41:24.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015062599999999999 2024-09-17T22:41:24.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.692810+0000", 2024-09-17T22:41:24.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7194000000000002e-05 2024-09-17T22:41:24.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.692887+0000", 2024-09-17T22:41:24.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7310000000000004e-05 2024-09-17T22:41:24.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.696280+0000", 2024-09-17T22:41:24.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033933420000000001 2024-09-17T22:41:24.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.698742+0000", 2024-09-17T22:41:24.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024613650000000001 2024-09-17T22:41:24.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.698790+0000", 2024-09-17T22:41:24.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8531e-05 2024-09-17T22:41:24.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.698830+0000", 2024-09-17T22:41:24.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0139999999999999e-05 2024-09-17T22:41:24.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.640 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.640 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.640 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.902335+0000", 2024-09-17T22:41:24.641 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.707827046999999, 2024-09-17T22:41:24.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011889003, 2024-09-17T22:41:24.641 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.641 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.641 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.641 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.641 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.641 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.641 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 112 2024-09-17T22:41:24.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.641 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902335+0000", 2024-09-17T22:41:24.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902335+0000", 2024-09-17T22:41:24.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902335+0000", 2024-09-17T22:41:24.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0699999999999999e-07 2024-09-17T22:41:24.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902413+0000", 2024-09-17T22:41:24.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8279e-05 2024-09-17T22:41:24.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902419+0000", 2024-09-17T22:41:24.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.677e-06 2024-09-17T22:41:24.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902422+0000", 2024-09-17T22:41:24.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2760000000000001e-06 2024-09-17T22:41:24.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.905774+0000", 2024-09-17T22:41:24.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033517809999999999 2024-09-17T22:41:24.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.905816+0000", 2024-09-17T22:41:24.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2265999999999998e-05 2024-09-17T22:41:24.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.905891+0000", 2024-09-17T22:41:24.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.47e-05 2024-09-17T22:41:24.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.911676+0000", 2024-09-17T22:41:24.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0057843529999999999 2024-09-17T22:41:24.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914066+0000", 2024-09-17T22:41:24.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023904439999999998 2024-09-17T22:41:24.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914191+0000", 2024-09-17T22:41:24.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012519000000000001 2024-09-17T22:41:24.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914224+0000", 2024-09-17T22:41:24.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2830000000000002e-05 2024-09-17T22:41:24.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.646 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.647 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.647 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.647 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.902440+0000", 2024-09-17T22:41:24.647 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.707722033, 2024-09-17T22:41:24.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011895823999999999, 2024-09-17T22:41:24.647 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.647 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.647 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.647 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.647 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.647 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.647 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113 2024-09-17T22:41:24.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902440+0000", 2024-09-17T22:41:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902440+0000", 2024-09-17T22:41:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902440+0000", 2024-09-17T22:41:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-09-17T22:41:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902443+0000", 2024-09-17T22:41:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1229999999999998e-06 2024-09-17T22:41:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902453+0000", 2024-09-17T22:41:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0380999999999999e-05 2024-09-17T22:41:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902456+0000", 2024-09-17T22:41:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0479999999999999e-06 2024-09-17T22:41:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.906256+0000", 2024-09-17T22:41:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037996760000000001 2024-09-17T22:41:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.906328+0000", 2024-09-17T22:41:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2637000000000003e-05 2024-09-17T22:41:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.906448+0000", 2024-09-17T22:41:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000119826 2024-09-17T22:41:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.911685+0000", 2024-09-17T22:41:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052368359999999999 2024-09-17T22:41:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914268+0000", 2024-09-17T22:41:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0025828700000000001 2024-09-17T22:41:24.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914299+0000", 2024-09-17T22:41:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0880999999999998e-05 2024-09-17T22:41:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914336+0000", 2024-09-17T22:41:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7156999999999997e-05 2024-09-17T22:41:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.902461+0000", 2024-09-17T22:41:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.707701149000002, 2024-09-17T22:41:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011999490999999999, 2024-09-17T22:41:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.653 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 114 2024-09-17T22:41:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902461+0000", 2024-09-17T22:41:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902461+0000", 2024-09-17T22:41:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902461+0000", 2024-09-17T22:41:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4999999999999999e-07 2024-09-17T22:41:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902463+0000", 2024-09-17T22:41:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.858e-06 2024-09-17T22:41:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902468+0000", 2024-09-17T22:41:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.101e-06 2024-09-17T22:41:24.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902471+0000", 2024-09-17T22:41:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7410000000000001e-06 2024-09-17T22:41:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.906768+0000", 2024-09-17T22:41:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042968069999999997 2024-09-17T22:41:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.906843+0000", 2024-09-17T22:41:24.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5053000000000001e-05 2024-09-17T22:41:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.906961+0000", 2024-09-17T22:41:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000118513 2024-09-17T22:41:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.911689+0000", 2024-09-17T22:41:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004727373 2024-09-17T22:41:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914379+0000", 2024-09-17T22:41:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026900579999999999 2024-09-17T22:41:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914424+0000", 2024-09-17T22:41:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5404000000000001e-05 2024-09-17T22:41:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914460+0000", 2024-09-17T22:41:24.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6332999999999999e-05 2024-09-17T22:41:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.902488+0000", 2024-09-17T22:41:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.707673655000001, 2024-09-17T22:41:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012064375, 2024-09-17T22:41:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 115 2024-09-17T22:41:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902488+0000", 2024-09-17T22:41:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902488+0000", 2024-09-17T22:41:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902489+0000", 2024-09-17T22:41:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-09-17T22:41:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902537+0000", 2024-09-17T22:41:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8665999999999998e-05 2024-09-17T22:41:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902543+0000", 2024-09-17T22:41:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3040000000000004e-06 2024-09-17T22:41:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902546+0000", 2024-09-17T22:41:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1580000000000001e-06 2024-09-17T22:41:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.907325+0000", 2024-09-17T22:41:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047793779999999999 2024-09-17T22:41:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.907372+0000", 2024-09-17T22:41:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7017000000000001e-05 2024-09-17T22:41:24.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.907445+0000", 2024-09-17T22:41:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2909000000000003e-05 2024-09-17T22:41:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.911693+0000", 2024-09-17T22:41:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042477360000000002 2024-09-17T22:41:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914495+0000", 2024-09-17T22:41:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028017630000000001 2024-09-17T22:41:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914522+0000", 2024-09-17T22:41:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7231999999999999e-05 2024-09-17T22:41:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914553+0000", 2024-09-17T22:41:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0849000000000002e-05 2024-09-17T22:41:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.902552+0000", 2024-09-17T22:41:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.707610428999999, 2024-09-17T22:41:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012123254999999999, 2024-09-17T22:41:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116 2024-09-17T22:41:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902552+0000", 2024-09-17T22:41:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902552+0000", 2024-09-17T22:41:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902552+0000", 2024-09-17T22:41:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4600000000000001e-07 2024-09-17T22:41:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902554+0000", 2024-09-17T22:41:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9230000000000001e-06 2024-09-17T22:41:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902558+0000", 2024-09-17T22:41:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7309999999999999e-06 2024-09-17T22:41:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902560+0000", 2024-09-17T22:41:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.757e-06 2024-09-17T22:41:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.907809+0000", 2024-09-17T22:41:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052484610000000003 2024-09-17T22:41:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.907877+0000", 2024-09-17T22:41:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7947000000000005e-05 2024-09-17T22:41:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.907995+0000", 2024-09-17T22:41:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011783 2024-09-17T22:41:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.911697+0000", 2024-09-17T22:41:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037020629999999998 2024-09-17T22:41:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914595+0000", 2024-09-17T22:41:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002898454 2024-09-17T22:41:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914640+0000", 2024-09-17T22:41:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4527999999999998e-05 2024-09-17T22:41:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914675+0000", 2024-09-17T22:41:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5315000000000003e-05 2024-09-17T22:41:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.902564+0000", 2024-09-17T22:41:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.707598392000001, 2024-09-17T22:41:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012193107, 2024-09-17T22:41:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.670 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117 2024-09-17T22:41:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902564+0000", 2024-09-17T22:41:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902564+0000", 2024-09-17T22:41:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902564+0000", 2024-09-17T22:41:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:41:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902566+0000", 2024-09-17T22:41:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.694e-06 2024-09-17T22:41:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902570+0000", 2024-09-17T22:41:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000003e-06 2024-09-17T22:41:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.902573+0000", 2024-09-17T22:41:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7659999999999999e-06 2024-09-17T22:41:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.908269+0000", 2024-09-17T22:41:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056967500000000004 2024-09-17T22:41:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.908316+0000", 2024-09-17T22:41:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7135000000000002e-05 2024-09-17T22:41:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.908387+0000", 2024-09-17T22:41:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0625999999999998e-05 2024-09-17T22:41:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.911701+0000", 2024-09-17T22:41:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033137560000000002 2024-09-17T22:41:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914700+0000", 2024-09-17T22:41:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029988889999999998 2024-09-17T22:41:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914725+0000", 2024-09-17T22:41:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5746e-05 2024-09-17T22:41:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.914757+0000", 2024-09-17T22:41:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1327e-05 2024-09-17T22:41:24.675 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.331914+0000", 2024-09-17T22:41:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.278248094999999, 2024-09-17T22:41:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00765082, 2024-09-17T22:41:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.676 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 134 2024-09-17T22:41:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.331914+0000", 2024-09-17T22:41:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.331914+0000", 2024-09-17T22:41:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.331915+0000", 2024-09-17T22:41:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4700000000000001e-07 2024-09-17T22:41:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332040+0000", 2024-09-17T22:41:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012565300000000001 2024-09-17T22:41:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332071+0000", 2024-09-17T22:41:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0437e-05 2024-09-17T22:41:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332080+0000", 2024-09-17T22:41:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1880000000000008e-06 2024-09-17T22:41:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.333900+0000", 2024-09-17T22:41:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001820161 2024-09-17T22:41:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.333943+0000", 2024-09-17T22:41:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2898999999999997e-05 2024-09-17T22:41:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.334019+0000", 2024-09-17T22:41:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5834999999999995e-05 2024-09-17T22:41:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.337825+0000", 2024-09-17T22:41:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038066060000000001 2024-09-17T22:41:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.339494+0000", 2024-09-17T22:41:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001668601 2024-09-17T22:41:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.339527+0000", 2024-09-17T22:41:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2712000000000001e-05 2024-09-17T22:41:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.339565+0000", 2024-09-17T22:41:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8080999999999997e-05 2024-09-17T22:41:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.681 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.332133+0000", 2024-09-17T22:41:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.278029037, 2024-09-17T22:41:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010216339, 2024-09-17T22:41:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.684 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 136 2024-09-17T22:41:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332133+0000", 2024-09-17T22:41:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332133+0000", 2024-09-17T22:41:24.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332134+0000", 2024-09-17T22:41:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1799999999999995e-07 2024-09-17T22:41:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332139+0000", 2024-09-17T22:41:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3399999999999997e-06 2024-09-17T22:41:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332156+0000", 2024-09-17T22:41:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7473e-05 2024-09-17T22:41:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332161+0000", 2024-09-17T22:41:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9840000000000001e-06 2024-09-17T22:41:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.334756+0000", 2024-09-17T22:41:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0025947800000000001 2024-09-17T22:41:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.334800+0000", 2024-09-17T22:41:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4375000000000001e-05 2024-09-17T22:41:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.334868+0000", 2024-09-17T22:41:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7705000000000003e-05 2024-09-17T22:41:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.337846+0000", 2024-09-17T22:41:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029778460000000001 2024-09-17T22:41:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342302+0000", 2024-09-17T22:41:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044555510000000003 2024-09-17T22:41:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342328+0000", 2024-09-17T22:41:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6233000000000001e-05 2024-09-17T22:41:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342349+0000", 2024-09-17T22:41:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1534000000000001e-05 2024-09-17T22:41:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.332171+0000", 2024-09-17T22:41:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.277990781, 2024-09-17T22:41:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010295187000000001, 2024-09-17T22:41:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 137 2024-09-17T22:41:24.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332171+0000", 2024-09-17T22:41:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332171+0000", 2024-09-17T22:41:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332172+0000", 2024-09-17T22:41:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0399999999999996e-07 2024-09-17T22:41:24.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332264+0000", 2024-09-17T22:41:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1873000000000006e-05 2024-09-17T22:41:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332272+0000", 2024-09-17T22:41:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3310000000000004e-06 2024-09-17T22:41:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332289+0000", 2024-09-17T22:41:24.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7501000000000001e-05 2024-09-17T22:41:24.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.335120+0000", 2024-09-17T22:41:24.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028308780000000002 2024-09-17T22:41:24.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.335164+0000", 2024-09-17T22:41:24.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4056e-05 2024-09-17T22:41:24.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.335230+0000", 2024-09-17T22:41:24.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6018000000000004e-05 2024-09-17T22:41:24.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.337857+0000", 2024-09-17T22:41:24.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026263689999999999 2024-09-17T22:41:24.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342395+0000", 2024-09-17T22:41:24.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045378730000000004 2024-09-17T22:41:24.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342430+0000", 2024-09-17T22:41:24.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5474e-05 2024-09-17T22:41:24.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342466+0000", 2024-09-17T22:41:24.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6310000000000003e-05 2024-09-17T22:41:24.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.695 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.695 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.695 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.332302+0000", 2024-09-17T22:41:24.695 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.277859573000001, 2024-09-17T22:41:24.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010241083, 2024-09-17T22:41:24.696 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.696 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.696 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.696 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 138 2024-09-17T22:41:24.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.696 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332302+0000", 2024-09-17T22:41:24.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332302+0000", 2024-09-17T22:41:24.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332303+0000", 2024-09-17T22:41:24.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8899999999999999e-07 2024-09-17T22:41:24.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332308+0000", 2024-09-17T22:41:24.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6530000000000002e-06 2024-09-17T22:41:24.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332316+0000", 2024-09-17T22:41:24.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.807e-06 2024-09-17T22:41:24.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332323+0000", 2024-09-17T22:41:24.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9149999999999996e-06 2024-09-17T22:41:24.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.335574+0000", 2024-09-17T22:41:24.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032509599999999998 2024-09-17T22:41:24.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.335618+0000", 2024-09-17T22:41:24.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3213000000000001e-05 2024-09-17T22:41:24.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.335685+0000", 2024-09-17T22:41:24.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7127999999999998e-05 2024-09-17T22:41:24.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.339686+0000", 2024-09-17T22:41:24.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040013160000000004 2024-09-17T22:41:24.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342498+0000", 2024-09-17T22:41:24.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028124309999999998 2024-09-17T22:41:24.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342525+0000", 2024-09-17T22:41:24.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6021999999999999e-05 2024-09-17T22:41:24.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342544+0000", 2024-09-17T22:41:24.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9049e-05 2024-09-17T22:41:24.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.701 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.701 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.701 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.332335+0000", 2024-09-17T22:41:24.701 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.277827218999999, 2024-09-17T22:41:24.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010302891, 2024-09-17T22:41:24.701 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.702 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.702 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.702 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 139 2024-09-17T22:41:24.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.702 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332335+0000", 2024-09-17T22:41:24.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332335+0000", 2024-09-17T22:41:24.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332335+0000", 2024-09-17T22:41:24.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7399999999999998e-07 2024-09-17T22:41:24.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332340+0000", 2024-09-17T22:41:24.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5360000000000003e-06 2024-09-17T22:41:24.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332347+0000", 2024-09-17T22:41:24.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4760000000000001e-06 2024-09-17T22:41:24.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332353+0000", 2024-09-17T22:41:24.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.203e-06 2024-09-17T22:41:24.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.335958+0000", 2024-09-17T22:41:24.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036055589999999999 2024-09-17T22:41:24.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.336004+0000", 2024-09-17T22:41:24.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6139e-05 2024-09-17T22:41:24.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.336085+0000", 2024-09-17T22:41:24.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1007000000000006e-05 2024-09-17T22:41:24.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.339702+0000", 2024-09-17T22:41:24.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036171490000000001 2024-09-17T22:41:24.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342578+0000", 2024-09-17T22:41:24.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028758999999999998 2024-09-17T22:41:24.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342605+0000", 2024-09-17T22:41:24.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6313999999999999e-05 2024-09-17T22:41:24.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342638+0000", 2024-09-17T22:41:24.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3133999999999998e-05 2024-09-17T22:41:24.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.707 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.707 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.707 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.332362+0000", 2024-09-17T22:41:24.707 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.277799998999999, 2024-09-17T22:41:24.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010346518000000001, 2024-09-17T22:41:24.707 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.708 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.708 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.708 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 140 2024-09-17T22:41:24.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.708 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332362+0000", 2024-09-17T22:41:24.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332362+0000", 2024-09-17T22:41:24.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332363+0000", 2024-09-17T22:41:24.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6800000000000001e-07 2024-09-17T22:41:24.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332425+0000", 2024-09-17T22:41:24.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2601000000000006e-05 2024-09-17T22:41:24.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332431+0000", 2024-09-17T22:41:24.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.297e-06 2024-09-17T22:41:24.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332435+0000", 2024-09-17T22:41:24.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4800000000000001e-06 2024-09-17T22:41:24.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.336306+0000", 2024-09-17T22:41:24.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038710989999999998 2024-09-17T22:41:24.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.336351+0000", 2024-09-17T22:41:24.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5411000000000003e-05 2024-09-17T22:41:24.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.336421+0000", 2024-09-17T22:41:24.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9550999999999999e-05 2024-09-17T22:41:24.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.339708+0000", 2024-09-17T22:41:24.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032866560000000002 2024-09-17T22:41:24.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342667+0000", 2024-09-17T22:41:24.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029590189999999998 2024-09-17T22:41:24.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342691+0000", 2024-09-17T22:41:24.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4556e-05 2024-09-17T22:41:24.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342709+0000", 2024-09-17T22:41:24.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.738e-05 2024-09-17T22:41:24.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.713 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.713 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.713 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.332444+0000", 2024-09-17T22:41:24.713 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.277717823, 2024-09-17T22:41:24.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010350679, 2024-09-17T22:41:24.713 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.713 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.713 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.714 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.714 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.714 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 141 2024-09-17T22:41:24.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.714 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332444+0000", 2024-09-17T22:41:24.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332444+0000", 2024-09-17T22:41:24.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332445+0000", 2024-09-17T22:41:24.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:41:24.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332451+0000", 2024-09-17T22:41:24.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.268e-06 2024-09-17T22:41:24.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332455+0000", 2024-09-17T22:41:24.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5809999999999998e-06 2024-09-17T22:41:24.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.332458+0000", 2024-09-17T22:41:24.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8090000000000001e-06 2024-09-17T22:41:24.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.336766+0000", 2024-09-17T22:41:24.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0043076240000000003 2024-09-17T22:41:24.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.336809+0000", 2024-09-17T22:41:24.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2821000000000002e-05 2024-09-17T22:41:24.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.336879+0000", 2024-09-17T22:41:24.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9876999999999995e-05 2024-09-17T22:41:24.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.339722+0000", 2024-09-17T22:41:24.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028431300000000001 2024-09-17T22:41:24.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342736+0000", 2024-09-17T22:41:24.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003014671 2024-09-17T22:41:24.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342764+0000", 2024-09-17T22:41:24.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7545000000000001e-05 2024-09-17T22:41:24.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.342795+0000", 2024-09-17T22:41:24.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0930999999999999e-05 2024-09-17T22:41:24.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.719 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.719 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.719 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.811897+0000", 2024-09-17T22:41:24.719 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.798264635999999, 2024-09-17T22:41:24.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0098504270000000001, 2024-09-17T22:41:24.720 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.720 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.720 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.720 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 157 2024-09-17T22:41:24.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.720 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811897+0000", 2024-09-17T22:41:24.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811897+0000", 2024-09-17T22:41:24.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811898+0000", 2024-09-17T22:41:24.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.46e-07 2024-09-17T22:41:24.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811900+0000", 2024-09-17T22:41:24.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3329999999999999e-06 2024-09-17T22:41:24.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811905+0000", 2024-09-17T22:41:24.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0359999999999997e-06 2024-09-17T22:41:24.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811908+0000", 2024-09-17T22:41:24.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9579999999999999e-06 2024-09-17T22:41:24.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.813587+0000", 2024-09-17T22:41:24.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001678761 2024-09-17T22:41:24.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.813630+0000", 2024-09-17T22:41:24.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3590000000000001e-05 2024-09-17T22:41:24.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.813697+0000", 2024-09-17T22:41:24.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6973999999999999e-05 2024-09-17T22:41:24.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.817999+0000", 2024-09-17T22:41:24.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0043015960000000004 2024-09-17T22:41:24.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.821677+0000", 2024-09-17T22:41:24.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036780129999999999 2024-09-17T22:41:24.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.821715+0000", 2024-09-17T22:41:24.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8059000000000003e-05 2024-09-17T22:41:24.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.821748+0000", 2024-09-17T22:41:24.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2761e-05 2024-09-17T22:41:24.725 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.725 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.725 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.725 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.725 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.811914+0000", 2024-09-17T22:41:24.725 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.798247552999999, 2024-09-17T22:41:24.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0099239910000000001, 2024-09-17T22:41:24.725 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.725 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.725 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.726 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 158 2024-09-17T22:41:24.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.726 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811914+0000", 2024-09-17T22:41:24.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811914+0000", 2024-09-17T22:41:24.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811915+0000", 2024-09-17T22:41:24.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.35e-07 2024-09-17T22:41:24.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811943+0000", 2024-09-17T22:41:24.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8696000000000001e-05 2024-09-17T22:41:24.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811951+0000", 2024-09-17T22:41:24.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4109999999999999e-06 2024-09-17T22:41:24.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811955+0000", 2024-09-17T22:41:24.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2089999999999999e-06 2024-09-17T22:41:24.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.813955+0000", 2024-09-17T22:41:24.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020004189999999998 2024-09-17T22:41:24.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.814008+0000", 2024-09-17T22:41:24.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2258e-05 2024-09-17T22:41:24.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.814082+0000", 2024-09-17T22:41:24.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4080999999999998e-05 2024-09-17T22:41:24.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.818002+0000", 2024-09-17T22:41:24.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039201690000000003 2024-09-17T22:41:24.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.821785+0000", 2024-09-17T22:41:24.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003782756 2024-09-17T22:41:24.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.821816+0000", 2024-09-17T22:41:24.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0781000000000002e-05 2024-09-17T22:41:24.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.821838+0000", 2024-09-17T22:41:24.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2976000000000002e-05 2024-09-17T22:41:24.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.731 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.731 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:24.731 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:04.811966+0000", 2024-09-17T22:41:24.731 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.798196389000001, 2024-09-17T22:41:24.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0099609099999999999, 2024-09-17T22:41:24.731 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:24.731 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:24.731 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:24.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:24.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:24.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:24.732 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 159 2024-09-17T22:41:24.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.732 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:24.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:24.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811966+0000", 2024-09-17T22:41:24.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:24.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811966+0000", 2024-09-17T22:41:24.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:24.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:24.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811966+0000", 2024-09-17T22:41:24.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8900000000000001e-07 2024-09-17T22:41:24.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:24.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811969+0000", 2024-09-17T22:41:24.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1930000000000002e-06 2024-09-17T22:41:24.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:24.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811974+0000", 2024-09-17T22:41:24.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1560000000000003e-06 2024-09-17T22:41:24.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:24.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.811977+0000", 2024-09-17T22:41:24.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0230000000000001e-06 2024-09-17T22:41:24.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:24.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.814540+0000", 2024-09-17T22:41:24.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002562272 2024-09-17T22:41:24.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:24.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.814587+0000", 2024-09-17T22:41:24.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7029e-05 2024-09-17T22:41:24.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:24.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.814664+0000", 2024-09-17T22:41:24.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7198000000000002e-05 2024-09-17T22:41:24.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:24.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.818006+0000", 2024-09-17T22:41:24.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033418670000000001 2024-09-17T22:41:24.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:24.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.821870+0000", 2024-09-17T22:41:24.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038639019999999998 2024-09-17T22:41:24.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:24.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.821895+0000", 2024-09-17T22:41:24.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5449999999999999e-05 2024-09-17T22:41:24.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:24.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:24.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:24.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:04.821927+0000", 2024-09-17T22:41:24.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1531e-05 2024-09-17T22:41:24.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.736 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:24.736 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:24.736 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:24.738 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:41:24.739 INFO:teuthology.orchestra.run.smithi115.stdout:148 writesame 0xb38c528 thru 0xb39b32d data_size 0x1(0xee06 bytes) 2024-09-17T22:41:24.739 INFO:teuthology.orchestra.run.smithi115.stdout:149 writesame 0x2452a1b thru 0x245438b data_size 0xd(0x1971 bytes) 2024-09-17T22:41:24.739 INFO:teuthology.orchestra.run.smithi115.stdout:151 write 0x6198434 thru 0x619f813 (0x73e0 bytes) 2024-09-17T22:41:24.739 INFO:teuthology.orchestra.run.smithi115.stdout:154 write 0xd516b04 thru 0xd524e8b (0xe388 bytes) 2024-09-17T22:41:24.739 INFO:teuthology.orchestra.run.smithi115.stdout:155 punch from 0x7918912 to 0x7920fc2, (0x86b0 bytes) 2024-09-17T22:41:24.739 INFO:teuthology.orchestra.run.smithi115.stdout:158 punch from 0x36e695e to 0x36e9a38, (0x30da bytes) 2024-09-17T22:41:24.739 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:24.739 INFO:teuthology.orchestra.run.smithi115.stdout:163 trunc from 0xd524e8c to 0x3708906 2024-09-17T22:41:24.740 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:24.735+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:24.744 INFO:teuthology.orchestra.run.smithi115.stdout:164 writesame 0x1ae541f thru 0x1ae8be0 data_size 0x4e(0x37c2 bytes) 2024-09-17T22:41:24.760 INFO:teuthology.orchestra.run.smithi115.stdout:165 writesame 0xb97baf8 thru 0xb986765 data_size 0x15(0xac6e bytes) 2024-09-17T22:41:24.764 INFO:teuthology.orchestra.run.smithi115.stdout:166 read 0x632b243 thru 0x632b870 (0x62e bytes) 2024-09-17T22:41:24.765 INFO:teuthology.orchestra.run.smithi115.stdout:167 punch from 0xab6d88a to 0xab75549, (0x7cbf bytes) 2024-09-17T22:41:24.765 INFO:teuthology.orchestra.run.smithi115.stdout:168 writesame 0x71f9bcc thru 0x720263a data_size 0x3(0x8a6f bytes) 2024-09-17T22:41:24.771 INFO:teuthology.orchestra.run.smithi115.stdout:170 read 0x2a025ae thru 0x2a1014a (0xdb9d bytes) 2024-09-17T22:41:24.772 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:24.773 INFO:teuthology.orchestra.run.smithi115.stdout:174 trunc from 0xb986766 to 0xcab9843 2024-09-17T22:41:24.779 INFO:teuthology.orchestra.run.smithi115.stdout:175 writesame 0x9104194 thru 0x9109fd9 data_size 0xb(0x5e46 bytes) 2024-09-17T22:41:24.781 INFO:teuthology.orchestra.run.smithi115.stdout:176 read 0x63a01e2 thru 0x63a455d (0x437c bytes) 2024-09-17T22:41:24.782 INFO:teuthology.orchestra.run.smithi115.stdout:177 write 0x8d10587 thru 0x8d1d8b0 (0xd32a bytes) 2024-09-17T22:41:24.783 INFO:teuthology.orchestra.run.smithi115.stdout:178 read 0x36324d thru 0x36eda8 (0xbb5c bytes) 2024-09-17T22:41:24.784 INFO:teuthology.orchestra.run.smithi115.stdout:179 writesame 0x8ae45cf thru 0x8af291d data_size 0x15(0xe34f bytes) 2024-09-17T22:41:24.786 INFO:teuthology.orchestra.run.smithi115.stdout:180 writesame 0x48e122c thru 0x48e168c data_size 0x1(0x461 bytes) 2024-09-17T22:41:24.789 INFO:teuthology.orchestra.run.smithi115.stdout:182 punch from 0x9eb31fe to 0x9ebb378, (0x817a bytes) 2024-09-17T22:41:24.789 INFO:teuthology.orchestra.run.smithi115.stdout:183 writesame 0x765a926 thru 0x766a0eb data_size 0x5(0xf7c6 bytes) 2024-09-17T22:41:24.796 INFO:teuthology.orchestra.run.smithi115.stdout:184 writesame 0xabcd160 thru 0xabd4974 data_size 0x1(0x7815 bytes) 2024-09-17T22:41:24.810 INFO:teuthology.orchestra.run.smithi115.stdout:185 write 0x100bd65 thru 0x100e9c7 (0x2c63 bytes) 2024-09-17T22:41:24.812 INFO:teuthology.orchestra.run.smithi115.stdout:186 writesame 0x971eb99 thru 0x9726655 data_size 0x1(0x7abd bytes) 2024-09-17T22:41:24.838 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:24.836+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:24.839 INFO:teuthology.orchestra.run.smithi115.stdout:187 trunc from 0xcab9843 to 0x11fdd57 2024-09-17T22:41:24.859 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:24.859 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:24.859 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:41:24.859 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:24.860 INFO:teuthology.orchestra.run.smithi115.stdout:188 trunc from 0x11fdd57 to 0xbf8a20c 2024-09-17T22:41:24.865 INFO:teuthology.orchestra.run.smithi115.stdout:189 write 0x7c79bd9 thru 0x7c82410 (0x8838 bytes) 2024-09-17T22:41:24.870 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:41:24.873 INFO:teuthology.orchestra.run.smithi115.stdout:190 write 0xdad94cc thru 0xdae75dd (0xe112 bytes) 2024-09-17T22:41:24.879 INFO:teuthology.orchestra.run.smithi115.stdout:193 trunc from 0xdae75de to 0xa52cae8 2024-09-17T22:41:24.890 INFO:teuthology.orchestra.run.smithi115.stdout:194 trunc from 0xa52cae8 to 0x6671559 2024-09-17T22:41:24.898 INFO:teuthology.orchestra.run.smithi115.stdout:195 write 0x82e5a4 thru 0x83dcb0 (0xf70d bytes) 2024-09-17T22:41:24.900 INFO:teuthology.orchestra.run.smithi115.stdout:196 read 0x46ffe91 thru 0x4707bf6 (0x7d66 bytes) 2024-09-17T22:41:24.901 INFO:teuthology.orchestra.run.smithi115.stdout:197 punch from 0x221f481 to 0x221f492, (0x11 bytes) 2024-09-17T22:41:24.901 INFO:teuthology.orchestra.run.smithi115.stdout:198 writesame 0x1dfe391 thru 0x1e0afb3 data_size 0x1(0xcc23 bytes) 2024-09-17T22:41:24.921 INFO:teuthology.orchestra.run.smithi115.stdout:200 read 0x53fb9be thru 0x5405f60 (0xa5a3 bytes) 2024-09-17T22:41:24.922 INFO:teuthology.orchestra.run.smithi115.stdout:201 punch from 0x6646488 to 0x66505e7, (0xa15f bytes) 2024-09-17T22:41:24.922 INFO:teuthology.orchestra.run.smithi115.stdout:202 punch from 0x11fd994 to 0x12079bc, (0xa028 bytes) 2024-09-17T22:41:24.922 INFO:teuthology.orchestra.run.smithi115.stdout:203 read 0x2a0761b thru 0x2a08b04 (0x14ea bytes) 2024-09-17T22:41:24.931 INFO:teuthology.orchestra.run.smithi115.stdout:204 trunc from 0x6671559 to 0xd7d2fad 2024-09-17T22:41:24.936 INFO:teuthology.orchestra.run.smithi115.stdout:205 write 0x19f8517 thru 0x1a0286e (0xa358 bytes) 2024-09-17T22:41:24.938 INFO:teuthology.orchestra.run.smithi115.stdout:206 clone 2 order 18 su 2048 sc 9 2024-09-17T22:41:24.939 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:24.936+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:24.983 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:24.983 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:24.983 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:41:24.983 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:41:24.983 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:24.991 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:41:25.039 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:25.037+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:25.136 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:25.136 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:41:25.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:41:25.136 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:41:25.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.136 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4229.0:1 2.3 e19/18 2:c4c92e5a:::rbd_trash:head v 19'1, mlcod=0'0)", 2024-09-17T22:41:25.136 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:09.389090+0000", 2024-09-17T22:41:25.136 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.743478047, 2024-09-17T22:41:25.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028010729999999998, 2024-09-17T22:41:25.137 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.137 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.137 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.137 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.389090+0000", 2024-09-17T22:41:25.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.389090+0000", 2024-09-17T22:41:25.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.389090+0000", 2024-09-17T22:41:25.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4999999999999999e-07 2024-09-17T22:41:25.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.389100+0000", 2024-09-17T22:41:25.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7680000000000003e-06 2024-09-17T22:41:25.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.389108+0000", 2024-09-17T22:41:25.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6500000000000002e-06 2024-09-17T22:41:25.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.389117+0000", 2024-09-17T22:41:25.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5809999999999997e-06 2024-09-17T22:41:25.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.389163+0000", 2024-09-17T22:41:25.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5818000000000001e-05 2024-09-17T22:41:25.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.389185+0000", 2024-09-17T22:41:25.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2422999999999999e-05 2024-09-17T22:41:25.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.391851+0000", 2024-09-17T22:41:25.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026662410000000002 2024-09-17T22:41:25.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.391891+0000", 2024-09-17T22:41:25.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9341999999999997e-05 2024-09-17T22:41:25.141 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.141 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.141 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.141 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:672 3.5 3:a0207ec9:::rbd_data.115f1ca54530.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.141 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.691959+0000", 2024-09-17T22:41:25.141 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.440609166, 2024-09-17T22:41:25.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019649300000000001, 2024-09-17T22:41:25.141 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.141 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.141 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.142 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 672 2024-09-17T22:41:25.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.142 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.691959+0000", 2024-09-17T22:41:25.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.691959+0000", 2024-09-17T22:41:25.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.691960+0000", 2024-09-17T22:41:25.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8419999999999999e-06 2024-09-17T22:41:25.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.691995+0000", 2024-09-17T22:41:25.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4075999999999997e-05 2024-09-17T22:41:25.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.692010+0000", 2024-09-17T22:41:25.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5179e-05 2024-09-17T22:41:25.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.692017+0000", 2024-09-17T22:41:25.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.131e-06 2024-09-17T22:41:25.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.692080+0000", 2024-09-17T22:41:25.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3336999999999993e-05 2024-09-17T22:41:25.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.692193+0000", 2024-09-17T22:41:25.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000113232 2024-09-17T22:41:25.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:25.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.692414+0000", 2024-09-17T22:41:25.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000220938 2024-09-17T22:41:25.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.693664+0000", 2024-09-17T22:41:25.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012497090000000001 2024-09-17T22:41:25.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.693846+0000", 2024-09-17T22:41:25.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000181781 2024-09-17T22:41:25.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.693895+0000", 2024-09-17T22:41:25.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8931000000000003e-05 2024-09-17T22:41:25.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.693924+0000", 2024-09-17T22:41:25.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8773999999999999e-05 2024-09-17T22:41:25.147 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.147 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.147 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.147 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:755 3.5 3:a0207ec9:::rbd_data.115f1ca54530.0000000000000002:head [stat,truncate 7208960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.147 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.725190+0000", 2024-09-17T22:41:25.147 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.40737823499999998, 2024-09-17T22:41:25.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040959000000000004, 2024-09-17T22:41:25.147 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.148 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.148 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.148 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 755 2024-09-17T22:41:25.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.148 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.725190+0000", 2024-09-17T22:41:25.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.725190+0000", 2024-09-17T22:41:25.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.725192+0000", 2024-09-17T22:41:25.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6249999999999999e-06 2024-09-17T22:41:25.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.725207+0000", 2024-09-17T22:41:25.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5243e-05 2024-09-17T22:41:25.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.725234+0000", 2024-09-17T22:41:25.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6441e-05 2024-09-17T22:41:25.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.725247+0000", 2024-09-17T22:41:25.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3419e-05 2024-09-17T22:41:25.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.725326+0000", 2024-09-17T22:41:25.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9036999999999996e-05 2024-09-17T22:41:25.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.725428+0000", 2024-09-17T22:41:25.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101525 2024-09-17T22:41:25.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:25.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.725598+0000", 2024-09-17T22:41:25.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017041199999999999 2024-09-17T22:41:25.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.726807+0000", 2024-09-17T22:41:25.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001209014 2024-09-17T22:41:25.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729188+0000", 2024-09-17T22:41:25.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023804450000000001 2024-09-17T22:41:25.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729242+0000", 2024-09-17T22:41:25.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4119999999999997e-05 2024-09-17T22:41:25.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729285+0000", 2024-09-17T22:41:25.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3618999999999997e-05 2024-09-17T22:41:25.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.153 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.153 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:903 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.153 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.742658+0000", 2024-09-17T22:41:25.153 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.38990971099999999, 2024-09-17T22:41:25.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012903929999999999, 2024-09-17T22:41:25.153 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.153 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.153 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.154 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 903 2024-09-17T22:41:25.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.154 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.742658+0000", 2024-09-17T22:41:25.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.742658+0000", 2024-09-17T22:41:25.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.742659+0000", 2024-09-17T22:41:25.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5300000000000002e-07 2024-09-17T22:41:25.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.742664+0000", 2024-09-17T22:41:25.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6090000000000003e-06 2024-09-17T22:41:25.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.742674+0000", 2024-09-17T22:41:25.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.006e-05 2024-09-17T22:41:25.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.742679+0000", 2024-09-17T22:41:25.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5219999999999998e-06 2024-09-17T22:41:25.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.742730+0000", 2024-09-17T22:41:25.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0525000000000002e-05 2024-09-17T22:41:25.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.742808+0000", 2024-09-17T22:41:25.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8434000000000002e-05 2024-09-17T22:41:25.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:25.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.742975+0000", 2024-09-17T22:41:25.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000167174 2024-09-17T22:41:25.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.743779+0000", 2024-09-17T22:41:25.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00080340900000000005 2024-09-17T22:41:25.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.743842+0000", 2024-09-17T22:41:25.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3718000000000002e-05 2024-09-17T22:41:25.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.743899+0000", 2024-09-17T22:41:25.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6826000000000001e-05 2024-09-17T22:41:25.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.743948+0000", 2024-09-17T22:41:25.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9162999999999997e-05 2024-09-17T22:41:25.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.159 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.159 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:917 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:head [stat,write 14916556~25652] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.159 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.768636+0000", 2024-09-17T22:41:25.159 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.36393223600000002, 2024-09-17T22:41:25.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013179089999999999, 2024-09-17T22:41:25.159 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.159 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.159 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.159 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.160 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.160 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.160 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 917 2024-09-17T22:41:25.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.160 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.768636+0000", 2024-09-17T22:41:25.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.768636+0000", 2024-09-17T22:41:25.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.768637+0000", 2024-09-17T22:41:25.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9489999999999999e-06 2024-09-17T22:41:25.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.768655+0000", 2024-09-17T22:41:25.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7442999999999999e-05 2024-09-17T22:41:25.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.768672+0000", 2024-09-17T22:41:25.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7116999999999999e-05 2024-09-17T22:41:25.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.768681+0000", 2024-09-17T22:41:25.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7290000000000006e-06 2024-09-17T22:41:25.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.768737+0000", 2024-09-17T22:41:25.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6530999999999998e-05 2024-09-17T22:41:25.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.768805+0000", 2024-09-17T22:41:25.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7355999999999997e-05 2024-09-17T22:41:25.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:25.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.768905+0000", 2024-09-17T22:41:25.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100454 2024-09-17T22:41:25.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.769662+0000", 2024-09-17T22:41:25.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00075717999999999998 2024-09-17T22:41:25.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.769885+0000", 2024-09-17T22:41:25.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022233999999999999 2024-09-17T22:41:25.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.769920+0000", 2024-09-17T22:41:25.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5751999999999999e-05 2024-09-17T22:41:25.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.769953+0000", 2024-09-17T22:41:25.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3058000000000001e-05 2024-09-17T22:41:25.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.165 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.165 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:932 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.165 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.782946+0000", 2024-09-17T22:41:25.165 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.34962183000000002, 2024-09-17T22:41:25.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021243920000000001, 2024-09-17T22:41:25.165 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.165 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.165 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.165 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.166 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 932 2024-09-17T22:41:25.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.166 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.782946+0000", 2024-09-17T22:41:25.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.782946+0000", 2024-09-17T22:41:25.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.782947+0000", 2024-09-17T22:41:25.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4249999999999999e-06 2024-09-17T22:41:25.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.782953+0000", 2024-09-17T22:41:25.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7039999999999999e-06 2024-09-17T22:41:25.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.782965+0000", 2024-09-17T22:41:25.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2007999999999999e-05 2024-09-17T22:41:25.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.782971+0000", 2024-09-17T22:41:25.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6400000000000002e-06 2024-09-17T22:41:25.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.783023+0000", 2024-09-17T22:41:25.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2796000000000001e-05 2024-09-17T22:41:25.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.783123+0000", 2024-09-17T22:41:25.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9591000000000002e-05 2024-09-17T22:41:25.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:25.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.783463+0000", 2024-09-17T22:41:25.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000339947 2024-09-17T22:41:25.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.784686+0000", 2024-09-17T22:41:25.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012231399999999999 2024-09-17T22:41:25.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.784966+0000", 2024-09-17T22:41:25.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027975 2024-09-17T22:41:25.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.785020+0000", 2024-09-17T22:41:25.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4500000000000003e-05 2024-09-17T22:41:25.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.785070+0000", 2024-09-17T22:41:25.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9891000000000001e-05 2024-09-17T22:41:25.171 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.171 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.171 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.171 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:933 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.171 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.785205+0000", 2024-09-17T22:41:25.171 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.34736284099999998, 2024-09-17T22:41:25.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023739220000000001, 2024-09-17T22:41:25.171 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.171 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.172 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.172 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 933 2024-09-17T22:41:25.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.172 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.785205+0000", 2024-09-17T22:41:25.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.785205+0000", 2024-09-17T22:41:25.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.785206+0000", 2024-09-17T22:41:25.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8700000000000004e-07 2024-09-17T22:41:25.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.785211+0000", 2024-09-17T22:41:25.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.412e-06 2024-09-17T22:41:25.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.785220+0000", 2024-09-17T22:41:25.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7029999999999995e-06 2024-09-17T22:41:25.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.785225+0000", 2024-09-17T22:41:25.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6380000000000003e-06 2024-09-17T22:41:25.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.785267+0000", 2024-09-17T22:41:25.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2661999999999999e-05 2024-09-17T22:41:25.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.785345+0000", 2024-09-17T22:41:25.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7696999999999998e-05 2024-09-17T22:41:25.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:25.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.785543+0000", 2024-09-17T22:41:25.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000198302 2024-09-17T22:41:25.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.786290+0000", 2024-09-17T22:41:25.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00074700799999999996 2024-09-17T22:41:25.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.787489+0000", 2024-09-17T22:41:25.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00119918 2024-09-17T22:41:25.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.787539+0000", 2024-09-17T22:41:25.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9237000000000003e-05 2024-09-17T22:41:25.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.787579+0000", 2024-09-17T22:41:25.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0195999999999999e-05 2024-09-17T22:41:25.177 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.177 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.177 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.177 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:942 3.5 3:a0207ec9:::rbd_data.115f1ca54530.0000000000000002:head [stat,truncate 2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.177 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.838768+0000", 2024-09-17T22:41:25.177 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.29379932600000003, 2024-09-17T22:41:25.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020802289999999998, 2024-09-17T22:41:25.177 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.177 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.177 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.177 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.178 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.178 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.178 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 942 2024-09-17T22:41:25.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.178 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838768+0000", 2024-09-17T22:41:25.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838768+0000", 2024-09-17T22:41:25.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838771+0000", 2024-09-17T22:41:25.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6419999999999999e-06 2024-09-17T22:41:25.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838784+0000", 2024-09-17T22:41:25.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2452999999999999e-05 2024-09-17T22:41:25.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838807+0000", 2024-09-17T22:41:25.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3264999999999998e-05 2024-09-17T22:41:25.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838818+0000", 2024-09-17T22:41:25.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0889e-05 2024-09-17T22:41:25.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838898+0000", 2024-09-17T22:41:25.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0241999999999995e-05 2024-09-17T22:41:25.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838984+0000", 2024-09-17T22:41:25.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5888000000000002e-05 2024-09-17T22:41:25.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:25.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839155+0000", 2024-09-17T22:41:25.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000171037 2024-09-17T22:41:25.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.840341+0000", 2024-09-17T22:41:25.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001186391 2024-09-17T22:41:25.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.840735+0000", 2024-09-17T22:41:25.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039403300000000001 2024-09-17T22:41:25.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.840803+0000", 2024-09-17T22:41:25.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7719999999999995e-05 2024-09-17T22:41:25.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.840849+0000", 2024-09-17T22:41:25.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5668999999999999e-05 2024-09-17T22:41:25.183 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.183 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.183 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.183 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:947 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:head [stat,truncate 2350423] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.183 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.838863+0000", 2024-09-17T22:41:25.183 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.293704417, 2024-09-17T22:41:25.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00232475, 2024-09-17T22:41:25.183 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.183 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.183 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.183 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.184 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 947 2024-09-17T22:41:25.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.184 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838863+0000", 2024-09-17T22:41:25.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838863+0000", 2024-09-17T22:41:25.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838864+0000", 2024-09-17T22:41:25.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7100000000000002e-07 2024-09-17T22:41:25.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838867+0000", 2024-09-17T22:41:25.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0920000000000002e-06 2024-09-17T22:41:25.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838874+0000", 2024-09-17T22:41:25.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3730000000000004e-06 2024-09-17T22:41:25.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838880+0000", 2024-09-17T22:41:25.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6570000000000002e-06 2024-09-17T22:41:25.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839475+0000", 2024-09-17T22:41:25.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00059497499999999997 2024-09-17T22:41:25.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839547+0000", 2024-09-17T22:41:25.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1694999999999997e-05 2024-09-17T22:41:25.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:25.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839678+0000", 2024-09-17T22:41:25.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013096399999999999 2024-09-17T22:41:25.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.840889+0000", 2024-09-17T22:41:25.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012115310000000001 2024-09-17T22:41:25.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.841121+0000", 2024-09-17T22:41:25.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023183700000000001 2024-09-17T22:41:25.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.841151+0000", 2024-09-17T22:41:25.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0125e-05 2024-09-17T22:41:25.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.841188+0000", 2024-09-17T22:41:25.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.693e-05 2024-09-17T22:41:25.189 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.189 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.189 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.189 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4447.0:940 3.1e e29/23 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:head v 29'8, mlcod=29'6)", 2024-09-17T22:41:25.189 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.839326+0000", 2024-09-17T22:41:25.189 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.29324205199999998, 2024-09-17T22:41:25.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013470279999999999, 2024-09-17T22:41:25.189 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.189 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.189 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.190 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839326+0000", 2024-09-17T22:41:25.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839326+0000", 2024-09-17T22:41:25.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839326+0000", 2024-09-17T22:41:25.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8599999999999997e-07 2024-09-17T22:41:25.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839333+0000", 2024-09-17T22:41:25.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0809999999999999e-06 2024-09-17T22:41:25.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839343+0000", 2024-09-17T22:41:25.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0455000000000001e-05 2024-09-17T22:41:25.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839350+0000", 2024-09-17T22:41:25.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5180000000000001e-06 2024-09-17T22:41:25.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839483+0000", 2024-09-17T22:41:25.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000133258 2024-09-17T22:41:25.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839502+0000", 2024-09-17T22:41:25.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8980000000000001e-05 2024-09-17T22:41:25.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.840635+0000", 2024-09-17T22:41:25.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001132758 2024-09-17T22:41:25.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.840673+0000", 2024-09-17T22:41:25.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8192000000000003e-05 2024-09-17T22:41:25.193 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.193 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.193 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.193 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "pg_update_log_missing(3.1 epoch 29/23 rep_tid 19 entries 29'2 (0'0) error 3:87dd89a3:::rbd_data.115f1ca54530.0000000000000005:head by client.4447.0:945 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-17T22:41:25.194 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.839388+0000", 2024-09-17T22:41:25.194 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.29318012700000001, 2024-09-17T22:41:25.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001283639, 2024-09-17T22:41:25.194 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.194 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.194 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-17T22:41:25.194 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839388+0000", 2024-09-17T22:41:25.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839388+0000", 2024-09-17T22:41:25.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839388+0000", 2024-09-17T22:41:25.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7300000000000001e-07 2024-09-17T22:41:25.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839396+0000", 2024-09-17T22:41:25.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.858e-06 2024-09-17T22:41:25.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839409+0000", 2024-09-17T22:41:25.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3198e-05 2024-09-17T22:41:25.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839420+0000", 2024-09-17T22:41:25.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0597e-05 2024-09-17T22:41:25.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839487+0000", 2024-09-17T22:41:25.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7535999999999998e-05 2024-09-17T22:41:25.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.840671+0000", 2024-09-17T22:41:25.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011839769999999999 2024-09-17T22:41:25.197 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.197 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.197 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.197 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:978 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:head [stat,write 28573696~30174] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.197 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.872316+0000", 2024-09-17T22:41:25.197 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.26025174699999998, 2024-09-17T22:41:25.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015477679999999999, 2024-09-17T22:41:25.197 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.197 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.198 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.198 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.198 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.198 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.198 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 978 2024-09-17T22:41:25.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.198 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.872316+0000", 2024-09-17T22:41:25.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.872316+0000", 2024-09-17T22:41:25.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.872318+0000", 2024-09-17T22:41:25.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1859999999999999e-06 2024-09-17T22:41:25.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.872342+0000", 2024-09-17T22:41:25.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3317999999999999e-05 2024-09-17T22:41:25.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.872361+0000", 2024-09-17T22:41:25.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9239e-05 2024-09-17T22:41:25.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.872370+0000", 2024-09-17T22:41:25.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4129999999999995e-06 2024-09-17T22:41:25.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.872440+0000", 2024-09-17T22:41:25.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0220000000000002e-05 2024-09-17T22:41:25.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.872536+0000", 2024-09-17T22:41:25.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5697000000000002e-05 2024-09-17T22:41:25.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:25.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.872698+0000", 2024-09-17T22:41:25.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000162208 2024-09-17T22:41:25.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.873405+0000", 2024-09-17T22:41:25.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070630900000000002 2024-09-17T22:41:25.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.873804+0000", 2024-09-17T22:41:25.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039966300000000003 2024-09-17T22:41:25.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.873836+0000", 2024-09-17T22:41:25.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1999000000000003e-05 2024-09-17T22:41:25.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.873864+0000", 2024-09-17T22:41:25.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7515999999999999e-05 2024-09-17T22:41:25.202 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.202 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.203 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.203 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:981 3.5 3:a0207ec9:::rbd_data.115f1ca54530.0000000000000002:head [stat,zero 21626880~7077888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.203 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.878155+0000", 2024-09-17T22:41:25.203 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.25441238999999999, 2024-09-17T22:41:25.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015160309999999999, 2024-09-17T22:41:25.203 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.203 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.203 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:25.204 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.204 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.204 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.204 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 981 2024-09-17T22:41:25.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.204 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878155+0000", 2024-09-17T22:41:25.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878155+0000", 2024-09-17T22:41:25.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878158+0000", 2024-09-17T22:41:25.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.717e-06 2024-09-17T22:41:25.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878175+0000", 2024-09-17T22:41:25.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6830999999999999e-05 2024-09-17T22:41:25.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878198+0000", 2024-09-17T22:41:25.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3391e-05 2024-09-17T22:41:25.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878210+0000", 2024-09-17T22:41:25.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2152e-05 2024-09-17T22:41:25.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878291+0000", 2024-09-17T22:41:25.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0666000000000003e-05 2024-09-17T22:41:25.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878408+0000", 2024-09-17T22:41:25.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116848 2024-09-17T22:41:25.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.879671+0000", 2024-09-17T22:41:25.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001263426 2024-09-17T22:41:25.207 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.207 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.207 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.208 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:986 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:head [stat,truncate 21626880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.208 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.878292+0000", 2024-09-17T22:41:25.208 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.254276222, 2024-09-17T22:41:25.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015877700000000001, 2024-09-17T22:41:25.208 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.208 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.208 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.208 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 986 2024-09-17T22:41:25.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.208 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878292+0000", 2024-09-17T22:41:25.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878292+0000", 2024-09-17T22:41:25.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878292+0000", 2024-09-17T22:41:25.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7100000000000002e-07 2024-09-17T22:41:25.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878296+0000", 2024-09-17T22:41:25.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0160000000000002e-06 2024-09-17T22:41:25.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878304+0000", 2024-09-17T22:41:25.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8280000000000003e-06 2024-09-17T22:41:25.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878326+0000", 2024-09-17T22:41:25.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2376000000000001e-05 2024-09-17T22:41:25.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878724+0000", 2024-09-17T22:41:25.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039772800000000002 2024-09-17T22:41:25.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878796+0000", 2024-09-17T22:41:25.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1914000000000003e-05 2024-09-17T22:41:25.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:25.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878933+0000", 2024-09-17T22:41:25.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013689400000000001 2024-09-17T22:41:25.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.879689+0000", 2024-09-17T22:41:25.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00075639000000000001 2024-09-17T22:41:25.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.879805+0000", 2024-09-17T22:41:25.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115956 2024-09-17T22:41:25.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.879851+0000", 2024-09-17T22:41:25.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5782000000000003e-05 2024-09-17T22:41:25.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.879879+0000", 2024-09-17T22:41:25.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8314999999999999e-05 2024-09-17T22:41:25.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.213 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.213 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:998 3.5 3:a0207ec9:::rbd_data.115f1ca54530.0000000000000002:head [stat,zero 13500416~8126464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.213 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.889597+0000", 2024-09-17T22:41:25.214 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.24297057899999999, 2024-09-17T22:41:25.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001430005, 2024-09-17T22:41:25.214 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.214 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.214 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:25.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.214 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 998 2024-09-17T22:41:25.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.214 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889597+0000", 2024-09-17T22:41:25.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889597+0000", 2024-09-17T22:41:25.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889599+0000", 2024-09-17T22:41:25.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.024e-06 2024-09-17T22:41:25.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889607+0000", 2024-09-17T22:41:25.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7039999999999994e-06 2024-09-17T22:41:25.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889626+0000", 2024-09-17T22:41:25.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9480999999999999e-05 2024-09-17T22:41:25.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889636+0000", 2024-09-17T22:41:25.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2299999999999997e-06 2024-09-17T22:41:25.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889719+0000", 2024-09-17T22:41:25.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2896000000000001e-05 2024-09-17T22:41:25.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889793+0000", 2024-09-17T22:41:25.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4880000000000001e-05 2024-09-17T22:41:25.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.891027+0000", 2024-09-17T22:41:25.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00123379 2024-09-17T22:41:25.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.217 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.218 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:1003 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:head [stat,truncate 13369344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.218 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.889674+0000", 2024-09-17T22:41:25.218 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.24289387200000001, 2024-09-17T22:41:25.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0022266640000000002, 2024-09-17T22:41:25.218 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.218 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.218 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.218 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1003 2024-09-17T22:41:25.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.218 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889674+0000", 2024-09-17T22:41:25.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889674+0000", 2024-09-17T22:41:25.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889674+0000", 2024-09-17T22:41:25.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2399999999999998e-07 2024-09-17T22:41:25.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889685+0000", 2024-09-17T22:41:25.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0137e-05 2024-09-17T22:41:25.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889692+0000", 2024-09-17T22:41:25.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9130000000000003e-06 2024-09-17T22:41:25.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889701+0000", 2024-09-17T22:41:25.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5490000000000006e-06 2024-09-17T22:41:25.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.890033+0000", 2024-09-17T22:41:25.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00033210599999999998 2024-09-17T22:41:25.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.890106+0000", 2024-09-17T22:41:25.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2636e-05 2024-09-17T22:41:25.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:25.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.890231+0000", 2024-09-17T22:41:25.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012568599999999999 2024-09-17T22:41:25.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.891108+0000", 2024-09-17T22:41:25.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00087691400000000001 2024-09-17T22:41:25.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.891793+0000", 2024-09-17T22:41:25.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0006845 2024-09-17T22:41:25.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.891858+0000", 2024-09-17T22:41:25.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5523000000000003e-05 2024-09-17T22:41:25.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.891901+0000", 2024-09-17T22:41:25.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2175999999999997e-05 2024-09-17T22:41:25.223 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.223 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.223 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.223 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:1017 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.224 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.899758+0000", 2024-09-17T22:41:25.224 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.23280978399999999, 2024-09-17T22:41:25.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035724300000000001, 2024-09-17T22:41:25.224 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.224 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.224 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.224 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1017 2024-09-17T22:41:25.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.224 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.899758+0000", 2024-09-17T22:41:25.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.899758+0000", 2024-09-17T22:41:25.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.899759+0000", 2024-09-17T22:41:25.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7000000000000003e-07 2024-09-17T22:41:25.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.899764+0000", 2024-09-17T22:41:25.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.322e-06 2024-09-17T22:41:25.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.899778+0000", 2024-09-17T22:41:25.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3346000000000001e-05 2024-09-17T22:41:25.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.899785+0000", 2024-09-17T22:41:25.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7039999999999994e-06 2024-09-17T22:41:25.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.899832+0000", 2024-09-17T22:41:25.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6796999999999999e-05 2024-09-17T22:41:25.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.899907+0000", 2024-09-17T22:41:25.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4869000000000004e-05 2024-09-17T22:41:25.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:25.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.900486+0000", 2024-09-17T22:41:25.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057868899999999996 2024-09-17T22:41:25.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.903172+0000", 2024-09-17T22:41:25.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026862969999999998 2024-09-17T22:41:25.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.903220+0000", 2024-09-17T22:41:25.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8405999999999997e-05 2024-09-17T22:41:25.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.903281+0000", 2024-09-17T22:41:25.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0739000000000002e-05 2024-09-17T22:41:25.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.903330+0000", 2024-09-17T22:41:25.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9291e-05 2024-09-17T22:41:25.229 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.229 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.229 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.229 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4447.0:1017 3.5 e29/23 ondisk, result = 0)", 2024-09-17T22:41:25.229 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.902115+0000", 2024-09-17T22:41:25.229 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.230453042, 2024-09-17T22:41:25.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001223607, 2024-09-17T22:41:25.229 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.230 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.230 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:25.230 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.902115+0000", 2024-09-17T22:41:25.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.902115+0000", 2024-09-17T22:41:25.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.902116+0000", 2024-09-17T22:41:25.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0699999999999996e-07 2024-09-17T22:41:25.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.902120+0000", 2024-09-17T22:41:25.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7709999999999997e-06 2024-09-17T22:41:25.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.902130+0000", 2024-09-17T22:41:25.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0023e-05 2024-09-17T22:41:25.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.902136+0000", 2024-09-17T22:41:25.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.694e-06 2024-09-17T22:41:25.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.903202+0000", 2024-09-17T22:41:25.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010663529999999999 2024-09-17T22:41:25.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.903213+0000", 2024-09-17T22:41:25.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0692e-05 2024-09-17T22:41:25.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.903338+0000", 2024-09-17T22:41:25.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000125167 2024-09-17T22:41:25.233 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.233 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.233 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.233 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4447.0:1016 3.1e e29/23 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:head v 29'11, mlcod=29'9)", 2024-09-17T22:41:25.233 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.908732+0000", 2024-09-17T22:41:25.233 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.22383556900000001, 2024-09-17T22:41:25.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017920569999999999, 2024-09-17T22:41:25.233 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.233 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.233 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.233 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.908732+0000", 2024-09-17T22:41:25.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.908732+0000", 2024-09-17T22:41:25.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.908733+0000", 2024-09-17T22:41:25.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2699999999999998e-07 2024-09-17T22:41:25.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.909752+0000", 2024-09-17T22:41:25.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001019289 2024-09-17T22:41:25.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.909765+0000", 2024-09-17T22:41:25.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2231e-05 2024-09-17T22:41:25.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.909773+0000", 2024-09-17T22:41:25.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.028e-06 2024-09-17T22:41:25.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.909819+0000", 2024-09-17T22:41:25.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6239999999999998e-05 2024-09-17T22:41:25.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.909834+0000", 2024-09-17T22:41:25.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5671e-05 2024-09-17T22:41:25.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.910472+0000", 2024-09-17T22:41:25.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00063788199999999999 2024-09-17T22:41:25.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.910524+0000", 2024-09-17T22:41:25.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1888999999999997e-05 2024-09-17T22:41:25.237 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.237 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.237 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.237 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:1031 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:head [stat,write 3376407~31465] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.237 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.935143+0000", 2024-09-17T22:41:25.237 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.19742483199999999, 2024-09-17T22:41:25.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001371356, 2024-09-17T22:41:25.237 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.237 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.237 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.238 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.238 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.238 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.238 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1031 2024-09-17T22:41:25.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.238 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.935143+0000", 2024-09-17T22:41:25.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.935143+0000", 2024-09-17T22:41:25.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.935145+0000", 2024-09-17T22:41:25.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.542e-06 2024-09-17T22:41:25.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.935169+0000", 2024-09-17T22:41:25.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3776000000000001e-05 2024-09-17T22:41:25.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.935190+0000", 2024-09-17T22:41:25.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0699999999999998e-05 2024-09-17T22:41:25.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.935201+0000", 2024-09-17T22:41:25.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0664e-05 2024-09-17T22:41:25.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.935256+0000", 2024-09-17T22:41:25.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5003999999999997e-05 2024-09-17T22:41:25.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.935320+0000", 2024-09-17T22:41:25.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3932999999999999e-05 2024-09-17T22:41:25.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:25.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.935410+0000", 2024-09-17T22:41:25.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0218999999999998e-05 2024-09-17T22:41:25.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.936158+0000", 2024-09-17T22:41:25.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00074790499999999999 2024-09-17T22:41:25.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.936433+0000", 2024-09-17T22:41:25.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027491800000000003 2024-09-17T22:41:25.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.936475+0000", 2024-09-17T22:41:25.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2239e-05 2024-09-17T22:41:25.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.936514+0000", 2024-09-17T22:41:25.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9456000000000003e-05 2024-09-17T22:41:25.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.242 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.242 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.242 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:25.244 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:41:25.244 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:25.241+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:25.344 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:25.342+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:25.348 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:25.348 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:25.348 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:41:25.348 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:25.357 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:41:25.445 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:25.442+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:25.469 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.7 to 1 (10000) 2024-09-17T22:41:25.484 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T22:41:25.484 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:41:25.507 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:25.507 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:25.508 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:41:25.508 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:41:25.508 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:25.517 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:41:25.536 INFO:teuthology.orchestra.run.smithi115.stdout:flattening image image_client.0-clone1 2024-09-17T22:41:25.545 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:25.543+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:25.646 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:25.644+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:25.648 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:25.648 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:41:25.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:41:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:41:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4229.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-17T22:41:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:09.387180+0000", 2024-09-17T22:41:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.257641938999999, 2024-09-17T22:41:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049037519999999999, 2024-09-17T22:41:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4229", 2024-09-17T22:41:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/3080427038", 2024-09-17T22:41:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-17T22:41:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.387180+0000", 2024-09-17T22:41:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.387180+0000", 2024-09-17T22:41:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.387184+0000", 2024-09-17T22:41:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0210000000000001e-06 2024-09-17T22:41:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.387190+0000", 2024-09-17T22:41:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8239999999999996e-06 2024-09-17T22:41:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.387201+0000", 2024-09-17T22:41:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1184e-05 2024-09-17T22:41:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.387211+0000", 2024-09-17T22:41:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7990000000000005e-06 2024-09-17T22:41:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.387277+0000", 2024-09-17T22:41:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6397999999999997e-05 2024-09-17T22:41:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:41:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.387285+0000", 2024-09-17T22:41:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.678e-06 2024-09-17T22:41:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.388709+0000", 2024-09-17T22:41:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014240259999999999 2024-09-17T22:41:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.388862+0000", 2024-09-17T22:41:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015260999999999999 2024-09-17T22:41:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-09-17T22:41:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.388973+0000", 2024-09-17T22:41:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000111235 2024-09-17T22:41:25.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.390141+0000", 2024-09-17T22:41:25.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001168522 2024-09-17T22:41:25.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.392011+0000", 2024-09-17T22:41:25.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001869346 2024-09-17T22:41:25.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.392054+0000", 2024-09-17T22:41:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2707000000000003e-05 2024-09-17T22:41:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.392084+0000", 2024-09-17T22:41:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0402000000000001e-05 2024-09-17T22:41:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4229.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-17T22:41:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:41:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 16.250212440999999, 2024-09-17T22:41:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25708877200000002, 2024-09-17T22:41:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-17T22:41:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4229", 2024-09-17T22:41:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/3080427038", 2024-09-17T22:41:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2 2024-09-17T22:41:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:41:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:41:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394611+0000", 2024-09-17T22:41:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3149999999999999e-06 2024-09-17T22:41:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394619+0000", 2024-09-17T22:41:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240000000000003e-06 2024-09-17T22:41:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394632+0000", 2024-09-17T22:41:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2921e-05 2024-09-17T22:41:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394639+0000", 2024-09-17T22:41:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-09-17T22:41:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394703+0000", 2024-09-17T22:41:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4407999999999997e-05 2024-09-17T22:41:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:41:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394711+0000", 2024-09-17T22:41:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5939999999999996e-06 2024-09-17T22:41:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651545+0000", 2024-09-17T22:41:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25683409200000001 2024-09-17T22:41:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651698+0000", 2024-09-17T22:41:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000153678 2024-09-17T22:41:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4229.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-17T22:41:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:10.397774+0000", 2024-09-17T22:41:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.247048211999999, 2024-09-17T22:41:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033808670000000001, 2024-09-17T22:41:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4229", 2024-09-17T22:41:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/3080427038", 2024-09-17T22:41:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5 2024-09-17T22:41:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:10.397774+0000", 2024-09-17T22:41:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:10.397774+0000", 2024-09-17T22:41:25.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:10.397776+0000", 2024-09-17T22:41:25.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5840000000000002e-06 2024-09-17T22:41:25.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:10.397791+0000", 2024-09-17T22:41:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4469e-05 2024-09-17T22:41:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:10.397813+0000", 2024-09-17T22:41:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1688999999999999e-05 2024-09-17T22:41:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:10.397823+0000", 2024-09-17T22:41:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0036e-05 2024-09-17T22:41:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:10.397900+0000", 2024-09-17T22:41:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7725000000000005e-05 2024-09-17T22:41:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op must wait for map", 2024-09-17T22:41:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:10.397906+0000", 2024-09-17T22:41:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4040000000000003e-06 2024-09-17T22:41:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:10.399534+0000", 2024-09-17T22:41:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016278320000000001 2024-09-17T22:41:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:10.399630+0000", 2024-09-17T22:41:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6143000000000004e-05 2024-09-17T22:41:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:10.399734+0000", 2024-09-17T22:41:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010375299999999999 2024-09-17T22:41:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:10.400452+0000", 2024-09-17T22:41:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000718737 2024-09-17T22:41:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:10.401087+0000", 2024-09-17T22:41:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000634805 2024-09-17T22:41:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:10.401124+0000", 2024-09-17T22:41:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6625999999999998e-05 2024-09-17T22:41:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:10.401155+0000", 2024-09-17T22:41:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1063999999999999e-05 2024-09-17T22:41:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4229.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-17T22:41:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:11.399497+0000", 2024-09-17T22:41:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.245324807999999, 2024-09-17T22:41:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018309019999999999, 2024-09-17T22:41:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4229", 2024-09-17T22:41:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/3080427038", 2024-09-17T22:41:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 7 2024-09-17T22:41:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:11.399497+0000", 2024-09-17T22:41:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:11.399497+0000", 2024-09-17T22:41:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:11.399499+0000", 2024-09-17T22:41:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4780000000000001e-06 2024-09-17T22:41:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:11.399506+0000", 2024-09-17T22:41:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5769999999999996e-06 2024-09-17T22:41:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:11.399520+0000", 2024-09-17T22:41:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3777999999999999e-05 2024-09-17T22:41:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:11.399529+0000", 2024-09-17T22:41:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1099999999999992e-06 2024-09-17T22:41:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:11.399588+0000", 2024-09-17T22:41:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8542999999999998e-05 2024-09-17T22:41:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:11.399663+0000", 2024-09-17T22:41:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4702000000000004e-05 2024-09-17T22:41:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:11.399776+0000", 2024-09-17T22:41:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000113292 2024-09-17T22:41:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:11.401166+0000", 2024-09-17T22:41:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013902509999999999 2024-09-17T22:41:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:11.401235+0000", 2024-09-17T22:41:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9210999999999999e-05 2024-09-17T22:41:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:11.401283+0000", 2024-09-17T22:41:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7858999999999997e-05 2024-09-17T22:41:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:11.401328+0000", 2024-09-17T22:41:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5101000000000001e-05 2024-09-17T22:41:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:553 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.512234+0000", 2024-09-17T22:41:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.1325876880000001, 2024-09-17T22:41:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001318334, 2024-09-17T22:41:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 553 2024-09-17T22:41:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.512234+0000", 2024-09-17T22:41:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.512234+0000", 2024-09-17T22:41:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.512237+0000", 2024-09-17T22:41:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3039999999999999e-06 2024-09-17T22:41:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.512245+0000", 2024-09-17T22:41:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9920000000000007e-06 2024-09-17T22:41:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.512261+0000", 2024-09-17T22:41:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5866999999999999e-05 2024-09-17T22:41:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.512268+0000", 2024-09-17T22:41:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1760000000000004e-06 2024-09-17T22:41:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.512330+0000", 2024-09-17T22:41:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2082000000000006e-05 2024-09-17T22:41:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.512465+0000", 2024-09-17T22:41:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000134735 2024-09-17T22:41:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-17T22:41:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.512654+0000", 2024-09-17T22:41:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018930499999999999 2024-09-17T22:41:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.513260+0000", 2024-09-17T22:41:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00060627499999999998 2024-09-17T22:41:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.513493+0000", 2024-09-17T22:41:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00023241700000000001 2024-09-17T22:41:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.513526+0000", 2024-09-17T22:41:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3423000000000002e-05 2024-09-17T22:41:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.513553+0000", 2024-09-17T22:41:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6758e-05 2024-09-17T22:41:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:698 3.1d 3:bd2464a5:::rbd_data.115f41cb28ed.000000000000000f:3 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.702115+0000", 2024-09-17T22:41:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.94270742299999999, 2024-09-17T22:41:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010710629999999999, 2024-09-17T22:41:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 698 2024-09-17T22:41:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.702115+0000", 2024-09-17T22:41:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.702115+0000", 2024-09-17T22:41:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.702115+0000", 2024-09-17T22:41:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.34e-07 2024-09-17T22:41:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.702117+0000", 2024-09-17T22:41:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2579999999999998e-06 2024-09-17T22:41:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.702126+0000", 2024-09-17T22:41:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4230000000000005e-06 2024-09-17T22:41:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.702131+0000", 2024-09-17T22:41:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5969999999999999e-06 2024-09-17T22:41:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.702687+0000", 2024-09-17T22:41:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00055551199999999998 2024-09-17T22:41:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.702759+0000", 2024-09-17T22:41:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2441000000000007e-05 2024-09-17T22:41:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.703186+0000", 2024-09-17T22:41:25.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00042639799999999998 2024-09-17T22:41:25.688 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.688 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.688 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.688 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:866 3.1d 3:bd2464a5:::rbd_data.115f41cb28ed.000000000000000f:3 [sparse-read 917504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.688 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.729000+0000", 2024-09-17T22:41:25.688 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.91582250499999995, 2024-09-17T22:41:25.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001005695, 2024-09-17T22:41:25.688 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.689 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.689 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:25.689 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.689 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.689 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.689 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 866 2024-09-17T22:41:25.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.689 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729000+0000", 2024-09-17T22:41:25.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729000+0000", 2024-09-17T22:41:25.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729000+0000", 2024-09-17T22:41:25.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2e-07 2024-09-17T22:41:25.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729002+0000", 2024-09-17T22:41:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1890000000000001e-06 2024-09-17T22:41:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729008+0000", 2024-09-17T22:41:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1240000000000002e-06 2024-09-17T22:41:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729015+0000", 2024-09-17T22:41:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1210000000000001e-06 2024-09-17T22:41:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729799+0000", 2024-09-17T22:41:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00078392499999999999 2024-09-17T22:41:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729875+0000", 2024-09-17T22:41:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5527999999999999e-05 2024-09-17T22:41:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.730005+0000", 2024-09-17T22:41:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000130388 2024-09-17T22:41:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:867 3.1d 3:bd2464a5:::rbd_data.115f41cb28ed.000000000000000f:3 [sparse-read 1966080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.729022+0000", 2024-09-17T22:41:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.91580026699999995, 2024-09-17T22:41:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011593549999999999, 2024-09-17T22:41:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 867 2024-09-17T22:41:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729022+0000", 2024-09-17T22:41:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729022+0000", 2024-09-17T22:41:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729022+0000", 2024-09-17T22:41:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-09-17T22:41:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729024+0000", 2024-09-17T22:41:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0229999999999999e-06 2024-09-17T22:41:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729030+0000", 2024-09-17T22:41:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0639999999999999e-06 2024-09-17T22:41:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729035+0000", 2024-09-17T22:41:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6920000000000001e-06 2024-09-17T22:41:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.730066+0000", 2024-09-17T22:41:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010306569999999999 2024-09-17T22:41:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.730113+0000", 2024-09-17T22:41:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6934000000000002e-05 2024-09-17T22:41:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.730181+0000", 2024-09-17T22:41:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8602000000000005e-05 2024-09-17T22:41:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:868 3.1d 3:bd2464a5:::rbd_data.115f41cb28ed.000000000000000f:3 [sparse-read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.729041+0000", 2024-09-17T22:41:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.91578093999999999, 2024-09-17T22:41:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001303186, 2024-09-17T22:41:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 868 2024-09-17T22:41:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729041+0000", 2024-09-17T22:41:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729041+0000", 2024-09-17T22:41:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729042+0000", 2024-09-17T22:41:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-09-17T22:41:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729051+0000", 2024-09-17T22:41:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.747e-06 2024-09-17T22:41:25.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729058+0000", 2024-09-17T22:41:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1439999999999997e-06 2024-09-17T22:41:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729064+0000", 2024-09-17T22:41:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2310000000000004e-06 2024-09-17T22:41:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.730213+0000", 2024-09-17T22:41:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011489479999999999 2024-09-17T22:41:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.730274+0000", 2024-09-17T22:41:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1606999999999995e-05 2024-09-17T22:41:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.730344+0000", 2024-09-17T22:41:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0112999999999998e-05 2024-09-17T22:41:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:869 3.1d 3:bd2464a5:::rbd_data.115f41cb28ed.000000000000000f:3 [sparse-read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.729070+0000", 2024-09-17T22:41:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.91575171600000005, 2024-09-17T22:41:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014872279999999999, 2024-09-17T22:41:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 869 2024-09-17T22:41:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729070+0000", 2024-09-17T22:41:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729070+0000", 2024-09-17T22:41:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729071+0000", 2024-09-17T22:41:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:41:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729073+0000", 2024-09-17T22:41:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0980000000000001e-06 2024-09-17T22:41:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729078+0000", 2024-09-17T22:41:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3820000000000003e-06 2024-09-17T22:41:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.729083+0000", 2024-09-17T22:41:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9490000000000004e-06 2024-09-17T22:41:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.730378+0000", 2024-09-17T22:41:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012947970000000001 2024-09-17T22:41:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.730461+0000", 2024-09-17T22:41:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3108000000000005e-05 2024-09-17T22:41:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.730558+0000", 2024-09-17T22:41:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6428999999999994e-05 2024-09-17T22:41:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:889 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 7208960~20709376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.731539+0000", 2024-09-17T22:41:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.91328264599999998, 2024-09-17T22:41:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011537310000000001, 2024-09-17T22:41:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 889 2024-09-17T22:41:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.731539+0000", 2024-09-17T22:41:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.731539+0000", 2024-09-17T22:41:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.731541+0000", 2024-09-17T22:41:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0780000000000001e-06 2024-09-17T22:41:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.731546+0000", 2024-09-17T22:41:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0610000000000003e-06 2024-09-17T22:41:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.731559+0000", 2024-09-17T22:41:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3794e-05 2024-09-17T22:41:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.731568+0000", 2024-09-17T22:41:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7930000000000003e-06 2024-09-17T22:41:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.731613+0000", 2024-09-17T22:41:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4443999999999997e-05 2024-09-17T22:41:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.731709+0000", 2024-09-17T22:41:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6366999999999994e-05 2024-09-17T22:41:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-17T22:41:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.731826+0000", 2024-09-17T22:41:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116896 2024-09-17T22:41:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.732387+0000", 2024-09-17T22:41:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00056154000000000002 2024-09-17T22:41:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.732600+0000", 2024-09-17T22:41:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021304599999999999 2024-09-17T22:41:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.732652+0000", 2024-09-17T22:41:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1839999999999998e-05 2024-09-17T22:41:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.732693+0000", 2024-09-17T22:41:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0871999999999998e-05 2024-09-17T22:41:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:913 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:head [stat,zero 24361720~17672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.759080+0000", 2024-09-17T22:41:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.88574206899999997, 2024-09-17T22:41:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010706719999999999, 2024-09-17T22:41:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 913 2024-09-17T22:41:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.759080+0000", 2024-09-17T22:41:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.759080+0000", 2024-09-17T22:41:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.759082+0000", 2024-09-17T22:41:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.351e-06 2024-09-17T22:41:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.759094+0000", 2024-09-17T22:41:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1231e-05 2024-09-17T22:41:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.759115+0000", 2024-09-17T22:41:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1005e-05 2024-09-17T22:41:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.759125+0000", 2024-09-17T22:41:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0304e-05 2024-09-17T22:41:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.759186+0000", 2024-09-17T22:41:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1258000000000002e-05 2024-09-17T22:41:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.759243+0000", 2024-09-17T22:41:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7246e-05 2024-09-17T22:41:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.760151+0000", 2024-09-17T22:41:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00090727699999999995 2024-09-17T22:41:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:935 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:head [stat,write 15466496~41196] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.792777+0000", 2024-09-17T22:41:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.85204479899999996, 2024-09-17T22:41:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014735900000000001, 2024-09-17T22:41:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 935 2024-09-17T22:41:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.792777+0000", 2024-09-17T22:41:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.792777+0000", 2024-09-17T22:41:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.792779+0000", 2024-09-17T22:41:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.779e-06 2024-09-17T22:41:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.792799+0000", 2024-09-17T22:41:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9777999999999999e-05 2024-09-17T22:41:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.792814+0000", 2024-09-17T22:41:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5031000000000001e-05 2024-09-17T22:41:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.792821+0000", 2024-09-17T22:41:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0659999999999998e-06 2024-09-17T22:41:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.792881+0000", 2024-09-17T22:41:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9830000000000001e-05 2024-09-17T22:41:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.792971+0000", 2024-09-17T22:41:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0134000000000001e-05 2024-09-17T22:41:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-17T22:41:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.793123+0000", 2024-09-17T22:41:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015223799999999999 2024-09-17T22:41:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.793890+0000", 2024-09-17T22:41:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00076697600000000005 2024-09-17T22:41:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.794179+0000", 2024-09-17T22:41:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028919500000000001 2024-09-17T22:41:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.794212+0000", 2024-09-17T22:41:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2257e-05 2024-09-17T22:41:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.794251+0000", 2024-09-17T22:41:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9305999999999999e-05 2024-09-17T22:41:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4447.0:939 3.12 e29/23 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:head v 29'6, mlcod=29'4)", 2024-09-17T22:41:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.817503+0000", 2024-09-17T22:41:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.82731865000000004, 2024-09-17T22:41:25.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013997770000000001, 2024-09-17T22:41:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.817503+0000", 2024-09-17T22:41:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.817503+0000", 2024-09-17T22:41:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.817505+0000", 2024-09-17T22:41:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3599999999999999e-06 2024-09-17T22:41:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.818145+0000", 2024-09-17T22:41:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00063999499999999997 2024-09-17T22:41:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.818161+0000", 2024-09-17T22:41:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6225000000000001e-05 2024-09-17T22:41:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.818169+0000", 2024-09-17T22:41:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1829999999999994e-06 2024-09-17T22:41:25.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.818228+0000", 2024-09-17T22:41:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8489000000000002e-05 2024-09-17T22:41:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.818250+0000", 2024-09-17T22:41:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2407000000000001e-05 2024-09-17T22:41:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.818871+0000", 2024-09-17T22:41:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00062129999999999998 2024-09-17T22:41:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.818903+0000", 2024-09-17T22:41:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1817999999999999e-05 2024-09-17T22:41:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:943 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:head [stat,truncate 2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.838768+0000", 2024-09-17T22:41:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.80605367900000002, 2024-09-17T22:41:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002188073, 2024-09-17T22:41:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 943 2024-09-17T22:41:25.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838768+0000", 2024-09-17T22:41:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838768+0000", 2024-09-17T22:41:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838771+0000", 2024-09-17T22:41:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9229999999999998e-06 2024-09-17T22:41:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838784+0000", 2024-09-17T22:41:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2193e-05 2024-09-17T22:41:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838806+0000", 2024-09-17T22:41:25.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2116999999999998e-05 2024-09-17T22:41:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838818+0000", 2024-09-17T22:41:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.203e-05 2024-09-17T22:41:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838903+0000", 2024-09-17T22:41:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4882999999999994e-05 2024-09-17T22:41:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.838992+0000", 2024-09-17T22:41:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8968999999999995e-05 2024-09-17T22:41:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-17T22:41:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839156+0000", 2024-09-17T22:41:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000163943 2024-09-17T22:41:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839904+0000", 2024-09-17T22:41:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00074882099999999999 2024-09-17T22:41:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.840848+0000", 2024-09-17T22:41:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00094390299999999995 2024-09-17T22:41:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.840901+0000", 2024-09-17T22:41:25.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3100999999999999e-05 2024-09-17T22:41:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.840957+0000", 2024-09-17T22:41:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5189999999999998e-05 2024-09-17T22:41:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4447.0:941 3.12 e29/23 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:head v 29'7, mlcod=29'5)", 2024-09-17T22:41:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.839294+0000", 2024-09-17T22:41:25.743 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.80552801799999996, 2024-09-17T22:41:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001710128, 2024-09-17T22:41:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839294+0000", 2024-09-17T22:41:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839294+0000", 2024-09-17T22:41:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839295+0000", 2024-09-17T22:41:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.023e-06 2024-09-17T22:41:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839304+0000", 2024-09-17T22:41:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6370000000000005e-06 2024-09-17T22:41:25.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839335+0000", 2024-09-17T22:41:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1560000000000003e-05 2024-09-17T22:41:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839346+0000", 2024-09-17T22:41:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1022999999999999e-05 2024-09-17T22:41:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839393+0000", 2024-09-17T22:41:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6455000000000002e-05 2024-09-17T22:41:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.839412+0000", 2024-09-17T22:41:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9258000000000001e-05 2024-09-17T22:41:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.840966+0000", 2024-09-17T22:41:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015537109999999999 2024-09-17T22:41:25.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.841004+0000", 2024-09-17T22:41:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8461000000000003e-05 2024-09-17T22:41:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:966 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:head [stat,write 16358361~25639] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.748 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.863548+0000", 2024-09-17T22:41:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.78127369199999996, 2024-09-17T22:41:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001348623, 2024-09-17T22:41:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 966 2024-09-17T22:41:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.863548+0000", 2024-09-17T22:41:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.863548+0000", 2024-09-17T22:41:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.863550+0000", 2024-09-17T22:41:25.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.734e-06 2024-09-17T22:41:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.863567+0000", 2024-09-17T22:41:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7005999999999999e-05 2024-09-17T22:41:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.863582+0000", 2024-09-17T22:41:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4691e-05 2024-09-17T22:41:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.863594+0000", 2024-09-17T22:41:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2488e-05 2024-09-17T22:41:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.863655+0000", 2024-09-17T22:41:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0909000000000002e-05 2024-09-17T22:41:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.863713+0000", 2024-09-17T22:41:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7785000000000003e-05 2024-09-17T22:41:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-17T22:41:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.863804+0000", 2024-09-17T22:41:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1127999999999999e-05 2024-09-17T22:41:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.864630+0000", 2024-09-17T22:41:25.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000825971 2024-09-17T22:41:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.864836+0000", 2024-09-17T22:41:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020535599999999999 2024-09-17T22:41:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.864869+0000", 2024-09-17T22:41:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3429000000000001e-05 2024-09-17T22:41:25.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.864897+0000", 2024-09-17T22:41:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8126000000000001e-05 2024-09-17T22:41:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:982 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:head [stat,zero 21626880~7077888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.878166+0000", 2024-09-17T22:41:25.755 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.76665660199999996, 2024-09-17T22:41:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00175706, 2024-09-17T22:41:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 982 2024-09-17T22:41:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878166+0000", 2024-09-17T22:41:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878166+0000", 2024-09-17T22:41:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878168+0000", 2024-09-17T22:41:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4150000000000002e-06 2024-09-17T22:41:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878176+0000", 2024-09-17T22:41:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0579999999999998e-06 2024-09-17T22:41:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878196+0000", 2024-09-17T22:41:25.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9894999999999998e-05 2024-09-17T22:41:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878209+0000", 2024-09-17T22:41:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2774e-05 2024-09-17T22:41:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878289+0000", 2024-09-17T22:41:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9986000000000003e-05 2024-09-17T22:41:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.878386+0000", 2024-09-17T22:41:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7232999999999996e-05 2024-09-17T22:41:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.879923+0000", 2024-09-17T22:41:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015366990000000001 2024-09-17T22:41:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:999 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:head [stat,truncate 13440345] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.889612+0000", 2024-09-17T22:41:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.75521022599999998, 2024-09-17T22:41:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014142180000000001, 2024-09-17T22:41:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:25.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 999 2024-09-17T22:41:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889612+0000", 2024-09-17T22:41:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889612+0000", 2024-09-17T22:41:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889614+0000", 2024-09-17T22:41:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0999999999999998e-06 2024-09-17T22:41:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889622+0000", 2024-09-17T22:41:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1859999999999992e-06 2024-09-17T22:41:25.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889641+0000", 2024-09-17T22:41:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9004e-05 2024-09-17T22:41:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889651+0000", 2024-09-17T22:41:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0088000000000001e-05 2024-09-17T22:41:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889719+0000", 2024-09-17T22:41:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7872000000000003e-05 2024-09-17T22:41:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889813+0000", 2024-09-17T22:41:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3494999999999998e-05 2024-09-17T22:41:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-09-17T22:41:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.889937+0000", 2024-09-17T22:41:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012470499999999999 2024-09-17T22:41:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.890575+0000", 2024-09-17T22:41:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00063745999999999996 2024-09-17T22:41:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:25.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.890942+0000", 2024-09-17T22:41:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00036695500000000003 2024-09-17T22:41:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.890980+0000", 2024-09-17T22:41:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7981000000000001e-05 2024-09-17T22:41:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.891026+0000", 2024-09-17T22:41:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6372000000000003e-05 2024-09-17T22:41:25.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4447.0:1013 3.12 e29/23 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:head v 29'10, mlcod=29'8)", 2024-09-17T22:41:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:24.897795+0000", 2024-09-17T22:41:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.74702742600000005, 2024-09-17T22:41:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0009924109999999999, 2024-09-17T22:41:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:25.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.897795+0000", 2024-09-17T22:41:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.897795+0000", 2024-09-17T22:41:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:25.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.897796+0000", 2024-09-17T22:41:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.026e-06 2024-09-17T22:41:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.897892+0000", 2024-09-17T22:41:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6599000000000001e-05 2024-09-17T22:41:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.897906+0000", 2024-09-17T22:41:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3658999999999999e-05 2024-09-17T22:41:25.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.897912+0000", 2024-09-17T22:41:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9410000000000004e-06 2024-09-17T22:41:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.897962+0000", 2024-09-17T22:41:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9910999999999997e-05 2024-09-17T22:41:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.897978+0000", 2024-09-17T22:41:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5775000000000001e-05 2024-09-17T22:41:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.898743+0000", 2024-09-17T22:41:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00076567699999999998 2024-09-17T22:41:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:25.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:24.898787+0000", 2024-09-17T22:41:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3822999999999997e-05 2024-09-17T22:41:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:25.773 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:25.773 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:25.775 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:41:25.775 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-17T22:41:25.776 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:25.772+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:25.808 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:41:25.809 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:41:25.820 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-09-17T22:41:25.465185+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":33305262,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":205,"num_read_kb":173,"num_write":207,"num_write_kb":3147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":170,"ondisk_log_size":170,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218704,"kb_used_data":2444,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756016,"statfs":{"total":773094113280,"available":772870160384,"internally_reserved":0,"allocated":2502656,"data_stored":2227411,"data_compressed":63783,"data_compressed_allocated":143360,"data_compressed_original":1314816,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32714875,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":89,"num_read_kb":73,"num_write":46,"num_write_kb":729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.019023"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.448946+0000","last_change":"2024-09-17T22:41:19.443915+0000","last_active":"2024-09-17T22:41:20.448946+0000","last_peered":"2024-09-17T22:41:20.448946+0000","last_clean":"2024-09-17T22:41:20.448946+0000","last_became_active":"2024-09-17T22:41:19.443658+0000","last_became_peered":"2024-09-17T22:41:19.443658+0000","last_unstale":"2024-09-17T22:41:20.448946+0000","last_undegraded":"2024-09-17T22:41:20.448946+0000","last_fullsized":"2024-09-17T22:41:20.448946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:24:54.635341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.448994+0000","last_change":"2024-09-17T22:41:19.449604+0000","last_active":"2024-09-17T22:41:20.448994+0000","last_peered":"2024-09-17T22:41:20.448994+0000","last_clean":"2024-09-17T22:41:20.448994+0000","last_became_active":"2024-09-17T22:41:19.449357+0000","last_became_peered":"2024-09-17T22:41:19.449357+0000","last_unstale":"2024-09-17T22:41:20.448994+0000","last_undegraded":"2024-09-17T22:41:20.448994+0000","last_fullsized":"2024-09-17T22:41:20.448994+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:22:57.833991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"28'1","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.730318+0000","last_change":"2024-09-17T22:41:21.446449+0000","last_active":"2024-09-17T22:41:24.730318+0000","last_peered":"2024-09-17T22:41:24.730318+0000","last_clean":"2024-09-17T22:41:24.730318+0000","last_became_active":"2024-09-17T22:41:19.447823+0000","last_became_peered":"2024-09-17T22:41:19.447823+0000","last_unstale":"2024-09-17T22:41:24.730318+0000","last_undegraded":"2024-09-17T22:41:24.730318+0000","last_fullsized":"2024-09-17T22:41:24.730318+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:01:38.780863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000382498,"stat_sum":{"num_bytes":3545871,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":1,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"26'1","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T22:41:21.453779+0000","last_change":"2024-09-17T22:41:21.446501+0000","last_active":"2024-09-17T22:41:21.453779+0000","last_peered":"2024-09-17T22:41:21.453779+0000","last_clean":"2024-09-17T22:41:21.453779+0000","last_became_active":"2024-09-17T22:41:19.452435+0000","last_became_peered":"2024-09-17T22:41:19.452435+0000","last_unstale":"2024-09-17T22:41:21.453779+0000","last_undegraded":"2024-09-17T22:41:21.453779+0000","last_fullsized":"2024-09-17T22:41:21.453779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:48.072503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001451078,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.445389+0000","last_change":"2024-09-17T22:41:19.450099+0000","last_active":"2024-09-17T22:41:20.445389+0000","last_peered":"2024-09-17T22:41:20.445389+0000","last_clean":"2024-09-17T22:41:20.445389+0000","last_became_active":"2024-09-17T22:41:19.449830+0000","last_became_peered":"2024-09-17T22:41:19.449830+0000","last_unstale":"2024-09-17T22:41:20.445389+0000","last_undegraded":"2024-09-17T22:41:20.445389+0000","last_fullsized":"2024-09-17T22:41:20.445389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:13:09.570462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"26'2","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T22:41:21.471424+0000","last_change":"2024-09-17T22:41:21.446945+0000","last_active":"2024-09-17T22:41:21.471424+0000","last_peered":"2024-09-17T22:41:21.471424+0000","last_clean":"2024-09-17T22:41:21.471424+0000","last_became_active":"2024-09-17T22:41:19.446541+0000","last_became_peered":"2024-09-17T22:41:19.446541+0000","last_unstale":"2024-09-17T22:41:21.471424+0000","last_undegraded":"2024-09-17T22:41:21.471424+0000","last_fullsized":"2024-09-17T22:41:21.471424+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:54.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001630631,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.445412+0000","last_change":"2024-09-17T22:41:19.450227+0000","last_active":"2024-09-17T22:41:20.445412+0000","last_peered":"2024-09-17T22:41:20.445412+0000","last_clean":"2024-09-17T22:41:20.445412+0000","last_became_active":"2024-09-17T22:41:19.449940+0000","last_became_peered":"2024-09-17T22:41:19.449940+0000","last_unstale":"2024-09-17T22:41:20.445412+0000","last_undegraded":"2024-09-17T22:41:20.445412+0000","last_fullsized":"2024-09-17T22:41:20.445412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:40:32.886261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.448962+0000","last_change":"2024-09-17T22:41:19.446570+0000","last_active":"2024-09-17T22:41:20.448962+0000","last_peered":"2024-09-17T22:41:20.448962+0000","last_clean":"2024-09-17T22:41:20.448962+0000","last_became_active":"2024-09-17T22:41:19.446287+0000","last_became_peered":"2024-09-17T22:41:19.446287+0000","last_unstale":"2024-09-17T22:41:20.448962+0000","last_undegraded":"2024-09-17T22:41:20.448962+0000","last_fullsized":"2024-09-17T22:41:20.448962+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:29.795127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.449064+0000","last_change":"2024-09-17T22:41:19.446207+0000","last_active":"2024-09-17T22:41:20.449064+0000","last_peered":"2024-09-17T22:41:20.449064+0000","last_clean":"2024-09-17T22:41:20.449064+0000","last_became_active":"2024-09-17T22:41:19.445923+0000","last_became_peered":"2024-09-17T22:41:19.445923+0000","last_unstale":"2024-09-17T22:41:20.449064+0000","last_undegraded":"2024-09-17T22:41:20.449064+0000","last_fullsized":"2024-09-17T22:41:20.449064+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:27:21.301851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"28'4","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.489869+0000","last_change":"2024-09-17T22:41:21.516305+0000","last_active":"2024-09-17T22:41:24.489869+0000","last_peered":"2024-09-17T22:41:24.489869+0000","last_clean":"2024-09-17T22:41:24.489869+0000","last_became_active":"2024-09-17T22:41:19.450587+0000","last_became_peered":"2024-09-17T22:41:19.450587+0000","last_unstale":"2024-09-17T22:41:24.489869+0000","last_undegraded":"2024-09-17T22:41:24.489869+0000","last_fullsized":"2024-09-17T22:41:24.489869+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:05:19.351470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000275164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T22:41:21.454760+0000","last_change":"2024-09-17T22:41:21.454760+0000","last_active":"2024-09-17T22:41:21.454760+0000","last_peered":"2024-09-17T22:41:21.454760+0000","last_clean":"2024-09-17T22:41:21.454760+0000","last_became_active":"2024-09-17T22:41:19.449329+0000","last_became_peered":"2024-09-17T22:41:19.449329+0000","last_unstale":"2024-09-17T22:41:21.454760+0000","last_undegraded":"2024-09-17T22:41:21.454760+0000","last_fullsized":"2024-09-17T22:41:21.454760+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:10:05.378151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041055799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.467439+0000","last_change":"2024-09-17T22:41:21.446210+0000","last_active":"2024-09-17T22:41:24.467439+0000","last_peered":"2024-09-17T22:41:24.467439+0000","last_clean":"2024-09-17T22:41:24.467439+0000","last_became_active":"2024-09-17T22:41:19.451631+0000","last_became_peered":"2024-09-17T22:41:19.451631+0000","last_unstale":"2024-09-17T22:41:24.467439+0000","last_undegraded":"2024-09-17T22:41:24.467439+0000","last_fullsized":"2024-09-17T22:41:24.467439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:28:37.033572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081193699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.467710+0000","last_change":"2024-09-17T22:41:11.397814+0000","last_active":"2024-09-17T22:41:24.467710+0000","last_peered":"2024-09-17T22:41:24.467710+0000","last_clean":"2024-09-17T22:41:24.467710+0000","last_became_active":"2024-09-17T22:41:09.506775+0000","last_became_peered":"2024-09-17T22:41:09.506775+0000","last_unstale":"2024-09-17T22:41:24.467710+0000","last_undegraded":"2024-09-17T22:41:24.467710+0000","last_fullsized":"2024-09-17T22:41:24.467710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:47:43.867775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.445455+0000","last_change":"2024-09-17T22:41:19.449258+0000","last_active":"2024-09-17T22:41:20.445455+0000","last_peered":"2024-09-17T22:41:20.445455+0000","last_clean":"2024-09-17T22:41:20.445455+0000","last_became_active":"2024-09-17T22:41:19.448966+0000","last_became_peered":"2024-09-17T22:41:19.448966+0000","last_unstale":"2024-09-17T22:41:20.445455+0000","last_undegraded":"2024-09-17T22:41:20.445455+0000","last_fullsized":"2024-09-17T22:41:20.445455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:51:35.372743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.445313+0000","last_change":"2024-09-17T22:41:11.395783+0000","last_active":"2024-09-17T22:41:20.445313+0000","last_peered":"2024-09-17T22:41:20.445313+0000","last_clean":"2024-09-17T22:41:20.445313+0000","last_became_active":"2024-09-17T22:41:09.507056+0000","last_became_peered":"2024-09-17T22:41:09.507056+0000","last_unstale":"2024-09-17T22:41:20.445313+0000","last_undegraded":"2024-09-17T22:41:20.445313+0000","last_fullsized":"2024-09-17T22:41:20.445313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:13.438783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.445388+0000","last_change":"2024-09-17T22:41:19.451905+0000","last_active":"2024-09-17T22:41:20.445388+0000","last_peered":"2024-09-17T22:41:20.445388+0000","last_clean":"2024-09-17T22:41:20.445388+0000","last_became_active":"2024-09-17T22:41:19.451637+0000","last_became_peered":"2024-09-17T22:41:19.451637+0000","last_unstale":"2024-09-17T22:41:20.445388+0000","last_undegraded":"2024-09-17T22:41:20.445388+0000","last_fullsized":"2024-09-17T22:41:20.445388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:36:48.942068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.468295+0000","last_change":"2024-09-17T22:41:11.397753+0000","last_active":"2024-09-17T22:41:24.468295+0000","last_peered":"2024-09-17T22:41:24.468295+0000","last_clean":"2024-09-17T22:41:24.468295+0000","last_became_active":"2024-09-17T22:41:09.387651+0000","last_became_peered":"2024-09-17T22:41:09.387651+0000","last_unstale":"2024-09-17T22:41:24.468295+0000","last_undegraded":"2024-09-17T22:41:24.468295+0000","last_fullsized":"2024-09-17T22:41:24.468295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:14:30.915047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.920491+0000","last_change":"2024-09-17T22:41:19.445172+0000","last_active":"2024-09-17T22:41:20.920491+0000","last_peered":"2024-09-17T22:41:20.920491+0000","last_clean":"2024-09-17T22:41:20.920491+0000","last_became_active":"2024-09-17T22:41:19.444893+0000","last_became_peered":"2024-09-17T22:41:19.444893+0000","last_unstale":"2024-09-17T22:41:20.920491+0000","last_undegraded":"2024-09-17T22:41:20.920491+0000","last_fullsized":"2024-09-17T22:41:20.920491+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:48.713940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.448848+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:41:20.448848+0000","last_peered":"2024-09-17T22:41:20.448848+0000","last_clean":"2024-09-17T22:41:20.448848+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:41:20.448848+0000","last_undegraded":"2024-09-17T22:41:20.448848+0000","last_fullsized":"2024-09-17T22:41:20.448848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"29'20","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.920676+0000","last_change":"2024-09-17T22:41:21.516349+0000","last_active":"2024-09-17T22:41:24.920676+0000","last_peered":"2024-09-17T22:41:24.920676+0000","last_clean":"2024-09-17T22:41:24.920676+0000","last_became_active":"2024-09-17T22:41:19.451723+0000","last_became_peered":"2024-09-17T22:41:19.451723+0000","last_unstale":"2024-09-17T22:41:24.920676+0000","last_undegraded":"2024-09-17T22:41:24.920676+0000","last_fullsized":"2024-09-17T22:41:24.920676+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:39:06.838900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040033199999999999,"stat_sum":{"num_bytes":15728640,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":20,"num_write_kb":326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T22:41:21.454999+0000","last_change":"2024-09-17T22:41:21.454999+0000","last_active":"2024-09-17T22:41:21.454999+0000","last_peered":"2024-09-17T22:41:21.454999+0000","last_clean":"2024-09-17T22:41:21.454999+0000","last_became_active":"2024-09-17T22:41:19.450477+0000","last_became_peered":"2024-09-17T22:41:19.450477+0000","last_unstale":"2024-09-17T22:41:21.454999+0000","last_undegraded":"2024-09-17T22:41:21.454999+0000","last_fullsized":"2024-09-17T22:41:21.454999+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:15:59.350460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063898899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":37,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.467922+0000","last_change":"2024-09-17T22:41:11.397488+0000","last_active":"2024-09-17T22:41:24.467922+0000","last_peered":"2024-09-17T22:41:24.467922+0000","last_clean":"2024-09-17T22:41:24.467922+0000","last_became_active":"2024-09-17T22:41:09.651390+0000","last_became_peered":"2024-09-17T22:41:09.651390+0000","last_unstale":"2024-09-17T22:41:24.467922+0000","last_undegraded":"2024-09-17T22:41:24.467922+0000","last_fullsized":"2024-09-17T22:41:24.467922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:00.598921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.448817+0000","last_change":"2024-09-17T22:41:19.444560+0000","last_active":"2024-09-17T22:41:20.448817+0000","last_peered":"2024-09-17T22:41:20.448817+0000","last_clean":"2024-09-17T22:41:20.448817+0000","last_became_active":"2024-09-17T22:41:19.444314+0000","last_became_peered":"2024-09-17T22:41:19.444314+0000","last_unstale":"2024-09-17T22:41:20.448817+0000","last_undegraded":"2024-09-17T22:41:20.448817+0000","last_fullsized":"2024-09-17T22:41:20.448817+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:51.835717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.920217+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:41:20.920217+0000","last_peered":"2024-09-17T22:41:20.920217+0000","last_clean":"2024-09-17T22:41:20.920217+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:41:20.920217+0000","last_undegraded":"2024-09-17T22:41:20.920217+0000","last_fullsized":"2024-09-17T22:41:20.920217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.920336+0000","last_change":"2024-09-17T22:41:19.450084+0000","last_active":"2024-09-17T22:41:20.920336+0000","last_peered":"2024-09-17T22:41:20.920336+0000","last_clean":"2024-09-17T22:41:20.920336+0000","last_became_active":"2024-09-17T22:41:19.449791+0000","last_became_peered":"2024-09-17T22:41:19.449791+0000","last_unstale":"2024-09-17T22:41:20.920336+0000","last_undegraded":"2024-09-17T22:41:20.920336+0000","last_fullsized":"2024-09-17T22:41:20.920336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:57:24.865653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.467024+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:41:24.467024+0000","last_peered":"2024-09-17T22:41:24.467024+0000","last_clean":"2024-09-17T22:41:24.467024+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:41:24.467024+0000","last_undegraded":"2024-09-17T22:41:24.467024+0000","last_fullsized":"2024-09-17T22:41:24.467024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.445354+0000","last_change":"2024-09-17T22:41:19.449591+0000","last_active":"2024-09-17T22:41:20.445354+0000","last_peered":"2024-09-17T22:41:20.445354+0000","last_clean":"2024-09-17T22:41:20.445354+0000","last_became_active":"2024-09-17T22:41:19.449279+0000","last_became_peered":"2024-09-17T22:41:19.449279+0000","last_unstale":"2024-09-17T22:41:20.445354+0000","last_undegraded":"2024-09-17T22:41:20.445354+0000","last_fullsized":"2024-09-17T22:41:20.445354+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:02:45.060890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.467511+0000","last_change":"2024-09-17T22:41:11.398108+0000","last_active":"2024-09-17T22:41:24.467511+0000","last_peered":"2024-09-17T22:41:24.467511+0000","last_clean":"2024-09-17T22:41:24.467511+0000","last_became_active":"2024-09-17T22:41:09.388184+0000","last_became_peered":"2024-09-17T22:41:09.388184+0000","last_unstale":"2024-09-17T22:41:24.467511+0000","last_undegraded":"2024-09-17T22:41:24.467511+0000","last_fullsized":"2024-09-17T22:41:24.467511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:43.506329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T22:41:21.444832+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:41:21.444832+0000","last_peered":"2024-09-17T22:41:21.444832+0000","last_clean":"2024-09-17T22:41:21.444832+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:41:21.444832+0000","last_undegraded":"2024-09-17T22:41:21.444832+0000","last_fullsized":"2024-09-17T22:41:21.444832+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.445326+0000","last_change":"2024-09-17T22:41:19.449256+0000","last_active":"2024-09-17T22:41:20.445326+0000","last_peered":"2024-09-17T22:41:20.445326+0000","last_clean":"2024-09-17T22:41:20.445326+0000","last_became_active":"2024-09-17T22:41:19.448956+0000","last_became_peered":"2024-09-17T22:41:19.448956+0000","last_unstale":"2024-09-17T22:41:20.445326+0000","last_undegraded":"2024-09-17T22:41:20.445326+0000","last_fullsized":"2024-09-17T22:41:20.445326+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:30:21.665573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.467319+0000","last_change":"2024-09-17T22:41:21.445421+0000","last_active":"2024-09-17T22:41:24.467319+0000","last_peered":"2024-09-17T22:41:24.467319+0000","last_clean":"2024-09-17T22:41:24.467319+0000","last_became_active":"2024-09-17T22:41:19.449003+0000","last_became_peered":"2024-09-17T22:41:19.449003+0000","last_unstale":"2024-09-17T22:41:24.467319+0000","last_undegraded":"2024-09-17T22:41:24.467319+0000","last_fullsized":"2024-09-17T22:41:24.467319+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:02:37.357737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051465400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"29'16","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.879874+0000","last_change":"2024-09-17T22:41:21.445728+0000","last_active":"2024-09-17T22:41:24.879874+0000","last_peered":"2024-09-17T22:41:24.879874+0000","last_clean":"2024-09-17T22:41:24.879874+0000","last_became_active":"2024-09-17T22:41:19.448011+0000","last_became_peered":"2024-09-17T22:41:19.448011+0000","last_unstale":"2024-09-17T22:41:24.879874+0000","last_undegraded":"2024-09-17T22:41:24.879874+0000","last_fullsized":"2024-09-17T22:41:24.879874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:57:14.142270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068494200000000002,"stat_sum":{"num_bytes":13440345,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":1,"num_write":13,"num_write_kb":199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.920327+0000","last_change":"2024-09-17T22:41:19.447493+0000","last_active":"2024-09-17T22:41:20.920327+0000","last_peered":"2024-09-17T22:41:20.920327+0000","last_clean":"2024-09-17T22:41:20.920327+0000","last_became_active":"2024-09-17T22:41:19.447215+0000","last_became_peered":"2024-09-17T22:41:19.447215+0000","last_unstale":"2024-09-17T22:41:20.920327+0000","last_undegraded":"2024-09-17T22:41:20.920327+0000","last_fullsized":"2024-09-17T22:41:20.920327+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:40.632223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"26'2","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T22:41:21.472904+0000","last_change":"2024-09-17T22:41:21.454754+0000","last_active":"2024-09-17T22:41:21.472904+0000","last_peered":"2024-09-17T22:41:21.472904+0000","last_clean":"2024-09-17T22:41:21.472904+0000","last_became_active":"2024-09-17T22:41:19.448928+0000","last_became_peered":"2024-09-17T22:41:19.448928+0000","last_unstale":"2024-09-17T22:41:21.472904+0000","last_undegraded":"2024-09-17T22:41:21.472904+0000","last_fullsized":"2024-09-17T22:41:21.472904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:00:49.892028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000395712,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":25,"num_read_kb":22,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.467854+0000","last_change":"2024-09-17T22:41:21.446562+0000","last_active":"2024-09-17T22:41:24.467854+0000","last_peered":"2024-09-17T22:41:24.467854+0000","last_clean":"2024-09-17T22:41:24.467854+0000","last_became_active":"2024-09-17T22:41:19.447631+0000","last_became_peered":"2024-09-17T22:41:19.447631+0000","last_unstale":"2024-09-17T22:41:24.467854+0000","last_undegraded":"2024-09-17T22:41:24.467854+0000","last_fullsized":"2024-09-17T22:41:24.467854+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:12.336781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000402657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.490011+0000","last_change":"2024-09-17T22:41:21.516505+0000","last_active":"2024-09-17T22:41:24.490011+0000","last_peered":"2024-09-17T22:41:24.490011+0000","last_clean":"2024-09-17T22:41:24.490011+0000","last_became_active":"2024-09-17T22:41:19.450839+0000","last_became_peered":"2024-09-17T22:41:19.450839+0000","last_unstale":"2024-09-17T22:41:24.490011+0000","last_undegraded":"2024-09-17T22:41:24.490011+0000","last_fullsized":"2024-09-17T22:41:24.490011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:00:58.916145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038538199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T22:41:21.454987+0000","last_change":"2024-09-17T22:41:21.454987+0000","last_active":"2024-09-17T22:41:21.454987+0000","last_peered":"2024-09-17T22:41:21.454987+0000","last_clean":"2024-09-17T22:41:21.454987+0000","last_became_active":"2024-09-17T22:41:19.451864+0000","last_became_peered":"2024-09-17T22:41:19.451864+0000","last_unstale":"2024-09-17T22:41:21.454987+0000","last_undegraded":"2024-09-17T22:41:21.454987+0000","last_fullsized":"2024-09-17T22:41:21.454987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:50:24.181205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000602379,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T22:41:21.446576+0000","last_change":"2024-09-17T22:41:21.446576+0000","last_active":"2024-09-17T22:41:21.446576+0000","last_peered":"2024-09-17T22:41:21.446576+0000","last_clean":"2024-09-17T22:41:21.446576+0000","last_became_active":"2024-09-17T22:41:19.447419+0000","last_became_peered":"2024-09-17T22:41:19.447419+0000","last_unstale":"2024-09-17T22:41:21.446576+0000","last_undegraded":"2024-09-17T22:41:21.446576+0000","last_fullsized":"2024-09-17T22:41:21.446576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:20:24.482993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001865071,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.467196+0000","last_change":"2024-09-17T22:41:21.445226+0000","last_active":"2024-09-17T22:41:24.467196+0000","last_peered":"2024-09-17T22:41:24.467196+0000","last_clean":"2024-09-17T22:41:24.467196+0000","last_became_active":"2024-09-17T22:41:19.454151+0000","last_became_peered":"2024-09-17T22:41:19.454151+0000","last_unstale":"2024-09-17T22:41:24.467196+0000","last_undegraded":"2024-09-17T22:41:24.467196+0000","last_fullsized":"2024-09-17T22:41:24.467196+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:40:15.336777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050545600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.920513+0000","last_change":"2024-09-17T22:41:19.446326+0000","last_active":"2024-09-17T22:41:20.920513+0000","last_peered":"2024-09-17T22:41:20.920513+0000","last_clean":"2024-09-17T22:41:20.920513+0000","last_became_active":"2024-09-17T22:41:19.446028+0000","last_became_peered":"2024-09-17T22:41:19.446028+0000","last_unstale":"2024-09-17T22:41:20.920513+0000","last_undegraded":"2024-09-17T22:41:20.920513+0000","last_fullsized":"2024-09-17T22:41:20.920513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:10:06.404741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"28'4","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.467379+0000","last_change":"2024-09-17T22:41:21.446196+0000","last_active":"2024-09-17T22:41:24.467379+0000","last_peered":"2024-09-17T22:41:24.467379+0000","last_clean":"2024-09-17T22:41:24.467379+0000","last_became_active":"2024-09-17T22:41:19.451662+0000","last_became_peered":"2024-09-17T22:41:19.451662+0000","last_unstale":"2024-09-17T22:41:24.467379+0000","last_undegraded":"2024-09-17T22:41:24.467379+0000","last_fullsized":"2024-09-17T22:41:24.467379+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:30.253758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062146200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":32714875,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":89,"num_read_kb":73,"num_write":46,"num_write_kb":729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":260942,"data_compressed":71726,"data_compressed_allocated":90112,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":693267,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":274432,"data_stored":102864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":262144,"data_stored":96661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":90561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":323584,"data_stored":687037,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":262144,"data_stored":96661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":246966,"data_compressed":53697,"data_compressed_allocated":61440,"data_compressed_original":110592,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":213394,"data_compressed":130,"data_compressed_allocated":8192,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":35,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":134581,"data_compressed":18029,"data_compressed_allocated":28672,"data_compressed_original":86016,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":126318,"data_compressed":53697,"data_compressed_allocated":61440,"data_compressed_original":110592,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:41:25.823 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:41:25.875 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:25.872+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:25.886 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:25.886 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:25.886 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:41:25.887 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:25.896 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:41:25.919 INFO:teuthology.orchestra.run.smithi115.stdout:207 read 0x700e102 thru 0x701bb55 (0xda54 bytes) 2024-09-17T22:41:25.922 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:25.923 INFO:teuthology.orchestra.run.smithi115.stdout:209 read 0x1764e2e thru 0x1766f71 (0x2144 bytes) 2024-09-17T22:41:25.931 INFO:teuthology.orchestra.run.smithi115.stdout:210 trunc from 0xd7d2fad to 0x730c7df 2024-09-17T22:41:25.979 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:25.977+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:26.016 INFO:teuthology.orchestra.run.smithi115.stdout:211 trunc from 0x730c7df to 0x34183fb 2024-09-17T22:41:26.062 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:26.062 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:26.063 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:26.063 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:26.063 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:26.072 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:41:26.076 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:26.074+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:26.177 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:26.175+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:26.186 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:41:26.187 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:41:26.198 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-09-17T22:41:25.465185+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":33305262,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":205,"num_read_kb":173,"num_write":207,"num_write_kb":3147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":170,"ondisk_log_size":170,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218704,"kb_used_data":2444,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756016,"statfs":{"total":773094113280,"available":772870160384,"internally_reserved":0,"allocated":2502656,"data_stored":2227411,"data_compressed":63783,"data_compressed_allocated":143360,"data_compressed_original":1314816,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32714875,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":89,"num_read_kb":73,"num_write":46,"num_write_kb":729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.019023"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.448946+0000","last_change":"2024-09-17T22:41:19.443915+0000","last_active":"2024-09-17T22:41:20.448946+0000","last_peered":"2024-09-17T22:41:20.448946+0000","last_clean":"2024-09-17T22:41:20.448946+0000","last_became_active":"2024-09-17T22:41:19.443658+0000","last_became_peered":"2024-09-17T22:41:19.443658+0000","last_unstale":"2024-09-17T22:41:20.448946+0000","last_undegraded":"2024-09-17T22:41:20.448946+0000","last_fullsized":"2024-09-17T22:41:20.448946+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:24:54.635341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.448994+0000","last_change":"2024-09-17T22:41:19.449604+0000","last_active":"2024-09-17T22:41:20.448994+0000","last_peered":"2024-09-17T22:41:20.448994+0000","last_clean":"2024-09-17T22:41:20.448994+0000","last_became_active":"2024-09-17T22:41:19.449357+0000","last_became_peered":"2024-09-17T22:41:19.449357+0000","last_unstale":"2024-09-17T22:41:20.448994+0000","last_undegraded":"2024-09-17T22:41:20.448994+0000","last_fullsized":"2024-09-17T22:41:20.448994+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:22:57.833991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"28'1","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.730318+0000","last_change":"2024-09-17T22:41:21.446449+0000","last_active":"2024-09-17T22:41:24.730318+0000","last_peered":"2024-09-17T22:41:24.730318+0000","last_clean":"2024-09-17T22:41:24.730318+0000","last_became_active":"2024-09-17T22:41:19.447823+0000","last_became_peered":"2024-09-17T22:41:19.447823+0000","last_unstale":"2024-09-17T22:41:24.730318+0000","last_undegraded":"2024-09-17T22:41:24.730318+0000","last_fullsized":"2024-09-17T22:41:24.730318+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:01:38.780863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000382498,"stat_sum":{"num_bytes":3545871,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":1,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"26'1","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T22:41:21.453779+0000","last_change":"2024-09-17T22:41:21.446501+0000","last_active":"2024-09-17T22:41:21.453779+0000","last_peered":"2024-09-17T22:41:21.453779+0000","last_clean":"2024-09-17T22:41:21.453779+0000","last_became_active":"2024-09-17T22:41:19.452435+0000","last_became_peered":"2024-09-17T22:41:19.452435+0000","last_unstale":"2024-09-17T22:41:21.453779+0000","last_undegraded":"2024-09-17T22:41:21.453779+0000","last_fullsized":"2024-09-17T22:41:21.453779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:48.072503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001451078,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.445389+0000","last_change":"2024-09-17T22:41:19.450099+0000","last_active":"2024-09-17T22:41:20.445389+0000","last_peered":"2024-09-17T22:41:20.445389+0000","last_clean":"2024-09-17T22:41:20.445389+0000","last_became_active":"2024-09-17T22:41:19.449830+0000","last_became_peered":"2024-09-17T22:41:19.449830+0000","last_unstale":"2024-09-17T22:41:20.445389+0000","last_undegraded":"2024-09-17T22:41:20.445389+0000","last_fullsized":"2024-09-17T22:41:20.445389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:13:09.570462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"26'2","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T22:41:21.471424+0000","last_change":"2024-09-17T22:41:21.446945+0000","last_active":"2024-09-17T22:41:21.471424+0000","last_peered":"2024-09-17T22:41:21.471424+0000","last_clean":"2024-09-17T22:41:21.471424+0000","last_became_active":"2024-09-17T22:41:19.446541+0000","last_became_peered":"2024-09-17T22:41:19.446541+0000","last_unstale":"2024-09-17T22:41:21.471424+0000","last_undegraded":"2024-09-17T22:41:21.471424+0000","last_fullsized":"2024-09-17T22:41:21.471424+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:54.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001630631,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.445412+0000","last_change":"2024-09-17T22:41:19.450227+0000","last_active":"2024-09-17T22:41:20.445412+0000","last_peered":"2024-09-17T22:41:20.445412+0000","last_clean":"2024-09-17T22:41:20.445412+0000","last_became_active":"2024-09-17T22:41:19.449940+0000","last_became_peered":"2024-09-17T22:41:19.449940+0000","last_unstale":"2024-09-17T22:41:20.445412+0000","last_undegraded":"2024-09-17T22:41:20.445412+0000","last_fullsized":"2024-09-17T22:41:20.445412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:40:32.886261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.448962+0000","last_change":"2024-09-17T22:41:19.446570+0000","last_active":"2024-09-17T22:41:20.448962+0000","last_peered":"2024-09-17T22:41:20.448962+0000","last_clean":"2024-09-17T22:41:20.448962+0000","last_became_active":"2024-09-17T22:41:19.446287+0000","last_became_peered":"2024-09-17T22:41:19.446287+0000","last_unstale":"2024-09-17T22:41:20.448962+0000","last_undegraded":"2024-09-17T22:41:20.448962+0000","last_fullsized":"2024-09-17T22:41:20.448962+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:14:29.795127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.449064+0000","last_change":"2024-09-17T22:41:19.446207+0000","last_active":"2024-09-17T22:41:20.449064+0000","last_peered":"2024-09-17T22:41:20.449064+0000","last_clean":"2024-09-17T22:41:20.449064+0000","last_became_active":"2024-09-17T22:41:19.445923+0000","last_became_peered":"2024-09-17T22:41:19.445923+0000","last_unstale":"2024-09-17T22:41:20.449064+0000","last_undegraded":"2024-09-17T22:41:20.449064+0000","last_fullsized":"2024-09-17T22:41:20.449064+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:27:21.301851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"28'4","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.489869+0000","last_change":"2024-09-17T22:41:21.516305+0000","last_active":"2024-09-17T22:41:24.489869+0000","last_peered":"2024-09-17T22:41:24.489869+0000","last_clean":"2024-09-17T22:41:24.489869+0000","last_became_active":"2024-09-17T22:41:19.450587+0000","last_became_peered":"2024-09-17T22:41:19.450587+0000","last_unstale":"2024-09-17T22:41:24.489869+0000","last_undegraded":"2024-09-17T22:41:24.489869+0000","last_fullsized":"2024-09-17T22:41:24.489869+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:05:19.351470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000275164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T22:41:21.454760+0000","last_change":"2024-09-17T22:41:21.454760+0000","last_active":"2024-09-17T22:41:21.454760+0000","last_peered":"2024-09-17T22:41:21.454760+0000","last_clean":"2024-09-17T22:41:21.454760+0000","last_became_active":"2024-09-17T22:41:19.449329+0000","last_became_peered":"2024-09-17T22:41:19.449329+0000","last_unstale":"2024-09-17T22:41:21.454760+0000","last_undegraded":"2024-09-17T22:41:21.454760+0000","last_fullsized":"2024-09-17T22:41:21.454760+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:10:05.378151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041055799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.467439+0000","last_change":"2024-09-17T22:41:21.446210+0000","last_active":"2024-09-17T22:41:24.467439+0000","last_peered":"2024-09-17T22:41:24.467439+0000","last_clean":"2024-09-17T22:41:24.467439+0000","last_became_active":"2024-09-17T22:41:19.451631+0000","last_became_peered":"2024-09-17T22:41:19.451631+0000","last_unstale":"2024-09-17T22:41:24.467439+0000","last_undegraded":"2024-09-17T22:41:24.467439+0000","last_fullsized":"2024-09-17T22:41:24.467439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:28:37.033572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081193699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.467710+0000","last_change":"2024-09-17T22:41:11.397814+0000","last_active":"2024-09-17T22:41:24.467710+0000","last_peered":"2024-09-17T22:41:24.467710+0000","last_clean":"2024-09-17T22:41:24.467710+0000","last_became_active":"2024-09-17T22:41:09.506775+0000","last_became_peered":"2024-09-17T22:41:09.506775+0000","last_unstale":"2024-09-17T22:41:24.467710+0000","last_undegraded":"2024-09-17T22:41:24.467710+0000","last_fullsized":"2024-09-17T22:41:24.467710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:47:43.867775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.445455+0000","last_change":"2024-09-17T22:41:19.449258+0000","last_active":"2024-09-17T22:41:20.445455+0000","last_peered":"2024-09-17T22:41:20.445455+0000","last_clean":"2024-09-17T22:41:20.445455+0000","last_became_active":"2024-09-17T22:41:19.448966+0000","last_became_peered":"2024-09-17T22:41:19.448966+0000","last_unstale":"2024-09-17T22:41:20.445455+0000","last_undegraded":"2024-09-17T22:41:20.445455+0000","last_fullsized":"2024-09-17T22:41:20.445455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:51:35.372743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.445313+0000","last_change":"2024-09-17T22:41:11.395783+0000","last_active":"2024-09-17T22:41:20.445313+0000","last_peered":"2024-09-17T22:41:20.445313+0000","last_clean":"2024-09-17T22:41:20.445313+0000","last_became_active":"2024-09-17T22:41:09.507056+0000","last_became_peered":"2024-09-17T22:41:09.507056+0000","last_unstale":"2024-09-17T22:41:20.445313+0000","last_undegraded":"2024-09-17T22:41:20.445313+0000","last_fullsized":"2024-09-17T22:41:20.445313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:13.438783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.445388+0000","last_change":"2024-09-17T22:41:19.451905+0000","last_active":"2024-09-17T22:41:20.445388+0000","last_peered":"2024-09-17T22:41:20.445388+0000","last_clean":"2024-09-17T22:41:20.445388+0000","last_became_active":"2024-09-17T22:41:19.451637+0000","last_became_peered":"2024-09-17T22:41:19.451637+0000","last_unstale":"2024-09-17T22:41:20.445388+0000","last_undegraded":"2024-09-17T22:41:20.445388+0000","last_fullsized":"2024-09-17T22:41:20.445388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:36:48.942068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.468295+0000","last_change":"2024-09-17T22:41:11.397753+0000","last_active":"2024-09-17T22:41:24.468295+0000","last_peered":"2024-09-17T22:41:24.468295+0000","last_clean":"2024-09-17T22:41:24.468295+0000","last_became_active":"2024-09-17T22:41:09.387651+0000","last_became_peered":"2024-09-17T22:41:09.387651+0000","last_unstale":"2024-09-17T22:41:24.468295+0000","last_undegraded":"2024-09-17T22:41:24.468295+0000","last_fullsized":"2024-09-17T22:41:24.468295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:14:30.915047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.920491+0000","last_change":"2024-09-17T22:41:19.445172+0000","last_active":"2024-09-17T22:41:20.920491+0000","last_peered":"2024-09-17T22:41:20.920491+0000","last_clean":"2024-09-17T22:41:20.920491+0000","last_became_active":"2024-09-17T22:41:19.444893+0000","last_became_peered":"2024-09-17T22:41:19.444893+0000","last_unstale":"2024-09-17T22:41:20.920491+0000","last_undegraded":"2024-09-17T22:41:20.920491+0000","last_fullsized":"2024-09-17T22:41:20.920491+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:48.713940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.448848+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:41:20.448848+0000","last_peered":"2024-09-17T22:41:20.448848+0000","last_clean":"2024-09-17T22:41:20.448848+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:41:20.448848+0000","last_undegraded":"2024-09-17T22:41:20.448848+0000","last_fullsized":"2024-09-17T22:41:20.448848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"29'20","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.920676+0000","last_change":"2024-09-17T22:41:21.516349+0000","last_active":"2024-09-17T22:41:24.920676+0000","last_peered":"2024-09-17T22:41:24.920676+0000","last_clean":"2024-09-17T22:41:24.920676+0000","last_became_active":"2024-09-17T22:41:19.451723+0000","last_became_peered":"2024-09-17T22:41:19.451723+0000","last_unstale":"2024-09-17T22:41:24.920676+0000","last_undegraded":"2024-09-17T22:41:24.920676+0000","last_fullsized":"2024-09-17T22:41:24.920676+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:39:06.838900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040033199999999999,"stat_sum":{"num_bytes":15728640,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":20,"num_write_kb":326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T22:41:21.454999+0000","last_change":"2024-09-17T22:41:21.454999+0000","last_active":"2024-09-17T22:41:21.454999+0000","last_peered":"2024-09-17T22:41:21.454999+0000","last_clean":"2024-09-17T22:41:21.454999+0000","last_became_active":"2024-09-17T22:41:19.450477+0000","last_became_peered":"2024-09-17T22:41:19.450477+0000","last_unstale":"2024-09-17T22:41:21.454999+0000","last_undegraded":"2024-09-17T22:41:21.454999+0000","last_fullsized":"2024-09-17T22:41:21.454999+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:15:59.350460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063898899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":37,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.467922+0000","last_change":"2024-09-17T22:41:11.397488+0000","last_active":"2024-09-17T22:41:24.467922+0000","last_peered":"2024-09-17T22:41:24.467922+0000","last_clean":"2024-09-17T22:41:24.467922+0000","last_became_active":"2024-09-17T22:41:09.651390+0000","last_became_peered":"2024-09-17T22:41:09.651390+0000","last_unstale":"2024-09-17T22:41:24.467922+0000","last_undegraded":"2024-09-17T22:41:24.467922+0000","last_fullsized":"2024-09-17T22:41:24.467922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:00.598921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.448817+0000","last_change":"2024-09-17T22:41:19.444560+0000","last_active":"2024-09-17T22:41:20.448817+0000","last_peered":"2024-09-17T22:41:20.448817+0000","last_clean":"2024-09-17T22:41:20.448817+0000","last_became_active":"2024-09-17T22:41:19.444314+0000","last_became_peered":"2024-09-17T22:41:19.444314+0000","last_unstale":"2024-09-17T22:41:20.448817+0000","last_undegraded":"2024-09-17T22:41:20.448817+0000","last_fullsized":"2024-09-17T22:41:20.448817+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:51.835717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.920217+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:41:20.920217+0000","last_peered":"2024-09-17T22:41:20.920217+0000","last_clean":"2024-09-17T22:41:20.920217+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:41:20.920217+0000","last_undegraded":"2024-09-17T22:41:20.920217+0000","last_fullsized":"2024-09-17T22:41:20.920217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.920336+0000","last_change":"2024-09-17T22:41:19.450084+0000","last_active":"2024-09-17T22:41:20.920336+0000","last_peered":"2024-09-17T22:41:20.920336+0000","last_clean":"2024-09-17T22:41:20.920336+0000","last_became_active":"2024-09-17T22:41:19.449791+0000","last_became_peered":"2024-09-17T22:41:19.449791+0000","last_unstale":"2024-09-17T22:41:20.920336+0000","last_undegraded":"2024-09-17T22:41:20.920336+0000","last_fullsized":"2024-09-17T22:41:20.920336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:57:24.865653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.467024+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:41:24.467024+0000","last_peered":"2024-09-17T22:41:24.467024+0000","last_clean":"2024-09-17T22:41:24.467024+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:41:24.467024+0000","last_undegraded":"2024-09-17T22:41:24.467024+0000","last_fullsized":"2024-09-17T22:41:24.467024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.445354+0000","last_change":"2024-09-17T22:41:19.449591+0000","last_active":"2024-09-17T22:41:20.445354+0000","last_peered":"2024-09-17T22:41:20.445354+0000","last_clean":"2024-09-17T22:41:20.445354+0000","last_became_active":"2024-09-17T22:41:19.449279+0000","last_became_peered":"2024-09-17T22:41:19.449279+0000","last_unstale":"2024-09-17T22:41:20.445354+0000","last_undegraded":"2024-09-17T22:41:20.445354+0000","last_fullsized":"2024-09-17T22:41:20.445354+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:02:45.060890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.467511+0000","last_change":"2024-09-17T22:41:11.398108+0000","last_active":"2024-09-17T22:41:24.467511+0000","last_peered":"2024-09-17T22:41:24.467511+0000","last_clean":"2024-09-17T22:41:24.467511+0000","last_became_active":"2024-09-17T22:41:09.388184+0000","last_became_peered":"2024-09-17T22:41:09.388184+0000","last_unstale":"2024-09-17T22:41:24.467511+0000","last_undegraded":"2024-09-17T22:41:24.467511+0000","last_fullsized":"2024-09-17T22:41:24.467511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:43.506329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T22:41:21.444832+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:41:21.444832+0000","last_peered":"2024-09-17T22:41:21.444832+0000","last_clean":"2024-09-17T22:41:21.444832+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:41:21.444832+0000","last_undegraded":"2024-09-17T22:41:21.444832+0000","last_fullsized":"2024-09-17T22:41:21.444832+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.445326+0000","last_change":"2024-09-17T22:41:19.449256+0000","last_active":"2024-09-17T22:41:20.445326+0000","last_peered":"2024-09-17T22:41:20.445326+0000","last_clean":"2024-09-17T22:41:20.445326+0000","last_became_active":"2024-09-17T22:41:19.448956+0000","last_became_peered":"2024-09-17T22:41:19.448956+0000","last_unstale":"2024-09-17T22:41:20.445326+0000","last_undegraded":"2024-09-17T22:41:20.445326+0000","last_fullsized":"2024-09-17T22:41:20.445326+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:30:21.665573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.467319+0000","last_change":"2024-09-17T22:41:21.445421+0000","last_active":"2024-09-17T22:41:24.467319+0000","last_peered":"2024-09-17T22:41:24.467319+0000","last_clean":"2024-09-17T22:41:24.467319+0000","last_became_active":"2024-09-17T22:41:19.449003+0000","last_became_peered":"2024-09-17T22:41:19.449003+0000","last_unstale":"2024-09-17T22:41:24.467319+0000","last_undegraded":"2024-09-17T22:41:24.467319+0000","last_fullsized":"2024-09-17T22:41:24.467319+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:02:37.357737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051465400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"29'16","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.879874+0000","last_change":"2024-09-17T22:41:21.445728+0000","last_active":"2024-09-17T22:41:24.879874+0000","last_peered":"2024-09-17T22:41:24.879874+0000","last_clean":"2024-09-17T22:41:24.879874+0000","last_became_active":"2024-09-17T22:41:19.448011+0000","last_became_peered":"2024-09-17T22:41:19.448011+0000","last_unstale":"2024-09-17T22:41:24.879874+0000","last_undegraded":"2024-09-17T22:41:24.879874+0000","last_fullsized":"2024-09-17T22:41:24.879874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:57:14.142270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068494200000000002,"stat_sum":{"num_bytes":13440345,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":1,"num_write":13,"num_write_kb":199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.920327+0000","last_change":"2024-09-17T22:41:19.447493+0000","last_active":"2024-09-17T22:41:20.920327+0000","last_peered":"2024-09-17T22:41:20.920327+0000","last_clean":"2024-09-17T22:41:20.920327+0000","last_became_active":"2024-09-17T22:41:19.447215+0000","last_became_peered":"2024-09-17T22:41:19.447215+0000","last_unstale":"2024-09-17T22:41:20.920327+0000","last_undegraded":"2024-09-17T22:41:20.920327+0000","last_fullsized":"2024-09-17T22:41:20.920327+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:40.632223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"26'2","reported_seq":29,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T22:41:21.472904+0000","last_change":"2024-09-17T22:41:21.454754+0000","last_active":"2024-09-17T22:41:21.472904+0000","last_peered":"2024-09-17T22:41:21.472904+0000","last_clean":"2024-09-17T22:41:21.472904+0000","last_became_active":"2024-09-17T22:41:19.448928+0000","last_became_peered":"2024-09-17T22:41:19.448928+0000","last_unstale":"2024-09-17T22:41:21.472904+0000","last_undegraded":"2024-09-17T22:41:21.472904+0000","last_fullsized":"2024-09-17T22:41:21.472904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:00:49.892028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000395712,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":25,"num_read_kb":22,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.467854+0000","last_change":"2024-09-17T22:41:21.446562+0000","last_active":"2024-09-17T22:41:24.467854+0000","last_peered":"2024-09-17T22:41:24.467854+0000","last_clean":"2024-09-17T22:41:24.467854+0000","last_became_active":"2024-09-17T22:41:19.447631+0000","last_became_peered":"2024-09-17T22:41:19.447631+0000","last_unstale":"2024-09-17T22:41:24.467854+0000","last_undegraded":"2024-09-17T22:41:24.467854+0000","last_fullsized":"2024-09-17T22:41:24.467854+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:12.336781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000402657,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.490011+0000","last_change":"2024-09-17T22:41:21.516505+0000","last_active":"2024-09-17T22:41:24.490011+0000","last_peered":"2024-09-17T22:41:24.490011+0000","last_clean":"2024-09-17T22:41:24.490011+0000","last_became_active":"2024-09-17T22:41:19.450839+0000","last_became_peered":"2024-09-17T22:41:19.450839+0000","last_unstale":"2024-09-17T22:41:24.490011+0000","last_undegraded":"2024-09-17T22:41:24.490011+0000","last_fullsized":"2024-09-17T22:41:24.490011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:00:58.916145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038538199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T22:41:21.454987+0000","last_change":"2024-09-17T22:41:21.454987+0000","last_active":"2024-09-17T22:41:21.454987+0000","last_peered":"2024-09-17T22:41:21.454987+0000","last_clean":"2024-09-17T22:41:21.454987+0000","last_became_active":"2024-09-17T22:41:19.451864+0000","last_became_peered":"2024-09-17T22:41:19.451864+0000","last_unstale":"2024-09-17T22:41:21.454987+0000","last_undegraded":"2024-09-17T22:41:21.454987+0000","last_fullsized":"2024-09-17T22:41:21.454987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:50:24.181205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000602379,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T22:41:21.446576+0000","last_change":"2024-09-17T22:41:21.446576+0000","last_active":"2024-09-17T22:41:21.446576+0000","last_peered":"2024-09-17T22:41:21.446576+0000","last_clean":"2024-09-17T22:41:21.446576+0000","last_became_active":"2024-09-17T22:41:19.447419+0000","last_became_peered":"2024-09-17T22:41:19.447419+0000","last_unstale":"2024-09-17T22:41:21.446576+0000","last_undegraded":"2024-09-17T22:41:21.446576+0000","last_fullsized":"2024-09-17T22:41:21.446576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:20:24.482993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001865071,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.467196+0000","last_change":"2024-09-17T22:41:21.445226+0000","last_active":"2024-09-17T22:41:24.467196+0000","last_peered":"2024-09-17T22:41:24.467196+0000","last_clean":"2024-09-17T22:41:24.467196+0000","last_became_active":"2024-09-17T22:41:19.454151+0000","last_became_peered":"2024-09-17T22:41:19.454151+0000","last_unstale":"2024-09-17T22:41:24.467196+0000","last_undegraded":"2024-09-17T22:41:24.467196+0000","last_fullsized":"2024-09-17T22:41:24.467196+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:40:15.336777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050545600000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-17T22:41:20.920513+0000","last_change":"2024-09-17T22:41:19.446326+0000","last_active":"2024-09-17T22:41:20.920513+0000","last_peered":"2024-09-17T22:41:20.920513+0000","last_clean":"2024-09-17T22:41:20.920513+0000","last_became_active":"2024-09-17T22:41:19.446028+0000","last_became_peered":"2024-09-17T22:41:19.446028+0000","last_unstale":"2024-09-17T22:41:20.920513+0000","last_undegraded":"2024-09-17T22:41:20.920513+0000","last_fullsized":"2024-09-17T22:41:20.920513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:10:06.404741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"28'4","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-17T22:41:24.467379+0000","last_change":"2024-09-17T22:41:21.446196+0000","last_active":"2024-09-17T22:41:24.467379+0000","last_peered":"2024-09-17T22:41:24.467379+0000","last_clean":"2024-09-17T22:41:24.467379+0000","last_became_active":"2024-09-17T22:41:19.451662+0000","last_became_peered":"2024-09-17T22:41:19.451662+0000","last_unstale":"2024-09-17T22:41:24.467379+0000","last_undegraded":"2024-09-17T22:41:24.467379+0000","last_fullsized":"2024-09-17T22:41:24.467379+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:30.253758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062146200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":32714875,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":89,"num_read_kb":73,"num_write":46,"num_write_kb":729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":260942,"data_compressed":71726,"data_compressed_allocated":90112,"data_compressed_original":196608,"omap_allocated":0,"internal_metadata":0},"log_size":50,"ondisk_log_size":50,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":693267,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":274432,"data_stored":102864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":262144,"data_stored":96661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":90561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":323584,"data_stored":687037,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":262144,"data_stored":96661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":246966,"data_compressed":53697,"data_compressed_allocated":61440,"data_compressed_original":110592,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":213394,"data_compressed":130,"data_compressed_allocated":8192,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":35,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":134581,"data_compressed":18029,"data_compressed_allocated":28672,"data_compressed_original":86016,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":126318,"data_compressed":53697,"data_compressed_allocated":61440,"data_compressed_original":110592,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:41:26.200 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T22:41:26.200 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:41:26.200 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-17T22:41:26.201 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T22:41:26.201 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T22:41:26.201 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-17T22:41:26.219 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:26.219 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:26.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:26.219 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:26.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.219 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:637 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:26.219 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:24.660367+0000", 2024-09-17T22:41:26.220 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.5555382899999999, 2024-09-17T22:41:26.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027568698999999999, 2024-09-17T22:41:26.220 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.220 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.220 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:26.220 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.220 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.220 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.220 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 637 2024-09-17T22:41:26.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.220 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660367+0000", 2024-09-17T22:41:26.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660367+0000", 2024-09-17T22:41:26.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660368+0000", 2024-09-17T22:41:26.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1299999999999999e-07 2024-09-17T22:41:26.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660372+0000", 2024-09-17T22:41:26.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3819999999999997e-06 2024-09-17T22:41:26.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660382+0000", 2024-09-17T22:41:26.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8470000000000008e-06 2024-09-17T22:41:26.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660387+0000", 2024-09-17T22:41:26.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-17T22:41:26.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660429+0000", 2024-09-17T22:41:26.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2194000000000003e-05 2024-09-17T22:41:26.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660518+0000", 2024-09-17T22:41:26.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8572999999999994e-05 2024-09-17T22:41:26.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-17T22:41:26.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.664974+0000", 2024-09-17T22:41:26.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0044561469999999997 2024-09-17T22:41:26.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:26.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.687829+0000", 2024-09-17T22:41:26.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022854695000000001 2024-09-17T22:41:26.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:41:26.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.687850+0000", 2024-09-17T22:41:26.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0933000000000001e-05 2024-09-17T22:41:26.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:26.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.687885+0000", 2024-09-17T22:41:26.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5301e-05 2024-09-17T22:41:26.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.687936+0000", 2024-09-17T22:41:26.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0726999999999997e-05 2024-09-17T22:41:26.225 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.225 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.225 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.226 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:1985 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 15259648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.226 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:25.942429+0000", 2024-09-17T22:41:26.226 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.27347665300000001, 2024-09-17T22:41:26.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011559396, 2024-09-17T22:41:26.226 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.226 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.226 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.226 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.226 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.226 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.226 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1985 2024-09-17T22:41:26.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.226 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942429+0000", 2024-09-17T22:41:26.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942429+0000", 2024-09-17T22:41:26.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942429+0000", 2024-09-17T22:41:26.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8099999999999998e-07 2024-09-17T22:41:26.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942431+0000", 2024-09-17T22:41:26.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0190000000000001e-06 2024-09-17T22:41:26.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942438+0000", 2024-09-17T22:41:26.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7689999999999999e-06 2024-09-17T22:41:26.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942443+0000", 2024-09-17T22:41:26.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3009999999999997e-06 2024-09-17T22:41:26.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.953825+0000", 2024-09-17T22:41:26.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011381364999999999 2024-09-17T22:41:26.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.953908+0000", 2024-09-17T22:41:26.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3750000000000003e-05 2024-09-17T22:41:26.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.953988+0000", 2024-09-17T22:41:26.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9810999999999999e-05 2024-09-17T22:41:26.230 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.230 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.230 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.230 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:1986 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 15278080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.230 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:25.942449+0000", 2024-09-17T22:41:26.230 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.27345609399999998, 2024-09-17T22:41:26.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011725958, 2024-09-17T22:41:26.231 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.231 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.231 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.231 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.231 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.231 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.231 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1986 2024-09-17T22:41:26.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.231 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942449+0000", 2024-09-17T22:41:26.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942449+0000", 2024-09-17T22:41:26.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942450+0000", 2024-09-17T22:41:26.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:41:26.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942452+0000", 2024-09-17T22:41:26.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0600000000000002e-06 2024-09-17T22:41:26.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942458+0000", 2024-09-17T22:41:26.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3949999999999998e-06 2024-09-17T22:41:26.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942463+0000", 2024-09-17T22:41:26.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1479999999999997e-06 2024-09-17T22:41:26.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.954018+0000", 2024-09-17T22:41:26.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011554988 2024-09-17T22:41:26.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.954095+0000", 2024-09-17T22:41:26.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7041999999999999e-05 2024-09-17T22:41:26.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.954175+0000", 2024-09-17T22:41:26.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9943000000000004e-05 2024-09-17T22:41:26.234 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.234 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.234 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.235 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:1987 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 15296512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.235 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:25.942469+0000", 2024-09-17T22:41:26.235 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.27343600200000001, 2024-09-17T22:41:26.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011889469, 2024-09-17T22:41:26.235 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.235 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.235 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.235 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.235 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.235 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.235 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1987 2024-09-17T22:41:26.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.235 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942469+0000", 2024-09-17T22:41:26.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942469+0000", 2024-09-17T22:41:26.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942470+0000", 2024-09-17T22:41:26.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:41:26.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942472+0000", 2024-09-17T22:41:26.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.886e-06 2024-09-17T22:41:26.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942478+0000", 2024-09-17T22:41:26.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3130000000000004e-06 2024-09-17T22:41:26.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942483+0000", 2024-09-17T22:41:26.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9869999999999999e-06 2024-09-17T22:41:26.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.954205+0000", 2024-09-17T22:41:26.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011722141 2024-09-17T22:41:26.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.954289+0000", 2024-09-17T22:41:26.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4309999999999997e-05 2024-09-17T22:41:26.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.954359+0000", 2024-09-17T22:41:26.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9431999999999995e-05 2024-09-17T22:41:26.239 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.239 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.239 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.239 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:1988 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 15314944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.239 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:25.942489+0000", 2024-09-17T22:41:26.239 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.27341621300000002, 2024-09-17T22:41:26.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012033155, 2024-09-17T22:41:26.239 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.239 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.239 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.239 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.240 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.240 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.240 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1988 2024-09-17T22:41:26.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.240 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942489+0000", 2024-09-17T22:41:26.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942489+0000", 2024-09-17T22:41:26.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.942489+0000", 2024-09-17T22:41:26.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T22:41:26.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943078+0000", 2024-09-17T22:41:26.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00058845800000000004 2024-09-17T22:41:26.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943090+0000", 2024-09-17T22:41:26.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1878e-05 2024-09-17T22:41:26.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943097+0000", 2024-09-17T22:41:26.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4089999999999997e-06 2024-09-17T22:41:26.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.954387+0000", 2024-09-17T22:41:26.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011290089 2024-09-17T22:41:26.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.954453+0000", 2024-09-17T22:41:26.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5771000000000004e-05 2024-09-17T22:41:26.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.954522+0000", 2024-09-17T22:41:26.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9179999999999998e-05 2024-09-17T22:41:26.243 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.243 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.243 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.243 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:2102 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 15073280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.243 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:25.943247+0000", 2024-09-17T22:41:26.243 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.27265845900000002, 2024-09-17T22:41:26.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011665718, 2024-09-17T22:41:26.244 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.244 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.244 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.244 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.244 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.244 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.244 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2102 2024-09-17T22:41:26.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.244 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943247+0000", 2024-09-17T22:41:26.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943247+0000", 2024-09-17T22:41:26.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943247+0000", 2024-09-17T22:41:26.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1799999999999995e-07 2024-09-17T22:41:26.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943250+0000", 2024-09-17T22:41:26.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3530000000000002e-06 2024-09-17T22:41:26.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943257+0000", 2024-09-17T22:41:26.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.844e-06 2024-09-17T22:41:26.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943263+0000", 2024-09-17T22:41:26.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0599999999999996e-06 2024-09-17T22:41:26.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.954718+0000", 2024-09-17T22:41:26.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011455523 2024-09-17T22:41:26.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.954785+0000", 2024-09-17T22:41:26.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6978999999999997e-05 2024-09-17T22:41:26.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.954913+0000", 2024-09-17T22:41:26.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000127441 2024-09-17T22:41:26.247 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.247 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.247 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.247 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:2103 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 15091712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.247 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:25.943271+0000", 2024-09-17T22:41:26.248 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.27263474300000001, 2024-09-17T22:41:26.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011812948, 2024-09-17T22:41:26.248 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.248 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.248 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.248 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.248 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.248 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.248 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2103 2024-09-17T22:41:26.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.248 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943271+0000", 2024-09-17T22:41:26.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943271+0000", 2024-09-17T22:41:26.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943271+0000", 2024-09-17T22:41:26.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8999999999999996e-07 2024-09-17T22:41:26.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943274+0000", 2024-09-17T22:41:26.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0800000000000002e-06 2024-09-17T22:41:26.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943281+0000", 2024-09-17T22:41:26.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0210000000000002e-06 2024-09-17T22:41:26.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943286+0000", 2024-09-17T22:41:26.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0170000000000004e-06 2024-09-17T22:41:26.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.954942+0000", 2024-09-17T22:41:26.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011655619000000001 2024-09-17T22:41:26.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.955016+0000", 2024-09-17T22:41:26.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3713000000000004e-05 2024-09-17T22:41:26.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.955084+0000", 2024-09-17T22:41:26.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7908000000000001e-05 2024-09-17T22:41:26.251 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.251 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.251 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.252 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:2104 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 15110144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.252 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:25.943450+0000", 2024-09-17T22:41:26.252 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.27245522599999999, 2024-09-17T22:41:26.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011801580000000001, 2024-09-17T22:41:26.252 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.252 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.252 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.252 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.252 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.252 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.252 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2104 2024-09-17T22:41:26.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.253 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943450+0000", 2024-09-17T22:41:26.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943450+0000", 2024-09-17T22:41:26.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943451+0000", 2024-09-17T22:41:26.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2900000000000003e-07 2024-09-17T22:41:26.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943455+0000", 2024-09-17T22:41:26.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9929999999999997e-06 2024-09-17T22:41:26.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943463+0000", 2024-09-17T22:41:26.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9359999999999999e-06 2024-09-17T22:41:26.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943476+0000", 2024-09-17T22:41:26.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2894000000000001e-05 2024-09-17T22:41:26.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.955113+0000", 2024-09-17T22:41:26.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011637023999999999 2024-09-17T22:41:26.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.955184+0000", 2024-09-17T22:41:26.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1550000000000004e-05 2024-09-17T22:41:26.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.955252+0000", 2024-09-17T22:41:26.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7453999999999995e-05 2024-09-17T22:41:26.256 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.256 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.256 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.256 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:2105 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 15128576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.256 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:25.943487+0000", 2024-09-17T22:41:26.256 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.272418665, 2024-09-17T22:41:26.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011938506, 2024-09-17T22:41:26.256 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.256 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.256 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.256 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.257 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.257 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.257 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2105 2024-09-17T22:41:26.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.257 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943487+0000", 2024-09-17T22:41:26.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943487+0000", 2024-09-17T22:41:26.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943487+0000", 2024-09-17T22:41:26.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2099999999999997e-07 2024-09-17T22:41:26.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943490+0000", 2024-09-17T22:41:26.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5679999999999998e-06 2024-09-17T22:41:26.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943496+0000", 2024-09-17T22:41:26.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8359999999999997e-06 2024-09-17T22:41:26.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943500+0000", 2024-09-17T22:41:26.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5630000000000002e-06 2024-09-17T22:41:26.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.955281+0000", 2024-09-17T22:41:26.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011781211999999999 2024-09-17T22:41:26.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.955355+0000", 2024-09-17T22:41:26.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3701000000000005e-05 2024-09-17T22:41:26.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.955425+0000", 2024-09-17T22:41:26.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0104999999999994e-05 2024-09-17T22:41:26.260 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.260 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.260 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.260 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:2106 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 15147008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.260 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:25.943507+0000", 2024-09-17T22:41:26.261 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.27239863800000003, 2024-09-17T22:41:26.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012085604999999999, 2024-09-17T22:41:26.261 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.261 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.261 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.261 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.261 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.261 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.261 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2106 2024-09-17T22:41:26.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.261 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943507+0000", 2024-09-17T22:41:26.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943507+0000", 2024-09-17T22:41:26.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943507+0000", 2024-09-17T22:41:26.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3999999999999997e-07 2024-09-17T22:41:26.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943509+0000", 2024-09-17T22:41:26.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.99e-06 2024-09-17T22:41:26.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943514+0000", 2024-09-17T22:41:26.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4779999999999999e-06 2024-09-17T22:41:26.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943519+0000", 2024-09-17T22:41:26.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7350000000000004e-06 2024-09-17T22:41:26.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.955455+0000", 2024-09-17T22:41:26.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011935624000000001 2024-09-17T22:41:26.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.955525+0000", 2024-09-17T22:41:26.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0512000000000005e-05 2024-09-17T22:41:26.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.955592+0000", 2024-09-17T22:41:26.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6926000000000003e-05 2024-09-17T22:41:26.265 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.265 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.265 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.265 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:2107 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 15165440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.265 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:25.943526+0000", 2024-09-17T22:41:26.265 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.27237952500000001, 2024-09-17T22:41:26.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012231870000000001, 2024-09-17T22:41:26.265 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.265 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.265 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.265 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.265 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.265 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.266 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2107 2024-09-17T22:41:26.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.266 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943526+0000", 2024-09-17T22:41:26.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943526+0000", 2024-09-17T22:41:26.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943526+0000", 2024-09-17T22:41:26.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T22:41:26.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943528+0000", 2024-09-17T22:41:26.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.03e-06 2024-09-17T22:41:26.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943534+0000", 2024-09-17T22:41:26.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8470000000000001e-06 2024-09-17T22:41:26.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943544+0000", 2024-09-17T22:41:26.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8209999999999997e-06 2024-09-17T22:41:26.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.955620+0000", 2024-09-17T22:41:26.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012076514 2024-09-17T22:41:26.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.955691+0000", 2024-09-17T22:41:26.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0612999999999996e-05 2024-09-17T22:41:26.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.955758+0000", 2024-09-17T22:41:26.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6674999999999994e-05 2024-09-17T22:41:26.269 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.269 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.269 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.269 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:2108 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 15183872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.269 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:25.943550+0000", 2024-09-17T22:41:26.269 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.27235488000000002, 2024-09-17T22:41:26.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012413288999999999, 2024-09-17T22:41:26.269 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.270 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.270 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.270 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.270 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.270 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.270 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2108 2024-09-17T22:41:26.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.270 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943550+0000", 2024-09-17T22:41:26.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943550+0000", 2024-09-17T22:41:26.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943551+0000", 2024-09-17T22:41:26.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0400000000000002e-07 2024-09-17T22:41:26.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943553+0000", 2024-09-17T22:41:26.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.046e-06 2024-09-17T22:41:26.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943559+0000", 2024-09-17T22:41:26.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7189999999999998e-06 2024-09-17T22:41:26.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943563+0000", 2024-09-17T22:41:26.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4739999999999999e-06 2024-09-17T22:41:26.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.955787+0000", 2024-09-17T22:41:26.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012224215 2024-09-17T22:41:26.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.955892+0000", 2024-09-17T22:41:26.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010453799999999999 2024-09-17T22:41:26.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.955964+0000", 2024-09-17T22:41:26.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1892999999999998e-05 2024-09-17T22:41:26.273 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.273 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.273 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.273 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:2109 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 15202304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.273 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:25.943569+0000", 2024-09-17T22:41:26.274 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.27233652899999999, 2024-09-17T22:41:26.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012564287, 2024-09-17T22:41:26.274 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.274 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.274 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.274 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.274 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.274 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.274 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2109 2024-09-17T22:41:26.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.274 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943569+0000", 2024-09-17T22:41:26.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943569+0000", 2024-09-17T22:41:26.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943569+0000", 2024-09-17T22:41:26.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2e-07 2024-09-17T22:41:26.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943571+0000", 2024-09-17T22:41:26.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0140000000000001e-06 2024-09-17T22:41:26.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943577+0000", 2024-09-17T22:41:26.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6439999999999997e-06 2024-09-17T22:41:26.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943581+0000", 2024-09-17T22:41:26.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4449999999999998e-06 2024-09-17T22:41:26.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.955993+0000", 2024-09-17T22:41:26.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012411803000000001 2024-09-17T22:41:26.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.956064+0000", 2024-09-17T22:41:26.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1253000000000004e-05 2024-09-17T22:41:26.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.956133+0000", 2024-09-17T22:41:26.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8707999999999993e-05 2024-09-17T22:41:26.277 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.278 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.278 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.278 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:2110 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 15206400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.278 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:25.943587+0000", 2024-09-17T22:41:26.278 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.27231852899999998, 2024-09-17T22:41:26.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012782599, 2024-09-17T22:41:26.278 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.278 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.278 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.278 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.278 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.278 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.278 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2110 2024-09-17T22:41:26.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.279 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943587+0000", 2024-09-17T22:41:26.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943587+0000", 2024-09-17T22:41:26.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943587+0000", 2024-09-17T22:41:26.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:41:26.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943594+0000", 2024-09-17T22:41:26.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9759999999999998e-06 2024-09-17T22:41:26.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943600+0000", 2024-09-17T22:41:26.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.054e-06 2024-09-17T22:41:26.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943605+0000", 2024-09-17T22:41:26.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.437e-06 2024-09-17T22:41:26.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.956172+0000", 2024-09-17T22:41:26.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012566974 2024-09-17T22:41:26.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.956245+0000", 2024-09-17T22:41:26.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3009000000000005e-05 2024-09-17T22:41:26.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.956369+0000", 2024-09-17T22:41:26.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000124747 2024-09-17T22:41:26.282 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.282 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.282 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.282 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:2111 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 15224832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.282 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:25.943611+0000", 2024-09-17T22:41:26.282 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.27229388799999998, 2024-09-17T22:41:26.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012936797, 2024-09-17T22:41:26.283 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.283 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.283 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.283 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.283 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.283 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.283 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2111 2024-09-17T22:41:26.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.283 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943611+0000", 2024-09-17T22:41:26.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943611+0000", 2024-09-17T22:41:26.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943612+0000", 2024-09-17T22:41:26.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T22:41:26.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943622+0000", 2024-09-17T22:41:26.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8260000000000005e-06 2024-09-17T22:41:26.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943627+0000", 2024-09-17T22:41:26.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5790000000000003e-06 2024-09-17T22:41:26.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943632+0000", 2024-09-17T22:41:26.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6269999999999999e-06 2024-09-17T22:41:26.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.956398+0000", 2024-09-17T22:41:26.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012766494 2024-09-17T22:41:26.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.956473+0000", 2024-09-17T22:41:26.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4820999999999994e-05 2024-09-17T22:41:26.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.956548+0000", 2024-09-17T22:41:26.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4989999999999999e-05 2024-09-17T22:41:26.287 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.287 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.287 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.287 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:2112 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 15243264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.287 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:25.943819+0000", 2024-09-17T22:41:26.287 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.27208636000000003, 2024-09-17T22:41:26.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012919066999999999, 2024-09-17T22:41:26.287 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.287 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.287 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.288 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.288 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.288 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.288 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2112 2024-09-17T22:41:26.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.288 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943819+0000", 2024-09-17T22:41:26.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943819+0000", 2024-09-17T22:41:26.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943820+0000", 2024-09-17T22:41:26.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5000000000000003e-07 2024-09-17T22:41:26.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943822+0000", 2024-09-17T22:41:26.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.909e-06 2024-09-17T22:41:26.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943829+0000", 2024-09-17T22:41:26.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3980000000000004e-06 2024-09-17T22:41:26.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943834+0000", 2024-09-17T22:41:26.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6099999999999997e-06 2024-09-17T22:41:26.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.956578+0000", 2024-09-17T22:41:26.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012743143 2024-09-17T22:41:26.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.291 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.291 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.956662+0000", 2024-09-17T22:41:26.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4902999999999997e-05 2024-09-17T22:41:26.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.291 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.291 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.956738+0000", 2024-09-17T22:41:26.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5554000000000002e-05 2024-09-17T22:41:26.291 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.291 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.291 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.292 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:2113 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 15261696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.292 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:25.943852+0000", 2024-09-17T22:41:26.292 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.27205295200000001, 2024-09-17T22:41:26.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.013073226, 2024-09-17T22:41:26.292 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.292 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.292 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.292 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.292 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.292 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.292 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2113 2024-09-17T22:41:26.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.293 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943852+0000", 2024-09-17T22:41:26.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943852+0000", 2024-09-17T22:41:26.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943853+0000", 2024-09-17T22:41:26.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6100000000000001e-07 2024-09-17T22:41:26.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943855+0000", 2024-09-17T22:41:26.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1789999999999998e-06 2024-09-17T22:41:26.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943861+0000", 2024-09-17T22:41:26.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1650000000000003e-06 2024-09-17T22:41:26.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943872+0000", 2024-09-17T22:41:26.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0936e-05 2024-09-17T22:41:26.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.956769+0000", 2024-09-17T22:41:26.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.012896493 2024-09-17T22:41:26.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.956845+0000", 2024-09-17T22:41:26.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6551000000000006e-05 2024-09-17T22:41:26.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.956926+0000", 2024-09-17T22:41:26.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0440999999999998e-05 2024-09-17T22:41:26.296 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.296 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.296 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.296 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:2114 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 15280128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.296 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:25.943879+0000", 2024-09-17T22:41:26.296 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.27202584299999999, 2024-09-17T22:41:26.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.013221281, 2024-09-17T22:41:26.297 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.297 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.297 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.297 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.297 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.297 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.297 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2114 2024-09-17T22:41:26.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.297 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943879+0000", 2024-09-17T22:41:26.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943879+0000", 2024-09-17T22:41:26.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943880+0000", 2024-09-17T22:41:26.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:41:26.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943882+0000", 2024-09-17T22:41:26.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.979e-06 2024-09-17T22:41:26.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943888+0000", 2024-09-17T22:41:26.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2319999999999998e-06 2024-09-17T22:41:26.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943893+0000", 2024-09-17T22:41:26.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:41:26.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.956952+0000", 2024-09-17T22:41:26.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.01305971 2024-09-17T22:41:26.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.957025+0000", 2024-09-17T22:41:26.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2392999999999996e-05 2024-09-17T22:41:26.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.957101+0000", 2024-09-17T22:41:26.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5947999999999999e-05 2024-09-17T22:41:26.300 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.300 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.300 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.301 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:2115 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 15298560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.301 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:25.943898+0000", 2024-09-17T22:41:26.301 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.272006884, 2024-09-17T22:41:26.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.013380276, 2024-09-17T22:41:26.301 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.301 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.301 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.301 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.301 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.301 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.301 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2115 2024-09-17T22:41:26.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.301 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943898+0000", 2024-09-17T22:41:26.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943898+0000", 2024-09-17T22:41:26.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943899+0000", 2024-09-17T22:41:26.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9299999999999999e-07 2024-09-17T22:41:26.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943901+0000", 2024-09-17T22:41:26.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6670000000000001e-06 2024-09-17T22:41:26.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943906+0000", 2024-09-17T22:41:26.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7259999999999999e-06 2024-09-17T22:41:26.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943911+0000", 2024-09-17T22:41:26.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4710000000000001e-06 2024-09-17T22:41:26.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.957129+0000", 2024-09-17T22:41:26.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.013217975 2024-09-17T22:41:26.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.957203+0000", 2024-09-17T22:41:26.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4036999999999996e-05 2024-09-17T22:41:26.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.957279+0000", 2024-09-17T22:41:26.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6007000000000006e-05 2024-09-17T22:41:26.305 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.305 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.305 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.305 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:2116 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 15316992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.305 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:25.943916+0000", 2024-09-17T22:41:26.305 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.27198894600000001, 2024-09-17T22:41:26.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.013591848, 2024-09-17T22:41:26.305 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.305 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.305 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.306 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.306 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.306 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.306 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 2116 2024-09-17T22:41:26.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.306 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943916+0000", 2024-09-17T22:41:26.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943916+0000", 2024-09-17T22:41:26.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943917+0000", 2024-09-17T22:41:26.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9299999999999999e-07 2024-09-17T22:41:26.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943918+0000", 2024-09-17T22:41:26.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6810000000000001e-06 2024-09-17T22:41:26.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943924+0000", 2024-09-17T22:41:26.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.699e-06 2024-09-17T22:41:26.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.943929+0000", 2024-09-17T22:41:26.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4059999999999998e-06 2024-09-17T22:41:26.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.957325+0000", 2024-09-17T22:41:26.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.013395961 2024-09-17T22:41:26.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.957416+0000", 2024-09-17T22:41:26.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1785000000000003e-05 2024-09-17T22:41:26.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:26.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:25.957508+0000", 2024-09-17T22:41:26.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1922999999999994e-05 2024-09-17T22:41:26.309 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.309 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.309 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.310 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.310 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.310 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:26.311 INFO:teuthology.orchestra.run.smithi115.stdout:213 read 0x22596ac thru 0x226874d (0xf0a2 bytes) 2024-09-17T22:41:26.311 INFO:teuthology.orchestra.run.smithi115.stdout:214 write 0x4e448f6 thru 0x4e461ea (0x18f5 bytes) 2024-09-17T22:41:26.312 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:26.308+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:26.312 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:41:26.313 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:26.313 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T22:41:26.313 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:26.313 INFO:teuthology.orchestra.run.smithi115.stdout:215 punch from 0x35fb04 to 0x366b2b, (0x7027 bytes) 2024-09-17T22:41:26.313 INFO:teuthology.orchestra.run.smithi115.stdout:216 writesame 0x3fa3157 thru 0x3fb245b data_size 0x1(0xf305 bytes) 2024-09-17T22:41:26.327 INFO:teuthology.orchestra.run.smithi115.stdout:218 writesame 0x2037bf5 thru 0x2039305 data_size 0x1(0x1711 bytes) 2024-09-17T22:41:26.355 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:26.361 INFO:teuthology.orchestra.run.smithi115.stdout:225 trunc from 0x4e461eb to 0xa521f71 2024-09-17T22:41:26.367 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:26.367 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:26.373 INFO:teuthology.orchestra.run.smithi115.stdout:228 trunc from 0xa521f71 to 0x500f7c8 2024-09-17T22:41:26.379 INFO:teuthology.orchestra.run.smithi115.stdout:229 read 0x2a17c8a thru 0x2a1a96d (0x2ce4 bytes) 2024-09-17T22:41:26.384 INFO:teuthology.orchestra.run.smithi115.stdout:230 trunc from 0x500f7c8 to 0x8c83e2f 2024-09-17T22:41:26.390 INFO:teuthology.orchestra.run.smithi115.stdout:233 punch from 0x748ba9d to 0x748be65, (0x3c8 bytes) 2024-09-17T22:41:26.396 INFO:teuthology.orchestra.run.smithi115.stdout:234 writesame 0x9bc9d3d thru 0x9bd6db6 data_size 0x2(0xd07a bytes) 2024-09-17T22:41:26.400 INFO:teuthology.orchestra.run.smithi115.stdout:235 trunc from 0x9bd6db7 to 0xd3d11c4 2024-09-17T22:41:26.406 INFO:teuthology.orchestra.run.smithi115.stdout:236 read 0x9782a85 thru 0x978a19e (0x771a bytes) 2024-09-17T22:41:26.406 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:26.406 INFO:teuthology.orchestra.run.smithi115.stdout:239 read 0x3854417 thru 0x38642f8 (0xfee2 bytes) 2024-09-17T22:41:26.406 INFO:teuthology.orchestra.run.smithi115.stdout:241 write 0xd2dcf3 thru 0xd3a6bd (0xc9cb bytes) 2024-09-17T22:41:26.412 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:26.410+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:26.465 INFO:teuthology.orchestra.run.smithi115.stdout:242 punch from 0x5478859 to 0x5485162, (0xc909 bytes) 2024-09-17T22:41:26.465 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:26.466 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:26.466 INFO:teuthology.orchestra.run.smithi115.stdout:246 write 0x28a0f93 thru 0x28b0273 (0xf2e1 bytes) 2024-09-17T22:41:26.487 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:26.487 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:26.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.488 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6105 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 5564416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.488 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.471471+0000", 2024-09-17T22:41:26.488 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.011397539999999999, 2024-09-17T22:41:26.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011427735, 2024-09-17T22:41:26.488 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.488 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.488 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:26.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.488 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6105 2024-09-17T22:41:26.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.489 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471471+0000", 2024-09-17T22:41:26.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471471+0000", 2024-09-17T22:41:26.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471471+0000", 2024-09-17T22:41:26.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7599999999999998e-07 2024-09-17T22:41:26.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471481+0000", 2024-09-17T22:41:26.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0207000000000001e-05 2024-09-17T22:41:26.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471488+0000", 2024-09-17T22:41:26.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0700000000000001e-06 2024-09-17T22:41:26.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471493+0000", 2024-09-17T22:41:26.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1900000000000003e-06 2024-09-17T22:41:26.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:26.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.481097+0000", 2024-09-17T22:41:26.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0096031459999999999 2024-09-17T22:41:26.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:26.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.481170+0000", 2024-09-17T22:41:26.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3187000000000002e-05 2024-09-17T22:41:26.491 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.491 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.492 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.492 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6479 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.492 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.474514+0000", 2024-09-17T22:41:26.492 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0083546120000000008, 2024-09-17T22:41:26.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0084600219999999993, 2024-09-17T22:41:26.492 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.492 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.492 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.492 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.492 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.492 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.493 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6479 2024-09-17T22:41:26.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.493 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474514+0000", 2024-09-17T22:41:26.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474514+0000", 2024-09-17T22:41:26.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474514+0000", 2024-09-17T22:41:26.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6700000000000003e-07 2024-09-17T22:41:26.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474518+0000", 2024-09-17T22:41:26.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1049999999999997e-06 2024-09-17T22:41:26.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474528+0000", 2024-09-17T22:41:26.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9410000000000002e-06 2024-09-17T22:41:26.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474534+0000", 2024-09-17T22:41:26.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4620000000000004e-06 2024-09-17T22:41:26.495 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.495 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.495 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.495 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6480 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.495 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.474543+0000", 2024-09-17T22:41:26.495 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0083251579999999992, 2024-09-17T22:41:26.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0084868760000000008, 2024-09-17T22:41:26.495 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.495 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.495 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.496 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.496 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.496 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.496 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6480 2024-09-17T22:41:26.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.496 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474543+0000", 2024-09-17T22:41:26.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474543+0000", 2024-09-17T22:41:26.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474543+0000", 2024-09-17T22:41:26.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9299999999999999e-07 2024-09-17T22:41:26.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474545+0000", 2024-09-17T22:41:26.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0229999999999999e-06 2024-09-17T22:41:26.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474549+0000", 2024-09-17T22:41:26.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.641e-06 2024-09-17T22:41:26.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474552+0000", 2024-09-17T22:41:26.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1360000000000001e-06 2024-09-17T22:41:26.498 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.498 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.498 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.498 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6481 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.498 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.474556+0000", 2024-09-17T22:41:26.498 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0083120969999999992, 2024-09-17T22:41:26.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0085264550000000005, 2024-09-17T22:41:26.499 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.499 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.499 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.499 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6481 2024-09-17T22:41:26.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.499 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474556+0000", 2024-09-17T22:41:26.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474556+0000", 2024-09-17T22:41:26.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474556+0000", 2024-09-17T22:41:26.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2100000000000001e-07 2024-09-17T22:41:26.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474563+0000", 2024-09-17T22:41:26.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0169999999999999e-06 2024-09-17T22:41:26.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474567+0000", 2024-09-17T22:41:26.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5530000000000001e-06 2024-09-17T22:41:26.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474570+0000", 2024-09-17T22:41:26.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.717e-06 2024-09-17T22:41:26.501 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.501 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.501 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.501 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6482 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.502 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.474576+0000", 2024-09-17T22:41:26.502 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0082925989999999995, 2024-09-17T22:41:26.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.008561206, 2024-09-17T22:41:26.502 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.502 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.502 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.502 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.502 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.502 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.502 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6482 2024-09-17T22:41:26.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.502 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474576+0000", 2024-09-17T22:41:26.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474576+0000", 2024-09-17T22:41:26.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474576+0000", 2024-09-17T22:41:26.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-07 2024-09-17T22:41:26.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474577+0000", 2024-09-17T22:41:26.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3489999999999999e-06 2024-09-17T22:41:26.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474581+0000", 2024-09-17T22:41:26.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4189999999999998e-06 2024-09-17T22:41:26.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474583+0000", 2024-09-17T22:41:26.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.667e-06 2024-09-17T22:41:26.504 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.504 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.504 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.505 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6483 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.505 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.474588+0000", 2024-09-17T22:41:26.505 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0082806200000000007, 2024-09-17T22:41:26.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0086023750000000006, 2024-09-17T22:41:26.505 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.505 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.505 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.505 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.505 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.505 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.505 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6483 2024-09-17T22:41:26.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.506 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474588+0000", 2024-09-17T22:41:26.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474588+0000", 2024-09-17T22:41:26.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474588+0000", 2024-09-17T22:41:26.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3099999999999999e-07 2024-09-17T22:41:26.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474595+0000", 2024-09-17T22:41:26.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3379999999999999e-06 2024-09-17T22:41:26.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474600+0000", 2024-09-17T22:41:26.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1789999999999999e-06 2024-09-17T22:41:26.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474603+0000", 2024-09-17T22:41:26.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6850000000000001e-06 2024-09-17T22:41:26.507 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.508 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.508 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.508 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6484 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.508 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.474633+0000", 2024-09-17T22:41:26.508 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0082350270000000007, 2024-09-17T22:41:26.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0086113830000000002, 2024-09-17T22:41:26.508 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.508 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.508 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.508 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.508 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.508 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.508 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6484 2024-09-17T22:41:26.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.509 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474633+0000", 2024-09-17T22:41:26.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474633+0000", 2024-09-17T22:41:26.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474633+0000", 2024-09-17T22:41:26.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0499999999999999e-07 2024-09-17T22:41:26.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474635+0000", 2024-09-17T22:41:26.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.361e-06 2024-09-17T22:41:26.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474641+0000", 2024-09-17T22:41:26.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5289999999999997e-06 2024-09-17T22:41:26.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474644+0000", 2024-09-17T22:41:26.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8940000000000002e-06 2024-09-17T22:41:26.511 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.511 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.511 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.511 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6485 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.511 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.474648+0000", 2024-09-17T22:41:26.511 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0082202209999999998, 2024-09-17T22:41:26.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0086498519999999995, 2024-09-17T22:41:26.511 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.511 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.512 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.512 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.512 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.512 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.512 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6485 2024-09-17T22:41:26.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.512 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474648+0000", 2024-09-17T22:41:26.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474648+0000", 2024-09-17T22:41:26.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474648+0000", 2024-09-17T22:41:26.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.28e-07 2024-09-17T22:41:26.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474649+0000", 2024-09-17T22:41:26.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.114e-06 2024-09-17T22:41:26.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474652+0000", 2024-09-17T22:41:26.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0800000000000002e-06 2024-09-17T22:41:26.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474655+0000", 2024-09-17T22:41:26.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6110000000000001e-06 2024-09-17T22:41:26.514 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.514 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.514 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.514 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6486 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.515 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.474658+0000", 2024-09-17T22:41:26.515 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0082102779999999997, 2024-09-17T22:41:26.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0086941510000000007, 2024-09-17T22:41:26.515 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.515 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.515 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.515 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.515 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.515 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.515 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6486 2024-09-17T22:41:26.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.515 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474658+0000", 2024-09-17T22:41:26.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474658+0000", 2024-09-17T22:41:26.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474658+0000", 2024-09-17T22:41:26.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0699999999999999e-07 2024-09-17T22:41:26.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474659+0000", 2024-09-17T22:41:26.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1699999999999997e-07 2024-09-17T22:41:26.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474662+0000", 2024-09-17T22:41:26.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9000000000000002e-06 2024-09-17T22:41:26.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474664+0000", 2024-09-17T22:41:26.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4250000000000001e-06 2024-09-17T22:41:26.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.518 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.518 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6487 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.518 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.474667+0000", 2024-09-17T22:41:26.518 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0082007680000000006, 2024-09-17T22:41:26.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0087414280000000007, 2024-09-17T22:41:26.518 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.518 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.518 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.519 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.519 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6487 2024-09-17T22:41:26.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.519 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474667+0000", 2024-09-17T22:41:26.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474667+0000", 2024-09-17T22:41:26.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474668+0000", 2024-09-17T22:41:26.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8900000000000001e-07 2024-09-17T22:41:26.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474669+0000", 2024-09-17T22:41:26.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.299e-06 2024-09-17T22:41:26.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474672+0000", 2024-09-17T22:41:26.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0120000000000001e-06 2024-09-17T22:41:26.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474675+0000", 2024-09-17T22:41:26.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.835e-06 2024-09-17T22:41:26.521 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.521 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.521 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.521 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6488 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.521 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.474679+0000", 2024-09-17T22:41:26.521 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0081893780000000006, 2024-09-17T22:41:26.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0087844080000000005, 2024-09-17T22:41:26.522 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.522 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.522 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.522 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6488 2024-09-17T22:41:26.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.524 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474679+0000", 2024-09-17T22:41:26.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474679+0000", 2024-09-17T22:41:26.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474679+0000", 2024-09-17T22:41:26.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:41:26.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474680+0000", 2024-09-17T22:41:26.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4600000000000003e-07 2024-09-17T22:41:26.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474683+0000", 2024-09-17T22:41:26.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9830000000000001e-06 2024-09-17T22:41:26.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474686+0000", 2024-09-17T22:41:26.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.666e-06 2024-09-17T22:41:26.526 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.526 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.526 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.526 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6489 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.527 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.474689+0000", 2024-09-17T22:41:26.527 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0081795129999999994, 2024-09-17T22:41:26.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0088276420000000001, 2024-09-17T22:41:26.527 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.527 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.527 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.527 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6489 2024-09-17T22:41:26.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.527 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474689+0000", 2024-09-17T22:41:26.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474689+0000", 2024-09-17T22:41:26.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474689+0000", 2024-09-17T22:41:26.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9999999999999999e-07 2024-09-17T22:41:26.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474690+0000", 2024-09-17T22:41:26.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8000000000000004e-07 2024-09-17T22:41:26.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474693+0000", 2024-09-17T22:41:26.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9179999999999998e-06 2024-09-17T22:41:26.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474695+0000", 2024-09-17T22:41:26.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.576e-06 2024-09-17T22:41:26.529 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.529 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.530 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.530 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6490 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.530 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.474698+0000", 2024-09-17T22:41:26.530 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0081700680000000008, 2024-09-17T22:41:26.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0088708099999999998, 2024-09-17T22:41:26.530 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.530 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.530 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.530 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6490 2024-09-17T22:41:26.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.531 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474698+0000", 2024-09-17T22:41:26.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474698+0000", 2024-09-17T22:41:26.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474698+0000", 2024-09-17T22:41:26.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9999999999999999e-07 2024-09-17T22:41:26.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474699+0000", 2024-09-17T22:41:26.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0389999999999999e-06 2024-09-17T22:41:26.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474702+0000", 2024-09-17T22:41:26.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9280000000000002e-06 2024-09-17T22:41:26.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474705+0000", 2024-09-17T22:41:26.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4820000000000001e-06 2024-09-17T22:41:26.533 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.533 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.533 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.533 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6491 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.533 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.474708+0000", 2024-09-17T22:41:26.533 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0081605289999999997, 2024-09-17T22:41:26.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0089154879999999992, 2024-09-17T22:41:26.533 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.534 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.534 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.534 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.534 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.534 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.534 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6491 2024-09-17T22:41:26.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.534 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474708+0000", 2024-09-17T22:41:26.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474708+0000", 2024-09-17T22:41:26.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474708+0000", 2024-09-17T22:41:26.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.99e-07 2024-09-17T22:41:26.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474709+0000", 2024-09-17T22:41:26.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7499999999999999e-07 2024-09-17T22:41:26.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474712+0000", 2024-09-17T22:41:26.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9490000000000001e-06 2024-09-17T22:41:26.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474714+0000", 2024-09-17T22:41:26.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.526e-06 2024-09-17T22:41:26.536 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.536 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.536 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.537 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6492 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.537 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.474717+0000", 2024-09-17T22:41:26.537 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0081510899999999997, 2024-09-17T22:41:26.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0089606270000000005, 2024-09-17T22:41:26.537 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.537 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.537 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.537 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.537 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.537 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.537 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6492 2024-09-17T22:41:26.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.537 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474717+0000", 2024-09-17T22:41:26.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474717+0000", 2024-09-17T22:41:26.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474717+0000", 2024-09-17T22:41:26.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.05e-07 2024-09-17T22:41:26.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474718+0000", 2024-09-17T22:41:26.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5199999999999995e-07 2024-09-17T22:41:26.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474722+0000", 2024-09-17T22:41:26.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.664e-06 2024-09-17T22:41:26.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.474725+0000", 2024-09-17T22:41:26.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7539999999999998e-06 2024-09-17T22:41:26.540 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.540 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.540 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.540 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6235 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.540 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.471859+0000", 2024-09-17T22:41:26.540 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.011009405999999999, 2024-09-17T22:41:26.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011875191, 2024-09-17T22:41:26.540 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.540 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.540 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.540 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.540 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.541 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.541 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6235 2024-09-17T22:41:26.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.541 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471859+0000", 2024-09-17T22:41:26.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471859+0000", 2024-09-17T22:41:26.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471859+0000", 2024-09-17T22:41:26.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4299999999999998e-07 2024-09-17T22:41:26.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471861+0000", 2024-09-17T22:41:26.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1069999999999999e-06 2024-09-17T22:41:26.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471869+0000", 2024-09-17T22:41:26.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7200000000000006e-06 2024-09-17T22:41:26.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471874+0000", 2024-09-17T22:41:26.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5630000000000002e-06 2024-09-17T22:41:26.543 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.543 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.543 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.543 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6236 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.543 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.471880+0000", 2024-09-17T22:41:26.543 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.010988296999999999, 2024-09-17T22:41:26.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.011907651999999999, 2024-09-17T22:41:26.543 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.544 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.544 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.544 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.544 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.544 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.544 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6236 2024-09-17T22:41:26.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.544 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471880+0000", 2024-09-17T22:41:26.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471880+0000", 2024-09-17T22:41:26.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471880+0000", 2024-09-17T22:41:26.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0499999999999999e-07 2024-09-17T22:41:26.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471882+0000", 2024-09-17T22:41:26.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9269999999999999e-06 2024-09-17T22:41:26.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471890+0000", 2024-09-17T22:41:26.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.498e-06 2024-09-17T22:41:26.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471895+0000", 2024-09-17T22:41:26.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8339999999999998e-06 2024-09-17T22:41:26.546 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.546 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.546 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.546 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6351 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.547 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.473557+0000", 2024-09-17T22:41:26.547 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0093115999999999997, 2024-09-17T22:41:26.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.010318011, 2024-09-17T22:41:26.547 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.547 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.547 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.547 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.547 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.547 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.547 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6351 2024-09-17T22:41:26.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.547 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473557+0000", 2024-09-17T22:41:26.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473557+0000", 2024-09-17T22:41:26.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473558+0000", 2024-09-17T22:41:26.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0100000000000001e-06 2024-09-17T22:41:26.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473574+0000", 2024-09-17T22:41:26.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6447999999999999e-05 2024-09-17T22:41:26.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473588+0000", 2024-09-17T22:41:26.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3692999999999999e-05 2024-09-17T22:41:26.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473601+0000", 2024-09-17T22:41:26.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3552e-05 2024-09-17T22:41:26.549 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.550 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.550 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.550 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6352 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.550 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.473619+0000", 2024-09-17T22:41:26.550 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0092495619999999994, 2024-09-17T22:41:26.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.010309932000000001, 2024-09-17T22:41:26.550 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.550 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.550 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.551 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6352 2024-09-17T22:41:26.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.551 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473619+0000", 2024-09-17T22:41:26.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473619+0000", 2024-09-17T22:41:26.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473619+0000", 2024-09-17T22:41:26.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2200000000000004e-07 2024-09-17T22:41:26.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473622+0000", 2024-09-17T22:41:26.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9569999999999998e-06 2024-09-17T22:41:26.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473629+0000", 2024-09-17T22:41:26.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9809999999999997e-06 2024-09-17T22:41:26.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473635+0000", 2024-09-17T22:41:26.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7490000000000004e-06 2024-09-17T22:41:26.553 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.553 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.553 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.553 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6353 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.553 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.473643+0000", 2024-09-17T22:41:26.553 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0092256500000000002, 2024-09-17T22:41:26.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.010343163000000001, 2024-09-17T22:41:26.553 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.554 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.554 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.554 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.554 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.554 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.554 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6353 2024-09-17T22:41:26.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.554 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473643+0000", 2024-09-17T22:41:26.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473643+0000", 2024-09-17T22:41:26.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473643+0000", 2024-09-17T22:41:26.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6000000000000003e-07 2024-09-17T22:41:26.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473645+0000", 2024-09-17T22:41:26.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.108e-06 2024-09-17T22:41:26.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473652+0000", 2024-09-17T22:41:26.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3940000000000001e-06 2024-09-17T22:41:26.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473657+0000", 2024-09-17T22:41:26.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2480000000000004e-06 2024-09-17T22:41:26.556 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.556 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.556 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.556 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6354 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.557 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.473673+0000", 2024-09-17T22:41:26.557 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0091951729999999992, 2024-09-17T22:41:26.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.010366143, 2024-09-17T22:41:26.557 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.557 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.557 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.557 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.557 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.557 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.557 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6354 2024-09-17T22:41:26.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.557 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473673+0000", 2024-09-17T22:41:26.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473673+0000", 2024-09-17T22:41:26.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473674+0000", 2024-09-17T22:41:26.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.82e-07 2024-09-17T22:41:26.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473676+0000", 2024-09-17T22:41:26.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3609999999999999e-06 2024-09-17T22:41:26.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473683+0000", 2024-09-17T22:41:26.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9199999999999998e-06 2024-09-17T22:41:26.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473688+0000", 2024-09-17T22:41:26.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3369999999999999e-06 2024-09-17T22:41:26.560 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.560 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.560 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.560 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6355 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.560 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.473695+0000", 2024-09-17T22:41:26.560 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0091727930000000003, 2024-09-17T22:41:26.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.010400131999999999, 2024-09-17T22:41:26.560 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.560 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.560 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.560 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.561 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.561 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.561 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6355 2024-09-17T22:41:26.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.561 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473695+0000", 2024-09-17T22:41:26.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473695+0000", 2024-09-17T22:41:26.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473696+0000", 2024-09-17T22:41:26.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4999999999999998e-07 2024-09-17T22:41:26.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473704+0000", 2024-09-17T22:41:26.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5189999999999992e-06 2024-09-17T22:41:26.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473712+0000", 2024-09-17T22:41:26.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8609999999999998e-06 2024-09-17T22:41:26.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473717+0000", 2024-09-17T22:41:26.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3009999999999997e-06 2024-09-17T22:41:26.563 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.563 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.563 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.563 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6356 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.563 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.473725+0000", 2024-09-17T22:41:26.564 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0091434540000000005, 2024-09-17T22:41:26.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.010423231, 2024-09-17T22:41:26.564 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.564 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.564 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.564 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.564 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.564 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.564 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6356 2024-09-17T22:41:26.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.564 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473725+0000", 2024-09-17T22:41:26.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473725+0000", 2024-09-17T22:41:26.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473725+0000", 2024-09-17T22:41:26.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:41:26.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473727+0000", 2024-09-17T22:41:26.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0600000000000002e-06 2024-09-17T22:41:26.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473733+0000", 2024-09-17T22:41:26.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8470000000000001e-06 2024-09-17T22:41:26.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473738+0000", 2024-09-17T22:41:26.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8400000000000002e-06 2024-09-17T22:41:26.566 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.566 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.566 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.567 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6357 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.567 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.473744+0000", 2024-09-17T22:41:26.567 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0091245019999999996, 2024-09-17T22:41:26.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.010455605, 2024-09-17T22:41:26.567 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.567 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.567 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.567 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.567 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.567 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.567 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6357 2024-09-17T22:41:26.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.567 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473744+0000", 2024-09-17T22:41:26.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473744+0000", 2024-09-17T22:41:26.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473744+0000", 2024-09-17T22:41:26.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.27e-07 2024-09-17T22:41:26.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473746+0000", 2024-09-17T22:41:26.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0360000000000001e-06 2024-09-17T22:41:26.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473752+0000", 2024-09-17T22:41:26.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7359999999999998e-06 2024-09-17T22:41:26.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473757+0000", 2024-09-17T22:41:26.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3319999999999999e-06 2024-09-17T22:41:26.569 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.569 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.570 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.570 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6358 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.570 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.473763+0000", 2024-09-17T22:41:26.570 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0091051559999999997, 2024-09-17T22:41:26.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.010490279, 2024-09-17T22:41:26.570 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.570 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.570 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.570 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.570 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.570 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.570 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6358 2024-09-17T22:41:26.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.570 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473763+0000", 2024-09-17T22:41:26.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473763+0000", 2024-09-17T22:41:26.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473763+0000", 2024-09-17T22:41:26.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4499999999999998e-07 2024-09-17T22:41:26.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473765+0000", 2024-09-17T22:41:26.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.885e-06 2024-09-17T22:41:26.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473771+0000", 2024-09-17T22:41:26.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7259999999999999e-06 2024-09-17T22:41:26.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473776+0000", 2024-09-17T22:41:26.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0950000000000003e-06 2024-09-17T22:41:26.573 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.573 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.573 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.573 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6359 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.573 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.473782+0000", 2024-09-17T22:41:26.573 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0090866300000000001, 2024-09-17T22:41:26.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.010526326000000001, 2024-09-17T22:41:26.573 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.573 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.573 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.573 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.573 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.573 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.573 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6359 2024-09-17T22:41:26.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.574 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473782+0000", 2024-09-17T22:41:26.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473782+0000", 2024-09-17T22:41:26.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473782+0000", 2024-09-17T22:41:26.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:41:26.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473784+0000", 2024-09-17T22:41:26.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7179999999999999e-06 2024-09-17T22:41:26.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473789+0000", 2024-09-17T22:41:26.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.648e-06 2024-09-17T22:41:26.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473794+0000", 2024-09-17T22:41:26.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7500000000000003e-06 2024-09-17T22:41:26.576 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.576 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.576 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.576 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6360 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.576 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.473800+0000", 2024-09-17T22:41:26.576 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0090685460000000002, 2024-09-17T22:41:26.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.010563886, 2024-09-17T22:41:26.576 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.576 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.576 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.576 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.577 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.577 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.577 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6360 2024-09-17T22:41:26.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.577 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473800+0000", 2024-09-17T22:41:26.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473800+0000", 2024-09-17T22:41:26.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473800+0000", 2024-09-17T22:41:26.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.53e-07 2024-09-17T22:41:26.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473802+0000", 2024-09-17T22:41:26.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8589999999999999e-06 2024-09-17T22:41:26.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473808+0000", 2024-09-17T22:41:26.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8889999999999999e-06 2024-09-17T22:41:26.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473813+0000", 2024-09-17T22:41:26.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8350000000000003e-06 2024-09-17T22:41:26.579 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.579 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.579 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.579 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6361 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.579 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.473818+0000", 2024-09-17T22:41:26.579 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0090498920000000004, 2024-09-17T22:41:26.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.010598491, 2024-09-17T22:41:26.579 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.579 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.580 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.580 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.580 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.580 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.580 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6361 2024-09-17T22:41:26.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.580 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473818+0000", 2024-09-17T22:41:26.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473818+0000", 2024-09-17T22:41:26.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473819+0000", 2024-09-17T22:41:26.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2500000000000001e-07 2024-09-17T22:41:26.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473820+0000", 2024-09-17T22:41:26.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7799999999999999e-06 2024-09-17T22:41:26.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473831+0000", 2024-09-17T22:41:26.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0939e-05 2024-09-17T22:41:26.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473837+0000", 2024-09-17T22:41:26.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3709999999999999e-06 2024-09-17T22:41:26.582 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.582 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.582 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.582 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6362 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.583 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.473850+0000", 2024-09-17T22:41:26.583 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0090182720000000008, 2024-09-17T22:41:26.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.010621945000000001, 2024-09-17T22:41:26.583 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.583 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.583 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.583 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6362 2024-09-17T22:41:26.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.583 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473850+0000", 2024-09-17T22:41:26.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473850+0000", 2024-09-17T22:41:26.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473850+0000", 2024-09-17T22:41:26.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:41:26.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473858+0000", 2024-09-17T22:41:26.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6629999999999993e-06 2024-09-17T22:41:26.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473866+0000", 2024-09-17T22:41:26.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8609999999999998e-06 2024-09-17T22:41:26.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473871+0000", 2024-09-17T22:41:26.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.13e-06 2024-09-17T22:41:26.585 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.585 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.585 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.586 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6363 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.586 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.473878+0000", 2024-09-17T22:41:26.586 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0089902590000000004, 2024-09-17T22:41:26.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.010646944, 2024-09-17T22:41:26.586 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.586 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.586 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.586 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.586 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.586 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.586 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6363 2024-09-17T22:41:26.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.586 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473878+0000", 2024-09-17T22:41:26.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473878+0000", 2024-09-17T22:41:26.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473878+0000", 2024-09-17T22:41:26.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6100000000000001e-07 2024-09-17T22:41:26.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473880+0000", 2024-09-17T22:41:26.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1239999999999999e-06 2024-09-17T22:41:26.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473886+0000", 2024-09-17T22:41:26.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:41:26.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473891+0000", 2024-09-17T22:41:26.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6619999999999996e-06 2024-09-17T22:41:26.588 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.588 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.589 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.589 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6364 3.1e 3.46afbbde (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:26.589 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.473897+0000", 2024-09-17T22:41:26.589 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0089710069999999996, 2024-09-17T22:41:26.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.010683104000000001, 2024-09-17T22:41:26.589 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:26.589 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:26.589 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:26.589 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:26.589 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:26.589 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:26.589 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6364 2024-09-17T22:41:26.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.589 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:26.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:26.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473897+0000", 2024-09-17T22:41:26.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:26.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473897+0000", 2024-09-17T22:41:26.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:26.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:26.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473898+0000", 2024-09-17T22:41:26.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.08e-07 2024-09-17T22:41:26.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:26.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473900+0000", 2024-09-17T22:41:26.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-17T22:41:26.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:26.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473906+0000", 2024-09-17T22:41:26.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1850000000000001e-06 2024-09-17T22:41:26.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:26.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:26.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:26.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.473911+0000", 2024-09-17T22:41:26.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8110000000000002e-06 2024-09-17T22:41:26.592 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.592 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:26.592 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.592 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:26.592 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T22:41:26.592 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 31 2024-09-17T22:41:26.592 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:26.592 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:41:26.593 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:41:26.595 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:41:26.596 INFO:teuthology.orchestra.run.smithi115.stdout:247 read 0xc9dd0eb thru 0xc9e8271 (0xb187 bytes) 2024-09-17T22:41:26.596 INFO:teuthology.orchestra.run.smithi115.stdout:248 write 0x4cd0b84 thru 0x4cd10ab (0x528 bytes) 2024-09-17T22:41:26.596 INFO:teuthology.orchestra.run.smithi115.stdout:249 write 0xc55dbb6 thru 0xc55e958 (0xda3 bytes) 2024-09-17T22:41:26.596 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:26.596 INFO:teuthology.orchestra.run.smithi115.stdout:251 read 0x5aa43cc thru 0x5aa99b1 (0x55e6 bytes) 2024-09-17T22:41:26.596 INFO:teuthology.orchestra.run.smithi115.stdout:252 punch from 0x5f43c6c to 0x5f4a984, (0x6d18 bytes) 2024-09-17T22:41:26.596 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:26.596 INFO:teuthology.orchestra.run.smithi115.stdout:255 read 0x8880802 thru 0x8889805 (0x9004 bytes) 2024-09-17T22:41:26.596 INFO:teuthology.orchestra.run.smithi115.stdout:258 writesame 0x66c1c8c thru 0x66c2401 data_size 0x5(0x776 bytes) 2024-09-17T22:41:26.597 INFO:teuthology.orchestra.run.smithi115.stdout:259 write 0x9d6e3a1 thru 0x9d7867d (0xa2dd bytes) 2024-09-17T22:41:26.597 INFO:teuthology.orchestra.run.smithi115.stdout:260 trunc from 0xd3d11c4 to 0x4d06ceb 2024-09-17T22:41:26.597 INFO:teuthology.orchestra.run.smithi115.stdout:261 punch from 0x4204d07 to 0x420dacf, (0x8dc8 bytes) 2024-09-17T22:41:26.597 INFO:teuthology.orchestra.run.smithi115.stdout:262 read 0x141544f thru 0x141fb51 (0xa703 bytes) 2024-09-17T22:41:26.597 INFO:teuthology.orchestra.run.smithi115.stdout:263 read 0x1bf7b20 thru 0x1bfd26a (0x574b bytes) 2024-09-17T22:41:26.597 INFO:teuthology.orchestra.run.smithi115.stdout:264 punch from 0x21efb53 to 0x21f5725, (0x5bd2 bytes) 2024-09-17T22:41:26.597 INFO:teuthology.orchestra.run.smithi115.stdout:266 write 0xb1f6002 thru 0xb1fb0f3 (0x50f2 bytes) 2024-09-17T22:41:26.597 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:26.591+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:26.607 INFO:teuthology.orchestra.run.smithi115.stdout:267 read 0x421eb9e thru 0x421f8a5 (0xd08 bytes) 2024-09-17T22:41:26.608 INFO:teuthology.orchestra.run.smithi115.stdout:270 read 0x317bcd thru 0x318a78 (0xeac bytes) 2024-09-17T22:41:26.608 INFO:teuthology.orchestra.run.smithi115.stdout:272 read 0x2bb9edd thru 0x2bc1557 (0x767b bytes) 2024-09-17T22:41:26.610 INFO:teuthology.orchestra.run.smithi115.stdout:273 punch from 0x863b348 to 0x863ec2c, (0x38e4 bytes) 2024-09-17T22:41:26.611 INFO:teuthology.orchestra.run.smithi115.stdout:274 write 0x6a56f15 thru 0x6a661df (0xf2cb bytes) 2024-09-17T22:41:26.626 INFO:teuthology.orchestra.run.smithi115.stdout:275 writesame 0x97ab802 thru 0x97ba46f data_size 0x35(0xec6e bytes) 2024-09-17T22:41:26.638 INFO:teuthology.orchestra.run.smithi115.stdout:276 clone 3 order 21 su 16384 sc 4 2024-09-17T22:41:26.694 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:26.692+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:26.725 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0: {} 2024-09-17T22:41:26.725 INFO:teuthology.orchestra.run.smithi115.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:41:26.738 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:26.738 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:26.738 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:26.738 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:26.738 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:26.742 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:41:26.742 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:41:26.748 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:41:26.758 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:41:26.759 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:41:26.775 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:41:26.775 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:41:26.791 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:41:26.791 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:41:26.794 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:26.792+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:26.808 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:41:26.808 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:41:26.824 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:41:26.824 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:41:26.840 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:41:26.840 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:41:26.892 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:26.892 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:26.895 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:26.892+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:26.901 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:41:26.995 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:26.993+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:27.004 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:27.004 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:27.012 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:41:27.097 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:27.094+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:27.115 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:27.116 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:27.125 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:41:27.198 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:27.195+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:27.228 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:27.229 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:27.237 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:41:27.299 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:27.296+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:27.341 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:27.341 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:27.349 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:41:27.400 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:27.398+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:27.455 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:27.455 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:27.464 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:41:27.487 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is unset 2024-09-17T22:41:27.504 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:27.502+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:27.541 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone2 from 0xb1fb0f4 (overlap 0x34183fb) to 0x0 2024-09-17T22:41:27.601 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:27.599+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:27.643 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:27.643 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:27.655 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:41:27.702 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:27.699+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:27.803 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:27.800+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:27.817 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:27.817 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:27.831 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:41:27.903 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:27.901+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:28.004 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:28.001+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:28.032 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:28.033 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:28.045 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:41:28.129 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:28.127+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:28.205 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:28.202+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:28.244 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:28.244 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:28.256 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:41:28.339 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:28.337+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:28.407 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:28.404+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:28.451 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:28.451 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:28.464 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:41:28.507 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:28.505+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:28.608 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:28.605+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:28.657 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:28.657 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:28.668 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:41:28.709 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:28.706+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:28.809 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:28.807+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:28.845 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:28.845 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:28.852 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:41:28.854 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:41:28.937 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:28.935+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:29.011 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:29.008+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:29.017 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:29.017 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:29.028 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:41:29.032 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0: {} 2024-09-17T22:41:29.033 INFO:teuthology.orchestra.run.smithi115.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:41:29.052 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-17T22:41:29.055 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:41:29.073 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:41:29.073 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:41:29.090 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:41:29.090 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:41:29.111 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:29.109+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:29.112 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:41:29.113 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:41:29.140 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:41:29.141 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:41:29.149 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:29.149 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:29.149 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:41:29.150 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:41:29.150 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:29.158 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:41:29.158 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:41:29.159 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:41:29.175 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:41:29.175 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:41:29.228 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:29.226+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:29.284 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:29.284 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:41:29.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:41:29.285 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:41:29.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.285 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5353 3.5 3:a0207ec9:::rbd_data.115f1ca54530.0000000000000002:4 [read 1724416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.285 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.418874+0000", 2024-09-17T22:41:29.285 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8622679409999998, 2024-09-17T22:41:29.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039393982000000001, 2024-09-17T22:41:29.285 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.285 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.285 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.285 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.285 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.285 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.285 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5353 2024-09-17T22:41:29.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.286 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418874+0000", 2024-09-17T22:41:29.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418874+0000", 2024-09-17T22:41:29.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418875+0000", 2024-09-17T22:41:29.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2599999999999998e-07 2024-09-17T22:41:29.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418876+0000", 2024-09-17T22:41:29.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.844e-06 2024-09-17T22:41:29.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418887+0000", 2024-09-17T22:41:29.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0898e-05 2024-09-17T22:41:29.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418892+0000", 2024-09-17T22:41:29.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6410000000000001e-06 2024-09-17T22:41:29.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.458173+0000", 2024-09-17T22:41:29.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039280590999999997 2024-09-17T22:41:29.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.458213+0000", 2024-09-17T22:41:29.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0642999999999997e-05 2024-09-17T22:41:29.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.458268+0000", 2024-09-17T22:41:29.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5038999999999999e-05 2024-09-17T22:41:29.289 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.289 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.289 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.289 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5354 3.5 3:a0207ec9:::rbd_data.115f1ca54530.0000000000000002:4 [read 1742848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.289 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.418899+0000", 2024-09-17T22:41:29.289 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8622432729999998, 2024-09-17T22:41:29.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039493847999999998, 2024-09-17T22:41:29.289 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.290 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.290 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.290 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.290 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.290 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.290 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5354 2024-09-17T22:41:29.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.290 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418899+0000", 2024-09-17T22:41:29.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418899+0000", 2024-09-17T22:41:29.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418899+0000", 2024-09-17T22:41:29.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-09-17T22:41:29.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418901+0000", 2024-09-17T22:41:29.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-09-17T22:41:29.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418907+0000", 2024-09-17T22:41:29.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5450000000000003e-06 2024-09-17T22:41:29.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418911+0000", 2024-09-17T22:41:29.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3669999999999999e-06 2024-09-17T22:41:29.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.458296+0000", 2024-09-17T22:41:29.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039384717999999999 2024-09-17T22:41:29.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.458337+0000", 2024-09-17T22:41:29.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0734e-05 2024-09-17T22:41:29.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.458393+0000", 2024-09-17T22:41:29.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6076999999999999e-05 2024-09-17T22:41:29.293 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.293 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.293 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.294 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5355 3.5 3:a0207ec9:::rbd_data.115f1ca54530.0000000000000002:4 [read 1761280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.294 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.418926+0000", 2024-09-17T22:41:29.294 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8622157330000002, 2024-09-17T22:41:29.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039599407000000003, 2024-09-17T22:41:29.294 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.294 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.294 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.294 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.294 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.294 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.294 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5355 2024-09-17T22:41:29.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.294 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418926+0000", 2024-09-17T22:41:29.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418926+0000", 2024-09-17T22:41:29.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418927+0000", 2024-09-17T22:41:29.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-09-17T22:41:29.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418929+0000", 2024-09-17T22:41:29.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.097e-06 2024-09-17T22:41:29.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418935+0000", 2024-09-17T22:41:29.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1419999999999998e-06 2024-09-17T22:41:29.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418941+0000", 2024-09-17T22:41:29.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6659999999999996e-06 2024-09-17T22:41:29.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.458429+0000", 2024-09-17T22:41:29.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039487860999999999 2024-09-17T22:41:29.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.458472+0000", 2024-09-17T22:41:29.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3494000000000001e-05 2024-09-17T22:41:29.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.458526+0000", 2024-09-17T22:41:29.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3757e-05 2024-09-17T22:41:29.297 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.297 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.298 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.298 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5356 3.5 3:a0207ec9:::rbd_data.115f1ca54530.0000000000000002:4 [read 1779712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.298 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.418947+0000", 2024-09-17T22:41:29.298 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8621946870000001, 2024-09-17T22:41:29.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039774706, 2024-09-17T22:41:29.298 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.298 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.298 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.298 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.298 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.298 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.298 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5356 2024-09-17T22:41:29.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.299 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418947+0000", 2024-09-17T22:41:29.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418947+0000", 2024-09-17T22:41:29.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418948+0000", 2024-09-17T22:41:29.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-07 2024-09-17T22:41:29.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418953+0000", 2024-09-17T22:41:29.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0900000000000004e-06 2024-09-17T22:41:29.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418958+0000", 2024-09-17T22:41:29.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4709999999999998e-06 2024-09-17T22:41:29.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418963+0000", 2024-09-17T22:41:29.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6999999999999999e-06 2024-09-17T22:41:29.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.458557+0000", 2024-09-17T22:41:29.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039593916 2024-09-17T22:41:29.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.458629+0000", 2024-09-17T22:41:29.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1673000000000003e-05 2024-09-17T22:41:29.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.458722+0000", 2024-09-17T22:41:29.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3494999999999998e-05 2024-09-17T22:41:29.302 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.302 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.302 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.302 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5357 3.5 3:a0207ec9:::rbd_data.115f1ca54530.0000000000000002:4 [read 1798144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.302 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.418969+0000", 2024-09-17T22:41:29.302 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8621728549999998, 2024-09-17T22:41:29.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039881914999999997, 2024-09-17T22:41:29.302 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.302 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.303 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.303 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.303 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.303 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.303 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5357 2024-09-17T22:41:29.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.303 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418969+0000", 2024-09-17T22:41:29.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418969+0000", 2024-09-17T22:41:29.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418970+0000", 2024-09-17T22:41:29.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:41:29.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418972+0000", 2024-09-17T22:41:29.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8819999999999999e-06 2024-09-17T22:41:29.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418978+0000", 2024-09-17T22:41:29.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4389999999999997e-06 2024-09-17T22:41:29.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418983+0000", 2024-09-17T22:41:29.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5680000000000001e-06 2024-09-17T22:41:29.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.458753+0000", 2024-09-17T22:41:29.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039770749000000001 2024-09-17T22:41:29.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.458797+0000", 2024-09-17T22:41:29.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3782999999999997e-05 2024-09-17T22:41:29.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.458851+0000", 2024-09-17T22:41:29.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4114999999999999e-05 2024-09-17T22:41:29.306 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.306 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.306 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.307 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5358 3.5 3:a0207ec9:::rbd_data.115f1ca54530.0000000000000002:4 [read 1816576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.307 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.418988+0000", 2024-09-17T22:41:29.307 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8621537249999998, 2024-09-17T22:41:29.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040020772000000003, 2024-09-17T22:41:29.307 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.307 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.307 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.307 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5358 2024-09-17T22:41:29.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.307 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418988+0000", 2024-09-17T22:41:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418988+0000", 2024-09-17T22:41:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418989+0000", 2024-09-17T22:41:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4799999999999999e-07 2024-09-17T22:41:29.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418991+0000", 2024-09-17T22:41:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.804e-06 2024-09-17T22:41:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.418996+0000", 2024-09-17T22:41:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8159999999999999e-06 2024-09-17T22:41:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419001+0000", 2024-09-17T22:41:29.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.977e-06 2024-09-17T22:41:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.458889+0000", 2024-09-17T22:41:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039887316999999999 2024-09-17T22:41:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.458939+0000", 2024-09-17T22:41:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0173000000000002e-05 2024-09-17T22:41:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459009+0000", 2024-09-17T22:41:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0337000000000001e-05 2024-09-17T22:41:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5423 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:4 [read 1447936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.419007+0000", 2024-09-17T22:41:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8621350560000001, 2024-09-17T22:41:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040130986, 2024-09-17T22:41:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5423 2024-09-17T22:41:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419007+0000", 2024-09-17T22:41:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419007+0000", 2024-09-17T22:41:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419008+0000", 2024-09-17T22:41:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.77e-07 2024-09-17T22:41:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419009+0000", 2024-09-17T22:41:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7439999999999999e-06 2024-09-17T22:41:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419015+0000", 2024-09-17T22:41:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6400000000000002e-06 2024-09-17T22:41:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419019+0000", 2024-09-17T22:41:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3189999999999996e-06 2024-09-17T22:41:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459036+0000", 2024-09-17T22:41:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040016446999999997 2024-09-17T22:41:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459082+0000", 2024-09-17T22:41:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6230000000000003e-05 2024-09-17T22:41:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459138+0000", 2024-09-17T22:41:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6229e-05 2024-09-17T22:41:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5424 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:4 [read 1466368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.419025+0000", 2024-09-17T22:41:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8621173090000003, 2024-09-17T22:41:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040239401000000001, 2024-09-17T22:41:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5424 2024-09-17T22:41:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419025+0000", 2024-09-17T22:41:29.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419025+0000", 2024-09-17T22:41:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419025+0000", 2024-09-17T22:41:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.65e-07 2024-09-17T22:41:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419027+0000", 2024-09-17T22:41:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5910000000000001e-06 2024-09-17T22:41:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419033+0000", 2024-09-17T22:41:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7189999999999998e-06 2024-09-17T22:41:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419037+0000", 2024-09-17T22:41:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5179999999999998e-06 2024-09-17T22:41:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459166+0000", 2024-09-17T22:41:29.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040129066999999997 2024-09-17T22:41:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459209+0000", 2024-09-17T22:41:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3309999999999997e-05 2024-09-17T22:41:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459264+0000", 2024-09-17T22:41:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4830999999999997e-05 2024-09-17T22:41:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5425 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:4 [read 1484800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.419043+0000", 2024-09-17T22:41:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8620996349999999, 2024-09-17T22:41:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040348242999999999, 2024-09-17T22:41:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5425 2024-09-17T22:41:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419043+0000", 2024-09-17T22:41:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419043+0000", 2024-09-17T22:41:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419049+0000", 2024-09-17T22:41:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7209999999999997e-06 2024-09-17T22:41:29.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419051+0000", 2024-09-17T22:41:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.029e-06 2024-09-17T22:41:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419058+0000", 2024-09-17T22:41:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2369999999999997e-06 2024-09-17T22:41:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419062+0000", 2024-09-17T22:41:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6739999999999996e-06 2024-09-17T22:41:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459291+0000", 2024-09-17T22:41:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040229283999999997 2024-09-17T22:41:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459334+0000", 2024-09-17T22:41:29.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2262000000000003e-05 2024-09-17T22:41:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459391+0000", 2024-09-17T22:41:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7036000000000001e-05 2024-09-17T22:41:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5426 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:4 [read 1503232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.324 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.419069+0000", 2024-09-17T22:41:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8620732970000002, 2024-09-17T22:41:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040444593000000001, 2024-09-17T22:41:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5426 2024-09-17T22:41:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419069+0000", 2024-09-17T22:41:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419069+0000", 2024-09-17T22:41:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419069+0000", 2024-09-17T22:41:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:41:29.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419072+0000", 2024-09-17T22:41:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8909999999999999e-06 2024-09-17T22:41:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419078+0000", 2024-09-17T22:41:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8919999999999997e-06 2024-09-17T22:41:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419084+0000", 2024-09-17T22:41:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5949999999999998e-06 2024-09-17T22:41:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459417+0000", 2024-09-17T22:41:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040333487000000001 2024-09-17T22:41:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459462+0000", 2024-09-17T22:41:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4475999999999999e-05 2024-09-17T22:41:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459513+0000", 2024-09-17T22:41:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1851999999999998e-05 2024-09-17T22:41:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5427 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:4 [read 1521664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.419090+0000", 2024-09-17T22:41:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8620519079999998, 2024-09-17T22:41:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040545272, 2024-09-17T22:41:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.329 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5427 2024-09-17T22:41:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419090+0000", 2024-09-17T22:41:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419090+0000", 2024-09-17T22:41:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419091+0000", 2024-09-17T22:41:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:41:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419093+0000", 2024-09-17T22:41:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2129999999999998e-06 2024-09-17T22:41:29.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419098+0000", 2024-09-17T22:41:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5269999999999998e-06 2024-09-17T22:41:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419103+0000", 2024-09-17T22:41:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.532e-06 2024-09-17T22:41:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459538+0000", 2024-09-17T22:41:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040435295000000003 2024-09-17T22:41:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459580+0000", 2024-09-17T22:41:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1424999999999997e-05 2024-09-17T22:41:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459636+0000", 2024-09-17T22:41:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5881000000000003e-05 2024-09-17T22:41:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.333 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5428 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:4 [read 1540096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.419110+0000", 2024-09-17T22:41:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8620326199999999, 2024-09-17T22:41:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040636855999999999, 2024-09-17T22:41:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5428 2024-09-17T22:41:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419110+0000", 2024-09-17T22:41:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419110+0000", 2024-09-17T22:41:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419110+0000", 2024-09-17T22:41:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4400000000000001e-07 2024-09-17T22:41:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419112+0000", 2024-09-17T22:41:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.748e-06 2024-09-17T22:41:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419117+0000", 2024-09-17T22:41:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7300000000000002e-06 2024-09-17T22:41:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419122+0000", 2024-09-17T22:41:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4469999999999999e-06 2024-09-17T22:41:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459661+0000", 2024-09-17T22:41:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040539487999999999 2024-09-17T22:41:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459696+0000", 2024-09-17T22:41:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4545999999999998e-05 2024-09-17T22:41:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459746+0000", 2024-09-17T22:41:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0553000000000002e-05 2024-09-17T22:41:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5429 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:4 [read 1558528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.419134+0000", 2024-09-17T22:41:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.862007728, 2024-09-17T22:41:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040734266999999998, 2024-09-17T22:41:29.338 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5429 2024-09-17T22:41:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419134+0000", 2024-09-17T22:41:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419134+0000", 2024-09-17T22:41:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419135+0000", 2024-09-17T22:41:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:41:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419137+0000", 2024-09-17T22:41:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8169999999999999e-06 2024-09-17T22:41:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419142+0000", 2024-09-17T22:41:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5300000000000004e-06 2024-09-17T22:41:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419147+0000", 2024-09-17T22:41:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7140000000000001e-06 2024-09-17T22:41:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459772+0000", 2024-09-17T22:41:29.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040624594999999999 2024-09-17T22:41:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459813+0000", 2024-09-17T22:41:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1371000000000001e-05 2024-09-17T22:41:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459869+0000", 2024-09-17T22:41:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5838000000000003e-05 2024-09-17T22:41:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5430 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:4 [read 1581056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.419153+0000", 2024-09-17T22:41:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8619890149999998, 2024-09-17T22:41:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040830459999999999, 2024-09-17T22:41:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5430 2024-09-17T22:41:29.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419153+0000", 2024-09-17T22:41:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419153+0000", 2024-09-17T22:41:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419154+0000", 2024-09-17T22:41:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:41:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419155+0000", 2024-09-17T22:41:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7999999999999999e-06 2024-09-17T22:41:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419161+0000", 2024-09-17T22:41:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9540000000000001e-06 2024-09-17T22:41:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419166+0000", 2024-09-17T22:41:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7060000000000003e-06 2024-09-17T22:41:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459894+0000", 2024-09-17T22:41:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040728417000000003 2024-09-17T22:41:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459934+0000", 2024-09-17T22:41:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9235e-05 2024-09-17T22:41:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.459984+0000", 2024-09-17T22:41:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9963000000000003e-05 2024-09-17T22:41:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5431 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:4 [read 1599488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.419172+0000", 2024-09-17T22:41:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8619704920000002, 2024-09-17T22:41:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040937711000000002, 2024-09-17T22:41:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.347 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5431 2024-09-17T22:41:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419172+0000", 2024-09-17T22:41:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419172+0000", 2024-09-17T22:41:29.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419172+0000", 2024-09-17T22:41:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:41:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419174+0000", 2024-09-17T22:41:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8190000000000001e-06 2024-09-17T22:41:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419180+0000", 2024-09-17T22:41:29.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6849999999999998e-06 2024-09-17T22:41:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419185+0000", 2024-09-17T22:41:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.135e-06 2024-09-17T22:41:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.460008+0000", 2024-09-17T22:41:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040823115 2024-09-17T22:41:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.460057+0000", 2024-09-17T22:41:29.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8927000000000001e-05 2024-09-17T22:41:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.460109+0000", 2024-09-17T22:41:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2630000000000003e-05 2024-09-17T22:41:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5432 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:4 [read 1617920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.419190+0000", 2024-09-17T22:41:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8619520559999998, 2024-09-17T22:41:29.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041043617999999997, 2024-09-17T22:41:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5432 2024-09-17T22:41:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419190+0000", 2024-09-17T22:41:29.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419190+0000", 2024-09-17T22:41:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419190+0000", 2024-09-17T22:41:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.65e-07 2024-09-17T22:41:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419197+0000", 2024-09-17T22:41:29.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1619999999999996e-06 2024-09-17T22:41:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419203+0000", 2024-09-17T22:41:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9120000000000003e-06 2024-09-17T22:41:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419208+0000", 2024-09-17T22:41:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8170000000000004e-06 2024-09-17T22:41:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.460137+0000", 2024-09-17T22:41:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040929048000000003 2024-09-17T22:41:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.460179+0000", 2024-09-17T22:41:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1145e-05 2024-09-17T22:41:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.460234+0000", 2024-09-17T22:41:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5169e-05 2024-09-17T22:41:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.355 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5433 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:4 [read 1636352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.419215+0000", 2024-09-17T22:41:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.861927321, 2024-09-17T22:41:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041141690000000002, 2024-09-17T22:41:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5433 2024-09-17T22:41:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419215+0000", 2024-09-17T22:41:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419215+0000", 2024-09-17T22:41:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419215+0000", 2024-09-17T22:41:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:41:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419217+0000", 2024-09-17T22:41:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.733e-06 2024-09-17T22:41:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419223+0000", 2024-09-17T22:41:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7649999999999999e-06 2024-09-17T22:41:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419227+0000", 2024-09-17T22:41:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3889999999999998e-06 2024-09-17T22:41:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.460259+0000", 2024-09-17T22:41:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041031474999999998 2024-09-17T22:41:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.460302+0000", 2024-09-17T22:41:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3349000000000001e-05 2024-09-17T22:41:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.460357+0000", 2024-09-17T22:41:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4568000000000003e-05 2024-09-17T22:41:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5434 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:4 [read 1654784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.419233+0000", 2024-09-17T22:41:29.360 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.861909534, 2024-09-17T22:41:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041242763000000002, 2024-09-17T22:41:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5434 2024-09-17T22:41:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419233+0000", 2024-09-17T22:41:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419233+0000", 2024-09-17T22:41:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419233+0000", 2024-09-17T22:41:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5999999999999999e-07 2024-09-17T22:41:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419235+0000", 2024-09-17T22:41:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8059999999999999e-06 2024-09-17T22:41:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419241+0000", 2024-09-17T22:41:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0360000000000003e-06 2024-09-17T22:41:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419246+0000", 2024-09-17T22:41:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0289999999999996e-06 2024-09-17T22:41:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.460382+0000", 2024-09-17T22:41:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041136472 2024-09-17T22:41:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.460423+0000", 2024-09-17T22:41:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0154000000000002e-05 2024-09-17T22:41:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.460475+0000", 2024-09-17T22:41:29.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2905999999999998e-05 2024-09-17T22:41:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5435 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:4 [read 1673216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.419252+0000", 2024-09-17T22:41:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8618899139999998, 2024-09-17T22:41:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041346634, 2024-09-17T22:41:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.365 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5435 2024-09-17T22:41:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419252+0000", 2024-09-17T22:41:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419252+0000", 2024-09-17T22:41:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419253+0000", 2024-09-17T22:41:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:41:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419255+0000", 2024-09-17T22:41:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9199999999999998e-06 2024-09-17T22:41:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419260+0000", 2024-09-17T22:41:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6879999999999996e-06 2024-09-17T22:41:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419265+0000", 2024-09-17T22:41:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2899999999999996e-06 2024-09-17T22:41:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.460501+0000", 2024-09-17T22:41:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041235991999999999 2024-09-17T22:41:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.460544+0000", 2024-09-17T22:41:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3219000000000001e-05 2024-09-17T22:41:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.460599+0000", 2024-09-17T22:41:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5151000000000001e-05 2024-09-17T22:41:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:5436 3.5 3:a74480d9:::rbd_data.115f1ca54530.0000000000000007:4 [read 1691648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:29.369 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:26.419271+0000", 2024-09-17T22:41:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.8618713919999998, 2024-09-17T22:41:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041436307999999998, 2024-09-17T22:41:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 5436 2024-09-17T22:41:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419271+0000", 2024-09-17T22:41:29.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419271+0000", 2024-09-17T22:41:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419271+0000", 2024-09-17T22:41:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0499999999999999e-07 2024-09-17T22:41:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419273+0000", 2024-09-17T22:41:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7430000000000001e-06 2024-09-17T22:41:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419279+0000", 2024-09-17T22:41:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7610000000000004e-06 2024-09-17T22:41:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.419283+0000", 2024-09-17T22:41:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3139999999999997e-06 2024-09-17T22:41:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:29.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.460616+0000", 2024-09-17T22:41:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041333524000000003 2024-09-17T22:41:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.460656+0000", 2024-09-17T22:41:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9743000000000002e-05 2024-09-17T22:41:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:26.460707+0000", 2024-09-17T22:41:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0918000000000003e-05 2024-09-17T22:41:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.373 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.374 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.374 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.374 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:29.374 INFO:teuthology.orchestra.run.smithi115.stdout:277 write 0x835087b thru 0x835ba19 (0xb19f bytes) 2024-09-17T22:41:29.375 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:41:29.377 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:29.377 INFO:teuthology.orchestra.run.smithi115.stdout:279 trunc from 0xb1fb0f4 to 0xcccb26d 2024-09-17T22:41:29.377 INFO:teuthology.orchestra.run.smithi115.stdout:280 write 0x4f6bfae thru 0x4f72097 (0x60ea bytes) 2024-09-17T22:41:29.377 INFO:teuthology.orchestra.run.smithi115.stdout:282 punch from 0x5d02ebc to 0x5d03462, (0x5a6 bytes) 2024-09-17T22:41:29.377 INFO:teuthology.orchestra.run.smithi115.stdout:283 read 0xb692d9e thru 0xb694d5e (0x1fc1 bytes) 2024-09-17T22:41:29.377 INFO:teuthology.orchestra.run.smithi115.stdout:284 write 0xaa5a62a thru 0xaa61e64 (0x783b bytes) 2024-09-17T22:41:29.377 INFO:teuthology.orchestra.run.smithi115.stdout:285 write 0x96e76f1 thru 0x96effe4 (0x88f4 bytes) 2024-09-17T22:41:29.377 INFO:teuthology.orchestra.run.smithi115.stdout:287 punch from 0x119cbd3 to 0x119eba4, (0x1fd1 bytes) 2024-09-17T22:41:29.377 INFO:teuthology.orchestra.run.smithi115.stdout:288 punch from 0x40b3ccf to 0x40c1871, (0xdba2 bytes) 2024-09-17T22:41:29.377 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:29.377 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:29.372+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:29.379 INFO:teuthology.orchestra.run.smithi115.stdout:290 trunc from 0xcccb26d to 0x8f70dfa 2024-09-17T22:41:29.405 INFO:teuthology.orchestra.run.smithi115.stdout:291 read 0x3a9d796 thru 0x3aaa58f (0xcdfa bytes) 2024-09-17T22:41:29.415 INFO:teuthology.orchestra.run.smithi115.stdout:293 write 0xcf172e4 thru 0xcf19104 (0x1e21 bytes) 2024-09-17T22:41:29.425 INFO:teuthology.orchestra.run.smithi115.stdout:294 write 0xeb99e8d thru 0xeba851d (0xe691 bytes) 2024-09-17T22:41:29.432 INFO:teuthology.orchestra.run.smithi115.stdout:295 punch from 0x93931c2 to 0x939cf01, (0x9d3f bytes) 2024-09-17T22:41:29.450 INFO:teuthology.orchestra.run.smithi115.stdout:296 trunc from 0xeba851e to 0x12d814d 2024-09-17T22:41:29.476 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:29.473+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:29.503 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:41:29.553 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:41:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:34938 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.520783+0000", 2024-09-17T22:41:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020589176000000001, 2024-09-17T22:41:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020616352000000001, 2024-09-17T22:41:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34938 2024-09-17T22:41:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520783+0000", 2024-09-17T22:41:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520783+0000", 2024-09-17T22:41:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520784+0000", 2024-09-17T22:41:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-09-17T22:41:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520786+0000", 2024-09-17T22:41:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.855e-06 2024-09-17T22:41:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520792+0000", 2024-09-17T22:41:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9880000000000001e-06 2024-09-17T22:41:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520797+0000", 2024-09-17T22:41:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0669999999999999e-06 2024-09-17T22:41:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35175 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521442+0000", 2024-09-17T22:41:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019930440000000001, 2024-09-17T22:41:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020021189000000002, 2024-09-17T22:41:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35175 2024-09-17T22:41:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521442+0000", 2024-09-17T22:41:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521442+0000", 2024-09-17T22:41:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521443+0000", 2024-09-17T22:41:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3500000000000002e-07 2024-09-17T22:41:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521445+0000", 2024-09-17T22:41:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0150000000000002e-06 2024-09-17T22:41:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521450+0000", 2024-09-17T22:41:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6790000000000002e-06 2024-09-17T22:41:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521455+0000", 2024-09-17T22:41:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1499999999999998e-06 2024-09-17T22:41:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35410 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.525239+0000", 2024-09-17T22:41:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.016133445999999999, 2024-09-17T22:41:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016283841, 2024-09-17T22:41:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35410 2024-09-17T22:41:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525239+0000", 2024-09-17T22:41:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525239+0000", 2024-09-17T22:41:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525240+0000", 2024-09-17T22:41:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4399999999999999e-07 2024-09-17T22:41:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525244+0000", 2024-09-17T22:41:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-06 2024-09-17T22:41:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525667+0000", 2024-09-17T22:41:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00042268900000000002 2024-09-17T22:41:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525673+0000", 2024-09-17T22:41:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2920000000000001e-06 2024-09-17T22:41:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35701 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540145+0000", 2024-09-17T22:41:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001227522, 2024-09-17T22:41:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014425589999999999, 2024-09-17T22:41:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35701 2024-09-17T22:41:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540145+0000", 2024-09-17T22:41:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540145+0000", 2024-09-17T22:41:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540146+0000", 2024-09-17T22:41:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2e-07 2024-09-17T22:41:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540148+0000", 2024-09-17T22:41:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3190000000000002e-06 2024-09-17T22:41:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540154+0000", 2024-09-17T22:41:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2310000000000001e-06 2024-09-17T22:41:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540159+0000", 2024-09-17T22:41:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.69e-06 2024-09-17T22:41:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:34954 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.520803+0000", 2024-09-17T22:41:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020569687999999999, 2024-09-17T22:41:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020837903000000001, 2024-09-17T22:41:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34954 2024-09-17T22:41:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520803+0000", 2024-09-17T22:41:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520803+0000", 2024-09-17T22:41:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520803+0000", 2024-09-17T22:41:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-09-17T22:41:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520805+0000", 2024-09-17T22:41:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.996e-06 2024-09-17T22:41:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520811+0000", 2024-09-17T22:41:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.083e-06 2024-09-17T22:41:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520816+0000", 2024-09-17T22:41:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.707e-06 2024-09-17T22:41:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35176 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521462+0000", 2024-09-17T22:41:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019910699, 2024-09-17T22:41:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020234064999999999, 2024-09-17T22:41:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35176 2024-09-17T22:41:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521462+0000", 2024-09-17T22:41:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521462+0000", 2024-09-17T22:41:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521462+0000", 2024-09-17T22:41:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3799999999999998e-07 2024-09-17T22:41:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521464+0000", 2024-09-17T22:41:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0109999999999999e-06 2024-09-17T22:41:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521471+0000", 2024-09-17T22:41:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2219999999999999e-06 2024-09-17T22:41:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521475+0000", 2024-09-17T22:41:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6870000000000002e-06 2024-09-17T22:41:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35411 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.525684+0000", 2024-09-17T22:41:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015688631000000001, 2024-09-17T22:41:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016074403000000001, 2024-09-17T22:41:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35411 2024-09-17T22:41:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525684+0000", 2024-09-17T22:41:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525684+0000", 2024-09-17T22:41:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525685+0000", 2024-09-17T22:41:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3799999999999997e-07 2024-09-17T22:41:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525688+0000", 2024-09-17T22:41:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.168e-06 2024-09-17T22:41:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525697+0000", 2024-09-17T22:41:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8049999999999996e-06 2024-09-17T22:41:29.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525702+0000", 2024-09-17T22:41:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1370000000000001e-06 2024-09-17T22:41:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35719 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540165+0000", 2024-09-17T22:41:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0012079840000000001, 2024-09-17T22:41:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001647974, 2024-09-17T22:41:29.577 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35719 2024-09-17T22:41:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540165+0000", 2024-09-17T22:41:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540165+0000", 2024-09-17T22:41:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540165+0000", 2024-09-17T22:41:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3999999999999997e-07 2024-09-17T22:41:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540167+0000", 2024-09-17T22:41:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2649999999999999e-06 2024-09-17T22:41:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540173+0000", 2024-09-17T22:41:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8799999999999996e-06 2024-09-17T22:41:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540178+0000", 2024-09-17T22:41:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6269999999999999e-06 2024-09-17T22:41:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.580 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:34955 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.520822+0000", 2024-09-17T22:41:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020550427999999999, 2024-09-17T22:41:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021043857999999999, 2024-09-17T22:41:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34955 2024-09-17T22:41:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520822+0000", 2024-09-17T22:41:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520822+0000", 2024-09-17T22:41:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520823+0000", 2024-09-17T22:41:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.77e-07 2024-09-17T22:41:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520824+0000", 2024-09-17T22:41:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8929999999999999e-06 2024-09-17T22:41:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520830+0000", 2024-09-17T22:41:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2379999999999997e-06 2024-09-17T22:41:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520834+0000", 2024-09-17T22:41:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3610000000000003e-06 2024-09-17T22:41:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35189 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521482+0000", 2024-09-17T22:41:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019890959, 2024-09-17T22:41:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020438569, 2024-09-17T22:41:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.584 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35189 2024-09-17T22:41:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521482+0000", 2024-09-17T22:41:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521482+0000", 2024-09-17T22:41:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521482+0000", 2024-09-17T22:41:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-17T22:41:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521484+0000", 2024-09-17T22:41:29.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1210000000000001e-06 2024-09-17T22:41:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521491+0000", 2024-09-17T22:41:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4819999999999999e-06 2024-09-17T22:41:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521496+0000", 2024-09-17T22:41:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0309999999999998e-06 2024-09-17T22:41:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35431 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.525710+0000", 2024-09-17T22:41:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015662801000000001, 2024-09-17T22:41:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016271016999999999, 2024-09-17T22:41:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.588 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35431 2024-09-17T22:41:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525710+0000", 2024-09-17T22:41:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525710+0000", 2024-09-17T22:41:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525710+0000", 2024-09-17T22:41:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.82e-07 2024-09-17T22:41:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525713+0000", 2024-09-17T22:41:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4200000000000001e-06 2024-09-17T22:41:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525721+0000", 2024-09-17T22:41:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0569999999999993e-06 2024-09-17T22:41:29.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525726+0000", 2024-09-17T22:41:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1189999999999996e-06 2024-09-17T22:41:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35733 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540183+0000", 2024-09-17T22:41:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001189311, 2024-09-17T22:41:29.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001847036, 2024-09-17T22:41:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35733 2024-09-17T22:41:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540183+0000", 2024-09-17T22:41:29.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540183+0000", 2024-09-17T22:41:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540184+0000", 2024-09-17T22:41:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:41:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540186+0000", 2024-09-17T22:41:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0339999999999999e-06 2024-09-17T22:41:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540192+0000", 2024-09-17T22:41:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.801e-06 2024-09-17T22:41:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540196+0000", 2024-09-17T22:41:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.374e-06 2024-09-17T22:41:29.594 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:34969 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.520845+0000", 2024-09-17T22:41:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020527628999999999, 2024-09-17T22:41:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021241922, 2024-09-17T22:41:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.595 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34969 2024-09-17T22:41:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520845+0000", 2024-09-17T22:41:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520845+0000", 2024-09-17T22:41:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520845+0000", 2024-09-17T22:41:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5400000000000002e-07 2024-09-17T22:41:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520848+0000", 2024-09-17T22:41:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.446e-06 2024-09-17T22:41:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520855+0000", 2024-09-17T22:41:29.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7700000000000004e-06 2024-09-17T22:41:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520860+0000", 2024-09-17T22:41:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.062e-06 2024-09-17T22:41:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35190 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521506+0000", 2024-09-17T22:41:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019866907999999999, 2024-09-17T22:41:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020642425999999998, 2024-09-17T22:41:29.598 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35190 2024-09-17T22:41:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521506+0000", 2024-09-17T22:41:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521506+0000", 2024-09-17T22:41:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521506+0000", 2024-09-17T22:41:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.01e-07 2024-09-17T22:41:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521508+0000", 2024-09-17T22:41:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1050000000000002e-06 2024-09-17T22:41:29.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521516+0000", 2024-09-17T22:41:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9419999999999995e-06 2024-09-17T22:41:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521522+0000", 2024-09-17T22:41:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4859999999999997e-06 2024-09-17T22:41:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35433 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.525733+0000", 2024-09-17T22:41:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015639799999999999, 2024-09-17T22:41:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016456502000000001, 2024-09-17T22:41:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35433 2024-09-17T22:41:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.602 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525733+0000", 2024-09-17T22:41:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525733+0000", 2024-09-17T22:41:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525733+0000", 2024-09-17T22:41:29.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6699999999999999e-07 2024-09-17T22:41:29.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525735+0000", 2024-09-17T22:41:29.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9309999999999998e-06 2024-09-17T22:41:29.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525743+0000", 2024-09-17T22:41:29.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3969999999999997e-06 2024-09-17T22:41:29.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.525748+0000", 2024-09-17T22:41:29.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0939999999999998e-06 2024-09-17T22:41:29.605 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.605 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.605 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.605 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35734 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.605 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540202+0000", 2024-09-17T22:41:29.605 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001170218, 2024-09-17T22:41:29.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020146119999999998, 2024-09-17T22:41:29.606 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.606 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.606 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.606 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35734 2024-09-17T22:41:29.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.606 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540202+0000", 2024-09-17T22:41:29.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540202+0000", 2024-09-17T22:41:29.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540203+0000", 2024-09-17T22:41:29.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2599999999999998e-07 2024-09-17T22:41:29.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540205+0000", 2024-09-17T22:41:29.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3439999999999999e-06 2024-09-17T22:41:29.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540211+0000", 2024-09-17T22:41:29.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8130000000000001e-06 2024-09-17T22:41:29.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540216+0000", 2024-09-17T22:41:29.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5549999999999996e-06 2024-09-17T22:41:29.608 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.608 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.609 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.609 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:34970 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.609 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.520867+0000", 2024-09-17T22:41:29.609 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020505954999999999, 2024-09-17T22:41:29.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021401110000000001, 2024-09-17T22:41:29.609 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.609 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.609 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.609 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.609 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.609 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.610 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34970 2024-09-17T22:41:29.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.610 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520867+0000", 2024-09-17T22:41:29.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520867+0000", 2024-09-17T22:41:29.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520867+0000", 2024-09-17T22:41:29.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0900000000000002e-07 2024-09-17T22:41:29.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520869+0000", 2024-09-17T22:41:29.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0040000000000002e-06 2024-09-17T22:41:29.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520876+0000", 2024-09-17T22:41:29.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5930000000000002e-06 2024-09-17T22:41:29.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520881+0000", 2024-09-17T22:41:29.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-09-17T22:41:29.612 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.612 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.612 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.612 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35203 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.612 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521528+0000", 2024-09-17T22:41:29.613 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019844253999999999, 2024-09-17T22:41:29.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020805832, 2024-09-17T22:41:29.613 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.613 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.613 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.613 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.613 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.613 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.613 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35203 2024-09-17T22:41:29.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.613 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521528+0000", 2024-09-17T22:41:29.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521528+0000", 2024-09-17T22:41:29.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521529+0000", 2024-09-17T22:41:29.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9299999999999999e-07 2024-09-17T22:41:29.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521531+0000", 2024-09-17T22:41:29.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1339999999999998e-06 2024-09-17T22:41:29.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521538+0000", 2024-09-17T22:41:29.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2949999999999996e-06 2024-09-17T22:41:29.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521543+0000", 2024-09-17T22:41:29.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1139999999999997e-06 2024-09-17T22:41:29.615 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.615 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.615 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.616 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35455 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.616 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.530070+0000", 2024-09-17T22:41:29.616 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.011302515000000001, 2024-09-17T22:41:29.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012316931999999999, 2024-09-17T22:41:29.616 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.616 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.616 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.616 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35455 2024-09-17T22:41:29.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.617 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530070+0000", 2024-09-17T22:41:29.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530070+0000", 2024-09-17T22:41:29.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530071+0000", 2024-09-17T22:41:29.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2640000000000001e-06 2024-09-17T22:41:29.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530077+0000", 2024-09-17T22:41:29.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6269999999999997e-06 2024-09-17T22:41:29.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530088+0000", 2024-09-17T22:41:29.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0492999999999999e-05 2024-09-17T22:41:29.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530095+0000", 2024-09-17T22:41:29.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0700000000000001e-06 2024-09-17T22:41:29.619 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.619 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.619 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.619 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35750 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.619 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540222+0000", 2024-09-17T22:41:29.619 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0011507609999999999, 2024-09-17T22:41:29.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002231144, 2024-09-17T22:41:29.619 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.619 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.619 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.619 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.619 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.620 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.620 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35750 2024-09-17T22:41:29.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.620 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540222+0000", 2024-09-17T22:41:29.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540222+0000", 2024-09-17T22:41:29.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540222+0000", 2024-09-17T22:41:29.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-09-17T22:41:29.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540225+0000", 2024-09-17T22:41:29.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.255e-06 2024-09-17T22:41:29.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540230+0000", 2024-09-17T22:41:29.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8889999999999999e-06 2024-09-17T22:41:29.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540235+0000", 2024-09-17T22:41:29.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.33e-06 2024-09-17T22:41:29.622 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.622 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.622 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.622 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:34987 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.622 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.520887+0000", 2024-09-17T22:41:29.623 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020485293000000002, 2024-09-17T22:41:29.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021623771, 2024-09-17T22:41:29.623 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.623 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.623 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.623 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.623 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.623 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.623 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34987 2024-09-17T22:41:29.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.623 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520887+0000", 2024-09-17T22:41:29.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520887+0000", 2024-09-17T22:41:29.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520888+0000", 2024-09-17T22:41:29.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0499999999999999e-07 2024-09-17T22:41:29.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520890+0000", 2024-09-17T22:41:29.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.877e-06 2024-09-17T22:41:29.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520896+0000", 2024-09-17T22:41:29.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5509999999999996e-06 2024-09-17T22:41:29.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520901+0000", 2024-09-17T22:41:29.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0420000000000002e-06 2024-09-17T22:41:29.626 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.626 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.626 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.626 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35204 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.626 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521550+0000", 2024-09-17T22:41:29.626 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019822843, 2024-09-17T22:41:29.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021031551999999998, 2024-09-17T22:41:29.626 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.626 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.626 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.627 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.627 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.627 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35204 2024-09-17T22:41:29.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.627 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521550+0000", 2024-09-17T22:41:29.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521550+0000", 2024-09-17T22:41:29.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521550+0000", 2024-09-17T22:41:29.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:41:29.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521552+0000", 2024-09-17T22:41:29.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.852e-06 2024-09-17T22:41:29.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521560+0000", 2024-09-17T22:41:29.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.571e-06 2024-09-17T22:41:29.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521565+0000", 2024-09-17T22:41:29.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3530000000000002e-06 2024-09-17T22:41:29.629 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.629 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.629 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.630 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35456 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.630 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.530111+0000", 2024-09-17T22:41:29.630 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.011261879000000001, 2024-09-17T22:41:29.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012523967, 2024-09-17T22:41:29.630 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.630 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.630 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.630 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.630 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.630 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.630 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35456 2024-09-17T22:41:29.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.630 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530111+0000", 2024-09-17T22:41:29.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530111+0000", 2024-09-17T22:41:29.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530111+0000", 2024-09-17T22:41:29.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4299999999999998e-07 2024-09-17T22:41:29.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530114+0000", 2024-09-17T22:41:29.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8279999999999999e-06 2024-09-17T22:41:29.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530122+0000", 2024-09-17T22:41:29.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1880000000000002e-06 2024-09-17T22:41:29.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530128+0000", 2024-09-17T22:41:29.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6500000000000001e-06 2024-09-17T22:41:29.633 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.633 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.633 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.633 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35751 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.633 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540241+0000", 2024-09-17T22:41:29.633 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001131565, 2024-09-17T22:41:29.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024209940000000001, 2024-09-17T22:41:29.633 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.634 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.634 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.634 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35751 2024-09-17T22:41:29.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.634 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540241+0000", 2024-09-17T22:41:29.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540241+0000", 2024-09-17T22:41:29.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540241+0000", 2024-09-17T22:41:29.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:41:29.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540244+0000", 2024-09-17T22:41:29.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.413e-06 2024-09-17T22:41:29.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540250+0000", 2024-09-17T22:41:29.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1190000000000002e-06 2024-09-17T22:41:29.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540255+0000", 2024-09-17T22:41:29.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7790000000000003e-06 2024-09-17T22:41:29.636 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.636 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.636 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.637 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:34988 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.637 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.520908+0000", 2024-09-17T22:41:29.637 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020464751, 2024-09-17T22:41:29.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021812477, 2024-09-17T22:41:29.637 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.637 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.637 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.637 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.637 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.637 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.637 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34988 2024-09-17T22:41:29.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.638 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520908+0000", 2024-09-17T22:41:29.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520908+0000", 2024-09-17T22:41:29.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520908+0000", 2024-09-17T22:41:29.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8799999999999998e-07 2024-09-17T22:41:29.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520910+0000", 2024-09-17T22:41:29.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.137e-06 2024-09-17T22:41:29.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520916+0000", 2024-09-17T22:41:29.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0739999999999998e-06 2024-09-17T22:41:29.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520921+0000", 2024-09-17T22:41:29.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6659999999999999e-06 2024-09-17T22:41:29.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.640 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.640 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35207 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.640 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521572+0000", 2024-09-17T22:41:29.640 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019800872000000001, 2024-09-17T22:41:29.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021206444000000001, 2024-09-17T22:41:29.640 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.640 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.640 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.641 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.641 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.641 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.641 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35207 2024-09-17T22:41:29.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.641 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521572+0000", 2024-09-17T22:41:29.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521572+0000", 2024-09-17T22:41:29.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521572+0000", 2024-09-17T22:41:29.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3799999999999998e-07 2024-09-17T22:41:29.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521590+0000", 2024-09-17T22:41:29.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7961e-05 2024-09-17T22:41:29.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521599+0000", 2024-09-17T22:41:29.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6710000000000005e-06 2024-09-17T22:41:29.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521605+0000", 2024-09-17T22:41:29.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6699999999999999e-06 2024-09-17T22:41:29.643 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.643 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.643 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.644 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35475 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.644 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.530135+0000", 2024-09-17T22:41:29.644 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.01123715, 2024-09-17T22:41:29.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012693713000000001, 2024-09-17T22:41:29.644 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.644 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.644 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.644 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.644 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.644 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.644 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35475 2024-09-17T22:41:29.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.644 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530135+0000", 2024-09-17T22:41:29.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530135+0000", 2024-09-17T22:41:29.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530136+0000", 2024-09-17T22:41:29.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4499999999999997e-07 2024-09-17T22:41:29.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530139+0000", 2024-09-17T22:41:29.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.768e-06 2024-09-17T22:41:29.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530146+0000", 2024-09-17T22:41:29.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3309999999999998e-06 2024-09-17T22:41:29.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530151+0000", 2024-09-17T22:41:29.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.045e-06 2024-09-17T22:41:29.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.646 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.647 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.647 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35763 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.647 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540261+0000", 2024-09-17T22:41:29.647 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001111419, 2024-09-17T22:41:29.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026292429999999999, 2024-09-17T22:41:29.647 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.647 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.647 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.647 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.647 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.647 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.647 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35763 2024-09-17T22:41:29.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.648 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540261+0000", 2024-09-17T22:41:29.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540261+0000", 2024-09-17T22:41:29.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540262+0000", 2024-09-17T22:41:29.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-09-17T22:41:29.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540264+0000", 2024-09-17T22:41:29.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.43e-06 2024-09-17T22:41:29.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540270+0000", 2024-09-17T22:41:29.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9529999999999996e-06 2024-09-17T22:41:29.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540274+0000", 2024-09-17T22:41:29.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4610000000000001e-06 2024-09-17T22:41:29.650 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.650 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.650 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.650 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35001 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.650 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.520927+0000", 2024-09-17T22:41:29.650 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020445193, 2024-09-17T22:41:29.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022022076000000002, 2024-09-17T22:41:29.650 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.650 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.650 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.651 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.651 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.651 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35001 2024-09-17T22:41:29.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.651 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520927+0000", 2024-09-17T22:41:29.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520927+0000", 2024-09-17T22:41:29.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520928+0000", 2024-09-17T22:41:29.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9499999999999998e-07 2024-09-17T22:41:29.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520930+0000", 2024-09-17T22:41:29.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9379999999999999e-06 2024-09-17T22:41:29.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520936+0000", 2024-09-17T22:41:29.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1199999999999999e-06 2024-09-17T22:41:29.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520941+0000", 2024-09-17T22:41:29.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0590000000000002e-06 2024-09-17T22:41:29.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.653 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.653 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35208 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.654 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521613+0000", 2024-09-17T22:41:29.654 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019760025000000001, 2024-09-17T22:41:29.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021388569, 2024-09-17T22:41:29.654 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.654 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.654 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.654 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35208 2024-09-17T22:41:29.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.654 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521613+0000", 2024-09-17T22:41:29.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521613+0000", 2024-09-17T22:41:29.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521613+0000", 2024-09-17T22:41:29.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:41:29.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521615+0000", 2024-09-17T22:41:29.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0250000000000001e-06 2024-09-17T22:41:29.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521622+0000", 2024-09-17T22:41:29.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1359999999999999e-06 2024-09-17T22:41:29.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521628+0000", 2024-09-17T22:41:29.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4890000000000003e-06 2024-09-17T22:41:29.656 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.656 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.656 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.657 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35476 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.657 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.530158+0000", 2024-09-17T22:41:29.657 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.011214261999999999, 2024-09-17T22:41:29.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012898623999999999, 2024-09-17T22:41:29.657 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.657 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.657 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.657 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35476 2024-09-17T22:41:29.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.657 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530158+0000", 2024-09-17T22:41:29.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530158+0000", 2024-09-17T22:41:29.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530159+0000", 2024-09-17T22:41:29.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6199999999999999e-07 2024-09-17T22:41:29.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530161+0000", 2024-09-17T22:41:29.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9810000000000002e-06 2024-09-17T22:41:29.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530169+0000", 2024-09-17T22:41:29.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4319999999999999e-06 2024-09-17T22:41:29.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530175+0000", 2024-09-17T22:41:29.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4269999999999999e-06 2024-09-17T22:41:29.659 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.660 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.660 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.660 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35764 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.660 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540281+0000", 2024-09-17T22:41:29.660 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001092123, 2024-09-17T22:41:29.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028351180000000002, 2024-09-17T22:41:29.660 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.660 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.660 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.660 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.660 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.660 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35764 2024-09-17T22:41:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540281+0000", 2024-09-17T22:41:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540281+0000", 2024-09-17T22:41:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540281+0000", 2024-09-17T22:41:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-09-17T22:41:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540287+0000", 2024-09-17T22:41:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9789999999999998e-06 2024-09-17T22:41:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540293+0000", 2024-09-17T22:41:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3779999999999998e-06 2024-09-17T22:41:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540298+0000", 2024-09-17T22:41:29.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4320000000000001e-06 2024-09-17T22:41:29.663 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.663 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.663 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.663 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35002 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.663 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.520947+0000", 2024-09-17T22:41:29.663 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020425398000000001, 2024-09-17T22:41:29.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022197096999999999, 2024-09-17T22:41:29.663 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.663 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.663 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.664 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35002 2024-09-17T22:41:29.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.664 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520947+0000", 2024-09-17T22:41:29.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520947+0000", 2024-09-17T22:41:29.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520948+0000", 2024-09-17T22:41:29.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000002e-07 2024-09-17T22:41:29.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520950+0000", 2024-09-17T22:41:29.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0140000000000001e-06 2024-09-17T22:41:29.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520956+0000", 2024-09-17T22:41:29.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4180000000000002e-06 2024-09-17T22:41:29.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520961+0000", 2024-09-17T22:41:29.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9860000000000002e-06 2024-09-17T22:41:29.666 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.666 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.666 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.666 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35221 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.666 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521637+0000", 2024-09-17T22:41:29.666 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019735925000000001, 2024-09-17T22:41:29.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02157249, 2024-09-17T22:41:29.667 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.667 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.667 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.667 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.667 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.667 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.667 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35221 2024-09-17T22:41:29.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.667 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521637+0000", 2024-09-17T22:41:29.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521637+0000", 2024-09-17T22:41:29.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521637+0000", 2024-09-17T22:41:29.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7899999999999999e-07 2024-09-17T22:41:29.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521640+0000", 2024-09-17T22:41:29.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3149999999999999e-06 2024-09-17T22:41:29.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521647+0000", 2024-09-17T22:41:29.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6720000000000004e-06 2024-09-17T22:41:29.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521653+0000", 2024-09-17T22:41:29.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7100000000000004e-06 2024-09-17T22:41:29.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.669 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.669 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35496 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.670 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.530181+0000", 2024-09-17T22:41:29.670 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.011191603, 2024-09-17T22:41:29.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013086647999999999, 2024-09-17T22:41:29.670 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.670 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.670 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.670 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35496 2024-09-17T22:41:29.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.670 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530181+0000", 2024-09-17T22:41:29.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530181+0000", 2024-09-17T22:41:29.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530181+0000", 2024-09-17T22:41:29.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.84e-07 2024-09-17T22:41:29.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530184+0000", 2024-09-17T22:41:29.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5730000000000002e-06 2024-09-17T22:41:29.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530190+0000", 2024-09-17T22:41:29.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0079999999999999e-06 2024-09-17T22:41:29.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530195+0000", 2024-09-17T22:41:29.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6369999999999998e-06 2024-09-17T22:41:29.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.673 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.673 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35781 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.673 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540304+0000", 2024-09-17T22:41:29.673 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00106854, 2024-09-17T22:41:29.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030455830000000001, 2024-09-17T22:41:29.673 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.673 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.673 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.673 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.674 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35781 2024-09-17T22:41:29.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.674 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540304+0000", 2024-09-17T22:41:29.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540304+0000", 2024-09-17T22:41:29.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540305+0000", 2024-09-17T22:41:29.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2900000000000004e-07 2024-09-17T22:41:29.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540307+0000", 2024-09-17T22:41:29.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2970000000000002e-06 2024-09-17T22:41:29.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540313+0000", 2024-09-17T22:41:29.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1639999999999998e-06 2024-09-17T22:41:29.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540318+0000", 2024-09-17T22:41:29.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4880000000000001e-06 2024-09-17T22:41:29.676 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.676 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.676 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.676 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35019 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.676 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.520967+0000", 2024-09-17T22:41:29.677 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020405172999999999, 2024-09-17T22:41:29.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022418798, 2024-09-17T22:41:29.677 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.677 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.677 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.677 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.677 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.677 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.677 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35019 2024-09-17T22:41:29.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.677 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520967+0000", 2024-09-17T22:41:29.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520967+0000", 2024-09-17T22:41:29.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520968+0000", 2024-09-17T22:41:29.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9499999999999998e-07 2024-09-17T22:41:29.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520970+0000", 2024-09-17T22:41:29.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9549999999999999e-06 2024-09-17T22:41:29.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520976+0000", 2024-09-17T22:41:29.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0120000000000002e-06 2024-09-17T22:41:29.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520981+0000", 2024-09-17T22:41:29.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8280000000000002e-06 2024-09-17T22:41:29.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.680 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.680 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.680 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35222 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.680 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521660+0000", 2024-09-17T22:41:29.680 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019712985999999998, 2024-09-17T22:41:29.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021778968999999999, 2024-09-17T22:41:29.680 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.680 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.680 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.680 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35222 2024-09-17T22:41:29.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.681 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521660+0000", 2024-09-17T22:41:29.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521660+0000", 2024-09-17T22:41:29.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521660+0000", 2024-09-17T22:41:29.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8400000000000005e-07 2024-09-17T22:41:29.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521662+0000", 2024-09-17T22:41:29.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2110000000000001e-06 2024-09-17T22:41:29.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521670+0000", 2024-09-17T22:41:29.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3370000000000002e-06 2024-09-17T22:41:29.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521675+0000", 2024-09-17T22:41:29.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.31e-06 2024-09-17T22:41:29.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.683 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.683 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35497 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.683 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.530201+0000", 2024-09-17T22:41:29.683 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.011171903, 2024-09-17T22:41:29.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013288177999999999, 2024-09-17T22:41:29.683 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.683 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.684 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.684 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35497 2024-09-17T22:41:29.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.684 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530201+0000", 2024-09-17T22:41:29.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530201+0000", 2024-09-17T22:41:29.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530201+0000", 2024-09-17T22:41:29.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000002e-07 2024-09-17T22:41:29.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530203+0000", 2024-09-17T22:41:29.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-09-17T22:41:29.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530209+0000", 2024-09-17T22:41:29.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4070000000000001e-06 2024-09-17T22:41:29.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530213+0000", 2024-09-17T22:41:29.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4109999999999998e-06 2024-09-17T22:41:29.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.686 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.686 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35782 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.686 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540324+0000", 2024-09-17T22:41:29.687 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0010484730000000001, 2024-09-17T22:41:29.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032260800000000001, 2024-09-17T22:41:29.687 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.687 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.687 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.687 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35782 2024-09-17T22:41:29.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.687 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540324+0000", 2024-09-17T22:41:29.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540324+0000", 2024-09-17T22:41:29.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540325+0000", 2024-09-17T22:41:29.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8700000000000001e-07 2024-09-17T22:41:29.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540327+0000", 2024-09-17T22:41:29.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-06 2024-09-17T22:41:29.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540333+0000", 2024-09-17T22:41:29.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.84e-06 2024-09-17T22:41:29.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540337+0000", 2024-09-17T22:41:29.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:41:29.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.689 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.690 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35020 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.690 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.520987+0000", 2024-09-17T22:41:29.690 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020385993000000002, 2024-09-17T22:41:29.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022624241999999999, 2024-09-17T22:41:29.690 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.690 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.690 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.690 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35020 2024-09-17T22:41:29.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.690 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520987+0000", 2024-09-17T22:41:29.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520987+0000", 2024-09-17T22:41:29.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520987+0000", 2024-09-17T22:41:29.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:41:29.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520991+0000", 2024-09-17T22:41:29.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3810000000000001e-06 2024-09-17T22:41:29.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.520998+0000", 2024-09-17T22:41:29.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2530000000000001e-06 2024-09-17T22:41:29.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521003+0000", 2024-09-17T22:41:29.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2889999999999997e-06 2024-09-17T22:41:29.693 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.693 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.693 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.693 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35236 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.693 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521682+0000", 2024-09-17T22:41:29.693 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019691034999999999, 2024-09-17T22:41:29.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022016061, 2024-09-17T22:41:29.693 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.693 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.693 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.693 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.693 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.694 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.694 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35236 2024-09-17T22:41:29.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.694 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521682+0000", 2024-09-17T22:41:29.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521682+0000", 2024-09-17T22:41:29.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521682+0000", 2024-09-17T22:41:29.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:41:29.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521684+0000", 2024-09-17T22:41:29.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1500000000000002e-06 2024-09-17T22:41:29.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521694+0000", 2024-09-17T22:41:29.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3500000000000003e-06 2024-09-17T22:41:29.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521699+0000", 2024-09-17T22:41:29.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5820000000000001e-06 2024-09-17T22:41:29.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.696 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.696 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35518 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.696 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.530219+0000", 2024-09-17T22:41:29.696 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.011153662, 2024-09-17T22:41:29.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013561396999999999, 2024-09-17T22:41:29.696 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.696 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.697 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.697 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35518 2024-09-17T22:41:29.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.697 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530219+0000", 2024-09-17T22:41:29.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530219+0000", 2024-09-17T22:41:29.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530219+0000", 2024-09-17T22:41:29.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-09-17T22:41:29.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530221+0000", 2024-09-17T22:41:29.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9879999999999999e-06 2024-09-17T22:41:29.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530227+0000", 2024-09-17T22:41:29.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7089999999999999e-06 2024-09-17T22:41:29.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.530232+0000", 2024-09-17T22:41:29.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2329999999999997e-06 2024-09-17T22:41:29.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.699 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.699 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35795 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.699 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540343+0000", 2024-09-17T22:41:29.700 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0010292369999999999, 2024-09-17T22:41:29.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0034840790000000002, 2024-09-17T22:41:29.700 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.700 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.700 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.700 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35795 2024-09-17T22:41:29.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.700 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540343+0000", 2024-09-17T22:41:29.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540343+0000", 2024-09-17T22:41:29.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540344+0000", 2024-09-17T22:41:29.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:41:29.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540346+0000", 2024-09-17T22:41:29.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1639999999999999e-06 2024-09-17T22:41:29.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540352+0000", 2024-09-17T22:41:29.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7300000000000002e-06 2024-09-17T22:41:29.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540356+0000", 2024-09-17T22:41:29.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2799999999999997e-06 2024-09-17T22:41:29.702 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.702 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.703 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.703 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35033 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.703 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521010+0000", 2024-09-17T22:41:29.703 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020363109000000001, 2024-09-17T22:41:29.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022865803000000001, 2024-09-17T22:41:29.703 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.703 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.703 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.703 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.703 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.703 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.704 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35033 2024-09-17T22:41:29.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.704 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521010+0000", 2024-09-17T22:41:29.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521010+0000", 2024-09-17T22:41:29.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521010+0000", 2024-09-17T22:41:29.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.84e-07 2024-09-17T22:41:29.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521012+0000", 2024-09-17T22:41:29.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9149999999999999e-06 2024-09-17T22:41:29.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521017+0000", 2024-09-17T22:41:29.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5910000000000003e-06 2024-09-17T22:41:29.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521022+0000", 2024-09-17T22:41:29.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0180000000000001e-06 2024-09-17T22:41:29.706 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.706 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.706 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.706 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35238 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.706 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521706+0000", 2024-09-17T22:41:29.706 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019666670000000001, 2024-09-17T22:41:29.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022257740000000002, 2024-09-17T22:41:29.707 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.707 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.707 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.707 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35238 2024-09-17T22:41:29.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.707 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521706+0000", 2024-09-17T22:41:29.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521706+0000", 2024-09-17T22:41:29.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521706+0000", 2024-09-17T22:41:29.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0699999999999998e-07 2024-09-17T22:41:29.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521709+0000", 2024-09-17T22:41:29.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1229999999999998e-06 2024-09-17T22:41:29.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521715+0000", 2024-09-17T22:41:29.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1229999999999997e-06 2024-09-17T22:41:29.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521719+0000", 2024-09-17T22:41:29.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6210000000000003e-06 2024-09-17T22:41:29.709 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.709 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.709 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.710 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35526 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.710 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.536531+0000", 2024-09-17T22:41:29.710 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0048412170000000001, 2024-09-17T22:41:29.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0075020369999999996, 2024-09-17T22:41:29.710 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.710 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.710 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.710 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.710 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.710 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.710 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35526 2024-09-17T22:41:29.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.710 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536531+0000", 2024-09-17T22:41:29.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536531+0000", 2024-09-17T22:41:29.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536533+0000", 2024-09-17T22:41:29.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.305e-06 2024-09-17T22:41:29.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536538+0000", 2024-09-17T22:41:29.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2360000000000004e-06 2024-09-17T22:41:29.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536549+0000", 2024-09-17T22:41:29.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0558e-05 2024-09-17T22:41:29.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536555+0000", 2024-09-17T22:41:29.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9689999999999997e-06 2024-09-17T22:41:29.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.713 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.713 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35796 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.713 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540361+0000", 2024-09-17T22:41:29.713 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0010111860000000001, 2024-09-17T22:41:29.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003713867, 2024-09-17T22:41:29.714 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.714 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.714 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.714 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.714 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.714 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.714 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35796 2024-09-17T22:41:29.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.714 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540361+0000", 2024-09-17T22:41:29.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540361+0000", 2024-09-17T22:41:29.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540362+0000", 2024-09-17T22:41:29.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5499999999999999e-07 2024-09-17T22:41:29.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540364+0000", 2024-09-17T22:41:29.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1440000000000001e-06 2024-09-17T22:41:29.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540370+0000", 2024-09-17T22:41:29.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8919999999999997e-06 2024-09-17T22:41:29.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540374+0000", 2024-09-17T22:41:29.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3379999999999998e-06 2024-09-17T22:41:29.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.716 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.717 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.717 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35048 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.717 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521029+0000", 2024-09-17T22:41:29.717 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.02034364, 2024-09-17T22:41:29.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023094472000000001, 2024-09-17T22:41:29.717 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.717 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.717 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.718 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35048 2024-09-17T22:41:29.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.718 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521029+0000", 2024-09-17T22:41:29.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521029+0000", 2024-09-17T22:41:29.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521029+0000", 2024-09-17T22:41:29.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:41:29.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521032+0000", 2024-09-17T22:41:29.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1090000000000001e-06 2024-09-17T22:41:29.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521038+0000", 2024-09-17T22:41:29.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4479999999999999e-06 2024-09-17T22:41:29.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521047+0000", 2024-09-17T22:41:29.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4169999999999998e-06 2024-09-17T22:41:29.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.720 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.720 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35239 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.720 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521726+0000", 2024-09-17T22:41:29.720 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019647116999999999, 2024-09-17T22:41:29.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022451202999999999, 2024-09-17T22:41:29.721 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.721 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.721 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.721 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35239 2024-09-17T22:41:29.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.721 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521726+0000", 2024-09-17T22:41:29.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521726+0000", 2024-09-17T22:41:29.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521726+0000", 2024-09-17T22:41:29.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9400000000000001e-07 2024-09-17T22:41:29.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521728+0000", 2024-09-17T22:41:29.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.962e-06 2024-09-17T22:41:29.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521734+0000", 2024-09-17T22:41:29.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3339999999999998e-06 2024-09-17T22:41:29.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521739+0000", 2024-09-17T22:41:29.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-06 2024-09-17T22:41:29.723 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.723 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.723 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.724 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35532 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.724 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.536572+0000", 2024-09-17T22:41:29.724 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0048011169999999997, 2024-09-17T22:41:29.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.007656876, 2024-09-17T22:41:29.724 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.724 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.724 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.725 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35532 2024-09-17T22:41:29.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.725 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536572+0000", 2024-09-17T22:41:29.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536572+0000", 2024-09-17T22:41:29.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536572+0000", 2024-09-17T22:41:29.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1399999999999997e-07 2024-09-17T22:41:29.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536574+0000", 2024-09-17T22:41:29.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4609999999999998e-06 2024-09-17T22:41:29.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536583+0000", 2024-09-17T22:41:29.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1300000000000001e-06 2024-09-17T22:41:29.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536589+0000", 2024-09-17T22:41:29.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0109999999999997e-06 2024-09-17T22:41:29.727 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.727 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.727 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.727 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35813 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.727 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540380+0000", 2024-09-17T22:41:29.727 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0009922869999999999, 2024-09-17T22:41:29.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039008390000000001, 2024-09-17T22:41:29.727 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.727 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.727 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.728 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35813 2024-09-17T22:41:29.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.728 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540380+0000", 2024-09-17T22:41:29.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540380+0000", 2024-09-17T22:41:29.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540381+0000", 2024-09-17T22:41:29.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.65e-07 2024-09-17T22:41:29.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540383+0000", 2024-09-17T22:41:29.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.159e-06 2024-09-17T22:41:29.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540389+0000", 2024-09-17T22:41:29.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9359999999999996e-06 2024-09-17T22:41:29.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540393+0000", 2024-09-17T22:41:29.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3810000000000001e-06 2024-09-17T22:41:29.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.730 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.730 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35061 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.731 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521054+0000", 2024-09-17T22:41:29.731 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020318175000000001, 2024-09-17T22:41:29.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023256031999999999, 2024-09-17T22:41:29.731 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.731 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.731 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.731 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35061 2024-09-17T22:41:29.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.731 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521054+0000", 2024-09-17T22:41:29.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521054+0000", 2024-09-17T22:41:29.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521055+0000", 2024-09-17T22:41:29.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.51e-07 2024-09-17T22:41:29.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521057+0000", 2024-09-17T22:41:29.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1849999999999998e-06 2024-09-17T22:41:29.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521063+0000", 2024-09-17T22:41:29.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1e-06 2024-09-17T22:41:29.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521068+0000", 2024-09-17T22:41:29.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:41:29.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.734 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.734 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35253 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.734 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521745+0000", 2024-09-17T22:41:29.734 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019627210999999999, 2024-09-17T22:41:29.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022611607999999998, 2024-09-17T22:41:29.734 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.734 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.734 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35253 2024-09-17T22:41:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521745+0000", 2024-09-17T22:41:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521745+0000", 2024-09-17T22:41:29.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521746+0000", 2024-09-17T22:41:29.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-09-17T22:41:29.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521748+0000", 2024-09-17T22:41:29.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.95e-06 2024-09-17T22:41:29.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521754+0000", 2024-09-17T22:41:29.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2249999999999997e-06 2024-09-17T22:41:29.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521759+0000", 2024-09-17T22:41:29.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.69e-06 2024-09-17T22:41:29.737 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.737 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.737 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.737 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35533 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.737 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.536598+0000", 2024-09-17T22:41:29.738 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.004774787, 2024-09-17T22:41:29.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.007808232, 2024-09-17T22:41:29.738 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.738 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.738 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.738 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.738 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.738 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.738 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35533 2024-09-17T22:41:29.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.738 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536598+0000", 2024-09-17T22:41:29.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536598+0000", 2024-09-17T22:41:29.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536598+0000", 2024-09-17T22:41:29.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.75e-07 2024-09-17T22:41:29.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536600+0000", 2024-09-17T22:41:29.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1330000000000002e-06 2024-09-17T22:41:29.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536608+0000", 2024-09-17T22:41:29.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3490000000000003e-06 2024-09-17T22:41:29.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536613+0000", 2024-09-17T22:41:29.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1279999999999999e-06 2024-09-17T22:41:29.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.740 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.741 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35814 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.741 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540399+0000", 2024-09-17T22:41:29.741 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00097316600000000003, 2024-09-17T22:41:29.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040663979999999997, 2024-09-17T22:41:29.741 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.741 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.741 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.741 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.741 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.741 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.741 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35814 2024-09-17T22:41:29.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.741 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540399+0000", 2024-09-17T22:41:29.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540399+0000", 2024-09-17T22:41:29.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540400+0000", 2024-09-17T22:41:29.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:41:29.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540402+0000", 2024-09-17T22:41:29.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9609999999999999e-06 2024-09-17T22:41:29.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540408+0000", 2024-09-17T22:41:29.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6890000000000001e-06 2024-09-17T22:41:29.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540412+0000", 2024-09-17T22:41:29.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3370000000000001e-06 2024-09-17T22:41:29.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.743 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.744 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.744 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35062 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.744 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521075+0000", 2024-09-17T22:41:29.744 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020297922999999999, 2024-09-17T22:41:29.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023447567999999998, 2024-09-17T22:41:29.744 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.744 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35062 2024-09-17T22:41:29.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.745 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521075+0000", 2024-09-17T22:41:29.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521075+0000", 2024-09-17T22:41:29.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521075+0000", 2024-09-17T22:41:29.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-09-17T22:41:29.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521081+0000", 2024-09-17T22:41:29.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:41:29.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521087+0000", 2024-09-17T22:41:29.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3960000000000003e-06 2024-09-17T22:41:29.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521092+0000", 2024-09-17T22:41:29.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9300000000000002e-06 2024-09-17T22:41:29.747 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.747 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.747 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.747 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35267 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.747 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521845+0000", 2024-09-17T22:41:29.747 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019527959000000001, 2024-09-17T22:41:29.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022736659999999999, 2024-09-17T22:41:29.747 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.747 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.747 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.747 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.747 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.748 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.748 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35267 2024-09-17T22:41:29.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.748 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521845+0000", 2024-09-17T22:41:29.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521845+0000", 2024-09-17T22:41:29.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521845+0000", 2024-09-17T22:41:29.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.92e-07 2024-09-17T22:41:29.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521849+0000", 2024-09-17T22:41:29.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2119999999999999e-06 2024-09-17T22:41:29.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521857+0000", 2024-09-17T22:41:29.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3469999999999999e-06 2024-09-17T22:41:29.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521863+0000", 2024-09-17T22:41:29.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0990000000000004e-06 2024-09-17T22:41:29.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.750 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.750 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35547 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.750 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.536619+0000", 2024-09-17T22:41:29.750 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0047534730000000002, 2024-09-17T22:41:29.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080152369999999997, 2024-09-17T22:41:29.750 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.750 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.750 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.751 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35547 2024-09-17T22:41:29.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.751 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536619+0000", 2024-09-17T22:41:29.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536619+0000", 2024-09-17T22:41:29.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536620+0000", 2024-09-17T22:41:29.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2399999999999999e-07 2024-09-17T22:41:29.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536622+0000", 2024-09-17T22:41:29.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.086e-06 2024-09-17T22:41:29.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536629+0000", 2024-09-17T22:41:29.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2590000000000003e-06 2024-09-17T22:41:29.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536634+0000", 2024-09-17T22:41:29.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0760000000000002e-06 2024-09-17T22:41:29.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.753 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.754 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35829 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.754 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540418+0000", 2024-09-17T22:41:29.754 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00095441500000000002, 2024-09-17T22:41:29.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042684250000000002, 2024-09-17T22:41:29.754 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.754 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.754 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.754 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35829 2024-09-17T22:41:29.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.754 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540418+0000", 2024-09-17T22:41:29.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540418+0000", 2024-09-17T22:41:29.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540419+0000", 2024-09-17T22:41:29.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6199999999999999e-07 2024-09-17T22:41:29.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540421+0000", 2024-09-17T22:41:29.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-17T22:41:29.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540426+0000", 2024-09-17T22:41:29.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6640000000000003e-06 2024-09-17T22:41:29.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540431+0000", 2024-09-17T22:41:29.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4360000000000004e-06 2024-09-17T22:41:29.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.757 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.757 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.757 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35075 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.757 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521099+0000", 2024-09-17T22:41:29.757 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020273625, 2024-09-17T22:41:29.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023647725000000001, 2024-09-17T22:41:29.757 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.757 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.757 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.757 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35075 2024-09-17T22:41:29.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.758 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521099+0000", 2024-09-17T22:41:29.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521099+0000", 2024-09-17T22:41:29.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521100+0000", 2024-09-17T22:41:29.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1600000000000001e-07 2024-09-17T22:41:29.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521102+0000", 2024-09-17T22:41:29.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2989999999999999e-06 2024-09-17T22:41:29.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521108+0000", 2024-09-17T22:41:29.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0469999999999999e-06 2024-09-17T22:41:29.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521113+0000", 2024-09-17T22:41:29.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9119999999999997e-06 2024-09-17T22:41:29.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.760 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.760 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35268 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.760 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521873+0000", 2024-09-17T22:41:29.760 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019499958000000001, 2024-09-17T22:41:29.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022927249, 2024-09-17T22:41:29.760 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.760 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.760 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.760 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.760 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.761 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35268 2024-09-17T22:41:29.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.761 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521873+0000", 2024-09-17T22:41:29.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521873+0000", 2024-09-17T22:41:29.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521873+0000", 2024-09-17T22:41:29.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.82e-07 2024-09-17T22:41:29.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521875+0000", 2024-09-17T22:41:29.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2110000000000001e-06 2024-09-17T22:41:29.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521882+0000", 2024-09-17T22:41:29.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5010000000000001e-06 2024-09-17T22:41:29.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521887+0000", 2024-09-17T22:41:29.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1909999999999999e-06 2024-09-17T22:41:29.763 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.763 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.763 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.763 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35561 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.763 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.536640+0000", 2024-09-17T22:41:29.763 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0047324680000000001, 2024-09-17T22:41:29.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0082120049999999997, 2024-09-17T22:41:29.764 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.764 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.764 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.764 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.764 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.764 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.764 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35561 2024-09-17T22:41:29.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.764 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536640+0000", 2024-09-17T22:41:29.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536640+0000", 2024-09-17T22:41:29.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536641+0000", 2024-09-17T22:41:29.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0900000000000002e-07 2024-09-17T22:41:29.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536643+0000", 2024-09-17T22:41:29.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0279999999999999e-06 2024-09-17T22:41:29.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536649+0000", 2024-09-17T22:41:29.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0929999999999999e-06 2024-09-17T22:41:29.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536653+0000", 2024-09-17T22:41:29.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6750000000000001e-06 2024-09-17T22:41:29.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.767 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.767 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35849 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.767 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540437+0000", 2024-09-17T22:41:29.767 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00093532499999999998, 2024-09-17T22:41:29.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044701919999999996, 2024-09-17T22:41:29.767 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.767 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.767 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.768 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35849 2024-09-17T22:41:29.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.768 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540437+0000", 2024-09-17T22:41:29.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540437+0000", 2024-09-17T22:41:29.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540438+0000", 2024-09-17T22:41:29.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:41:29.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540440+0000", 2024-09-17T22:41:29.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.103e-06 2024-09-17T22:41:29.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540446+0000", 2024-09-17T22:41:29.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8239999999999996e-06 2024-09-17T22:41:29.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540450+0000", 2024-09-17T22:41:29.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3039999999999998e-06 2024-09-17T22:41:29.770 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.770 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.770 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.770 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35076 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.770 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521119+0000", 2024-09-17T22:41:29.770 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020253673999999999, 2024-09-17T22:41:29.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02384441, 2024-09-17T22:41:29.771 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.771 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.771 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.771 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.771 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.771 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.771 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35076 2024-09-17T22:41:29.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.771 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521119+0000", 2024-09-17T22:41:29.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521119+0000", 2024-09-17T22:41:29.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521119+0000", 2024-09-17T22:41:29.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2300000000000002e-07 2024-09-17T22:41:29.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521121+0000", 2024-09-17T22:41:29.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:41:29.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521127+0000", 2024-09-17T22:41:29.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7039999999999999e-06 2024-09-17T22:41:29.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521132+0000", 2024-09-17T22:41:29.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6609999999999999e-06 2024-09-17T22:41:29.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.773 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.773 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35271 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.774 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521893+0000", 2024-09-17T22:41:29.774 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019479185999999999, 2024-09-17T22:41:29.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023124354, 2024-09-17T22:41:29.774 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.774 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.774 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.774 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35271 2024-09-17T22:41:29.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.774 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521893+0000", 2024-09-17T22:41:29.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521893+0000", 2024-09-17T22:41:29.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521894+0000", 2024-09-17T22:41:29.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9299999999999999e-07 2024-09-17T22:41:29.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521896+0000", 2024-09-17T22:41:29.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.057e-06 2024-09-17T22:41:29.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521902+0000", 2024-09-17T22:41:29.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3199999999999996e-06 2024-09-17T22:41:29.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521907+0000", 2024-09-17T22:41:29.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1279999999999999e-06 2024-09-17T22:41:29.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.776 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.777 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.777 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35575 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.777 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.536659+0000", 2024-09-17T22:41:29.777 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0047134270000000001, 2024-09-17T22:41:29.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0084255130000000008, 2024-09-17T22:41:29.777 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.777 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.777 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.777 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35575 2024-09-17T22:41:29.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.778 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536659+0000", 2024-09-17T22:41:29.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536659+0000", 2024-09-17T22:41:29.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536660+0000", 2024-09-17T22:41:29.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:41:29.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536662+0000", 2024-09-17T22:41:29.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1840000000000002e-06 2024-09-17T22:41:29.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536671+0000", 2024-09-17T22:41:29.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5650000000000007e-06 2024-09-17T22:41:29.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536677+0000", 2024-09-17T22:41:29.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3839999999999996e-06 2024-09-17T22:41:29.780 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.780 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.780 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.780 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35861 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.780 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.541161+0000", 2024-09-17T22:41:29.780 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00021167899999999999, 2024-09-17T22:41:29.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039767099999999996, 2024-09-17T22:41:29.780 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.780 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.781 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.781 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.781 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.781 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.781 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35861 2024-09-17T22:41:29.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.781 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.541161+0000", 2024-09-17T22:41:29.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.541161+0000", 2024-09-17T22:41:29.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.541162+0000", 2024-09-17T22:41:29.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4300000000000002e-07 2024-09-17T22:41:29.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.541166+0000", 2024-09-17T22:41:29.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2490000000000003e-06 2024-09-17T22:41:29.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.541174+0000", 2024-09-17T22:41:29.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3159999999999997e-06 2024-09-17T22:41:29.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.541181+0000", 2024-09-17T22:41:29.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3539999999999996e-06 2024-09-17T22:41:29.783 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.783 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.783 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.783 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35079 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.783 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521138+0000", 2024-09-17T22:41:29.783 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020234334999999999, 2024-09-17T22:41:29.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024051281000000001, 2024-09-17T22:41:29.784 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.784 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.784 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.784 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.784 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.784 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.784 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35079 2024-09-17T22:41:29.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.784 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521138+0000", 2024-09-17T22:41:29.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521138+0000", 2024-09-17T22:41:29.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521139+0000", 2024-09-17T22:41:29.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9799999999999999e-07 2024-09-17T22:41:29.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521141+0000", 2024-09-17T22:41:29.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1129999999999999e-06 2024-09-17T22:41:29.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521147+0000", 2024-09-17T22:41:29.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3330000000000002e-06 2024-09-17T22:41:29.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521152+0000", 2024-09-17T22:41:29.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4109999999999998e-06 2024-09-17T22:41:29.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.786 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.786 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35285 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.787 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521914+0000", 2024-09-17T22:41:29.787 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019459028999999999, 2024-09-17T22:41:29.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023357403999999998, 2024-09-17T22:41:29.787 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.787 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.787 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.787 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35285 2024-09-17T22:41:29.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.787 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521914+0000", 2024-09-17T22:41:29.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521914+0000", 2024-09-17T22:41:29.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521914+0000", 2024-09-17T22:41:29.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.53e-07 2024-09-17T22:41:29.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521916+0000", 2024-09-17T22:41:29.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9369999999999998e-06 2024-09-17T22:41:29.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521922+0000", 2024-09-17T22:41:29.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4810000000000003e-06 2024-09-17T22:41:29.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521927+0000", 2024-09-17T22:41:29.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8629999999999999e-06 2024-09-17T22:41:29.789 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.789 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.789 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.790 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35576 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.790 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.536683+0000", 2024-09-17T22:41:29.790 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0046894900000000002, 2024-09-17T22:41:29.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0086405030000000008, 2024-09-17T22:41:29.790 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.790 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.790 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.790 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35576 2024-09-17T22:41:29.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.790 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536683+0000", 2024-09-17T22:41:29.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536683+0000", 2024-09-17T22:41:29.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536683+0000", 2024-09-17T22:41:29.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3799999999999998e-07 2024-09-17T22:41:29.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536685+0000", 2024-09-17T22:41:29.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0040000000000002e-06 2024-09-17T22:41:29.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536691+0000", 2024-09-17T22:41:29.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3360000000000002e-06 2024-09-17T22:41:29.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536697+0000", 2024-09-17T22:41:29.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3269999999999997e-06 2024-09-17T22:41:29.792 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.792 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.793 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.793 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35862 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.793 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.541192+0000", 2024-09-17T22:41:29.793 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.000180185, 2024-09-17T22:41:29.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041883729999999996, 2024-09-17T22:41:29.793 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.793 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.793 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.793 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.793 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.793 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.793 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35862 2024-09-17T22:41:29.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.794 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.541192+0000", 2024-09-17T22:41:29.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.541192+0000", 2024-09-17T22:41:29.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.541193+0000", 2024-09-17T22:41:29.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7599999999999997e-07 2024-09-17T22:41:29.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.541195+0000", 2024-09-17T22:41:29.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5220000000000002e-06 2024-09-17T22:41:29.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.541202+0000", 2024-09-17T22:41:29.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8020000000000003e-06 2024-09-17T22:41:29.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.541207+0000", 2024-09-17T22:41:29.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1139999999999997e-06 2024-09-17T22:41:29.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.796 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.796 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35080 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.796 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521157+0000", 2024-09-17T22:41:29.796 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020215202000000002, 2024-09-17T22:41:29.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024280860000000001, 2024-09-17T22:41:29.796 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.797 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.797 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.797 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.797 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.797 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.797 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35080 2024-09-17T22:41:29.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.797 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521157+0000", 2024-09-17T22:41:29.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521157+0000", 2024-09-17T22:41:29.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521158+0000", 2024-09-17T22:41:29.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.08e-07 2024-09-17T22:41:29.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521160+0000", 2024-09-17T22:41:29.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1639999999999999e-06 2024-09-17T22:41:29.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521167+0000", 2024-09-17T22:41:29.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5379999999999999e-06 2024-09-17T22:41:29.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521171+0000", 2024-09-17T22:41:29.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4209999999999997e-06 2024-09-17T22:41:29.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.800 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.800 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35286 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.800 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521934+0000", 2024-09-17T22:41:29.800 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019439090999999999, 2024-09-17T22:41:29.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023557998, 2024-09-17T22:41:29.800 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.800 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.800 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.800 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35286 2024-09-17T22:41:29.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.801 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521934+0000", 2024-09-17T22:41:29.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521934+0000", 2024-09-17T22:41:29.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521934+0000", 2024-09-17T22:41:29.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3000000000000001e-07 2024-09-17T22:41:29.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521936+0000", 2024-09-17T22:41:29.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1040000000000001e-06 2024-09-17T22:41:29.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521942+0000", 2024-09-17T22:41:29.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9059999999999999e-06 2024-09-17T22:41:29.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521947+0000", 2024-09-17T22:41:29.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-09-17T22:41:29.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.803 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.803 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35591 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.803 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.536703+0000", 2024-09-17T22:41:29.803 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0046691780000000004, 2024-09-17T22:41:29.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0088393069999999994, 2024-09-17T22:41:29.804 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.804 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.804 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.804 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35591 2024-09-17T22:41:29.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.804 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536703+0000", 2024-09-17T22:41:29.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536703+0000", 2024-09-17T22:41:29.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536704+0000", 2024-09-17T22:41:29.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:41:29.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536706+0000", 2024-09-17T22:41:29.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1390000000000002e-06 2024-09-17T22:41:29.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536713+0000", 2024-09-17T22:41:29.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.781e-06 2024-09-17T22:41:29.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536718+0000", 2024-09-17T22:41:29.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1880000000000001e-06 2024-09-17T22:41:29.807 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.807 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.807 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.807 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35874 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.807 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.541215+0000", 2024-09-17T22:41:29.807 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00015775000000000001, 2024-09-17T22:41:29.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0043879820000000003, 2024-09-17T22:41:29.807 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.807 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.807 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.807 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.807 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.807 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.808 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35874 2024-09-17T22:41:29.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.808 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.541215+0000", 2024-09-17T22:41:29.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.541215+0000", 2024-09-17T22:41:29.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.541215+0000", 2024-09-17T22:41:29.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:41:29.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.541218+0000", 2024-09-17T22:41:29.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2670000000000001e-06 2024-09-17T22:41:29.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.541224+0000", 2024-09-17T22:41:29.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5180000000000001e-06 2024-09-17T22:41:29.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.541229+0000", 2024-09-17T22:41:29.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7360000000000001e-06 2024-09-17T22:41:29.810 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.810 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.810 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.810 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35093 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.810 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521178+0000", 2024-09-17T22:41:29.810 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020194700999999999, 2024-09-17T22:41:29.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024477802, 2024-09-17T22:41:29.810 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.811 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.811 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.811 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35093 2024-09-17T22:41:29.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.811 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521178+0000", 2024-09-17T22:41:29.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521178+0000", 2024-09-17T22:41:29.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521178+0000", 2024-09-17T22:41:29.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.27e-07 2024-09-17T22:41:29.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521180+0000", 2024-09-17T22:41:29.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9069999999999999e-06 2024-09-17T22:41:29.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521187+0000", 2024-09-17T22:41:29.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4289999999999998e-06 2024-09-17T22:41:29.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521192+0000", 2024-09-17T22:41:29.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.882e-06 2024-09-17T22:41:29.813 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.813 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.813 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.813 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35299 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.813 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521953+0000", 2024-09-17T22:41:29.814 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019419478, 2024-09-17T22:41:29.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023761978999999999, 2024-09-17T22:41:29.814 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.814 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.814 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.814 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.814 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.814 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.814 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35299 2024-09-17T22:41:29.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.814 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521953+0000", 2024-09-17T22:41:29.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521953+0000", 2024-09-17T22:41:29.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521954+0000", 2024-09-17T22:41:29.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9200000000000002e-07 2024-09-17T22:41:29.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521956+0000", 2024-09-17T22:41:29.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0549999999999998e-06 2024-09-17T22:41:29.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521962+0000", 2024-09-17T22:41:29.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9050000000000002e-06 2024-09-17T22:41:29.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521967+0000", 2024-09-17T22:41:29.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.237e-06 2024-09-17T22:41:29.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.816 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.817 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35592 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.817 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.536724+0000", 2024-09-17T22:41:29.817 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0046484029999999997, 2024-09-17T22:41:29.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0090511879999999999, 2024-09-17T22:41:29.817 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.817 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.817 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.817 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35592 2024-09-17T22:41:29.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.817 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536724+0000", 2024-09-17T22:41:29.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536724+0000", 2024-09-17T22:41:29.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536725+0000", 2024-09-17T22:41:29.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1199999999999998e-07 2024-09-17T22:41:29.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536727+0000", 2024-09-17T22:41:29.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.057e-06 2024-09-17T22:41:29.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536733+0000", 2024-09-17T22:41:29.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7089999999999996e-06 2024-09-17T22:41:29.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536738+0000", 2024-09-17T22:41:29.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.916e-06 2024-09-17T22:41:29.819 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.819 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.819 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.820 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35094 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.820 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521198+0000", 2024-09-17T22:41:29.820 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020175052999999998, 2024-09-17T22:41:29.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024637803999999999, 2024-09-17T22:41:29.820 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.820 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.820 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.820 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.820 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.820 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.820 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35094 2024-09-17T22:41:29.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.820 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521198+0000", 2024-09-17T22:41:29.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521198+0000", 2024-09-17T22:41:29.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521198+0000", 2024-09-17T22:41:29.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9200000000000002e-07 2024-09-17T22:41:29.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521200+0000", 2024-09-17T22:41:29.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9970000000000001e-06 2024-09-17T22:41:29.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521206+0000", 2024-09-17T22:41:29.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1129999999999997e-06 2024-09-17T22:41:29.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521211+0000", 2024-09-17T22:41:29.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7679999999999999e-06 2024-09-17T22:41:29.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.823 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.823 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35300 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.823 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521973+0000", 2024-09-17T22:41:29.823 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019399567, 2024-09-17T22:41:29.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023942714, 2024-09-17T22:41:29.823 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.823 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.823 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.824 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.824 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.824 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.824 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35300 2024-09-17T22:41:29.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.824 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521973+0000", 2024-09-17T22:41:29.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521973+0000", 2024-09-17T22:41:29.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521973+0000", 2024-09-17T22:41:29.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9200000000000002e-07 2024-09-17T22:41:29.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521977+0000", 2024-09-17T22:41:29.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1769999999999998e-06 2024-09-17T22:41:29.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521983+0000", 2024-09-17T22:41:29.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2820000000000002e-06 2024-09-17T22:41:29.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521988+0000", 2024-09-17T22:41:29.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9289999999999997e-06 2024-09-17T22:41:29.826 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.826 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.826 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.826 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35605 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.826 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.536744+0000", 2024-09-17T22:41:29.827 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0046283460000000002, 2024-09-17T22:41:29.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0092413059999999995, 2024-09-17T22:41:29.827 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.827 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.827 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.827 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.827 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.827 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.827 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35605 2024-09-17T22:41:29.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.827 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536744+0000", 2024-09-17T22:41:29.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536744+0000", 2024-09-17T22:41:29.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536745+0000", 2024-09-17T22:41:29.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4400000000000001e-07 2024-09-17T22:41:29.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536747+0000", 2024-09-17T22:41:29.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.04e-06 2024-09-17T22:41:29.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536754+0000", 2024-09-17T22:41:29.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1300000000000003e-06 2024-09-17T22:41:29.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536759+0000", 2024-09-17T22:41:29.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9099999999999996e-06 2024-09-17T22:41:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35108 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521217+0000", 2024-09-17T22:41:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020155747000000002, 2024-09-17T22:41:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024821916999999999, 2024-09-17T22:41:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.830 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35108 2024-09-17T22:41:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521217+0000", 2024-09-17T22:41:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521217+0000", 2024-09-17T22:41:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521217+0000", 2024-09-17T22:41:29.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:41:29.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521219+0000", 2024-09-17T22:41:29.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.863e-06 2024-09-17T22:41:29.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521225+0000", 2024-09-17T22:41:29.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0120000000000002e-06 2024-09-17T22:41:29.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521230+0000", 2024-09-17T22:41:29.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1089999999999997e-06 2024-09-17T22:41:29.833 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.833 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.833 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.833 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35303 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.833 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521998+0000", 2024-09-17T22:41:29.833 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019374685999999999, 2024-09-17T22:41:29.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024101438999999999, 2024-09-17T22:41:29.834 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.834 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.834 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.834 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.834 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.834 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.834 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35303 2024-09-17T22:41:29.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.834 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521998+0000", 2024-09-17T22:41:29.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521998+0000", 2024-09-17T22:41:29.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521998+0000", 2024-09-17T22:41:29.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.89e-07 2024-09-17T22:41:29.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.522001+0000", 2024-09-17T22:41:29.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.396e-06 2024-09-17T22:41:29.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.522008+0000", 2024-09-17T22:41:29.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0149999999999998e-06 2024-09-17T22:41:29.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.522013+0000", 2024-09-17T22:41:29.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1089999999999997e-06 2024-09-17T22:41:29.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.836 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.836 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35606 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.837 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.536765+0000", 2024-09-17T22:41:29.837 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0046074829999999999, 2024-09-17T22:41:29.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.009373101, 2024-09-17T22:41:29.837 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.837 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.837 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.837 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.837 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.837 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.837 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35606 2024-09-17T22:41:29.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.837 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536765+0000", 2024-09-17T22:41:29.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536765+0000", 2024-09-17T22:41:29.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536766+0000", 2024-09-17T22:41:29.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-09-17T22:41:29.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536768+0000", 2024-09-17T22:41:29.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0389999999999999e-06 2024-09-17T22:41:29.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536775+0000", 2024-09-17T22:41:29.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1330000000000001e-06 2024-09-17T22:41:29.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536780+0000", 2024-09-17T22:41:29.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9339999999999997e-06 2024-09-17T22:41:29.839 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.839 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.839 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.840 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35110 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.840 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521236+0000", 2024-09-17T22:41:29.840 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020136352999999999, 2024-09-17T22:41:29.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024937926999999999, 2024-09-17T22:41:29.840 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.840 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.840 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.840 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.840 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.840 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.840 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35110 2024-09-17T22:41:29.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.841 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521236+0000", 2024-09-17T22:41:29.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521236+0000", 2024-09-17T22:41:29.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521237+0000", 2024-09-17T22:41:29.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8799999999999998e-07 2024-09-17T22:41:29.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521239+0000", 2024-09-17T22:41:29.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9570000000000001e-06 2024-09-17T22:41:29.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521244+0000", 2024-09-17T22:41:29.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8459999999999996e-06 2024-09-17T22:41:29.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521249+0000", 2024-09-17T22:41:29.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8069999999999999e-06 2024-09-17T22:41:29.843 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.843 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.843 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.843 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35304 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.843 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.522020+0000", 2024-09-17T22:41:29.843 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.01935301, 2024-09-17T22:41:29.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024211884999999999, 2024-09-17T22:41:29.843 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.843 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.843 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.844 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35304 2024-09-17T22:41:29.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.844 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.522020+0000", 2024-09-17T22:41:29.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.522020+0000", 2024-09-17T22:41:29.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.522020+0000", 2024-09-17T22:41:29.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7399999999999999e-07 2024-09-17T22:41:29.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.522022+0000", 2024-09-17T22:41:29.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.148e-06 2024-09-17T22:41:29.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.522030+0000", 2024-09-17T22:41:29.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7589999999999997e-06 2024-09-17T22:41:29.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.522035+0000", 2024-09-17T22:41:29.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9599999999999999e-06 2024-09-17T22:41:29.846 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.846 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.846 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.846 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35622 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.846 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.536786+0000", 2024-09-17T22:41:29.846 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0045868050000000002, 2024-09-17T22:41:29.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0094960389999999995, 2024-09-17T22:41:29.846 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.846 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.846 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.847 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.847 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35622 2024-09-17T22:41:29.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.847 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536786+0000", 2024-09-17T22:41:29.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536786+0000", 2024-09-17T22:41:29.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536786+0000", 2024-09-17T22:41:29.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:41:29.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536793+0000", 2024-09-17T22:41:29.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3090000000000001e-06 2024-09-17T22:41:29.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536800+0000", 2024-09-17T22:41:29.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.559e-06 2024-09-17T22:41:29.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536805+0000", 2024-09-17T22:41:29.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3419999999999998e-06 2024-09-17T22:41:29.849 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.849 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.849 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.849 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35111 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.849 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521255+0000", 2024-09-17T22:41:29.849 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020117368, 2024-09-17T22:41:29.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025076184000000001, 2024-09-17T22:41:29.849 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.849 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.850 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.850 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.850 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.850 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.850 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35111 2024-09-17T22:41:29.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.850 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521255+0000", 2024-09-17T22:41:29.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521255+0000", 2024-09-17T22:41:29.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521256+0000", 2024-09-17T22:41:29.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6100000000000001e-07 2024-09-17T22:41:29.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521258+0000", 2024-09-17T22:41:29.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9120000000000001e-06 2024-09-17T22:41:29.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521264+0000", 2024-09-17T22:41:29.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0589999999999999e-06 2024-09-17T22:41:29.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521268+0000", 2024-09-17T22:41:29.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.639e-06 2024-09-17T22:41:29.852 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.852 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.852 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.852 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35317 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.852 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.523059+0000", 2024-09-17T22:41:29.852 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018313697, 2024-09-17T22:41:29.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023309631000000001, 2024-09-17T22:41:29.853 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.853 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.853 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.853 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.853 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.853 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.853 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35317 2024-09-17T22:41:29.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.853 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523059+0000", 2024-09-17T22:41:29.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523059+0000", 2024-09-17T22:41:29.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523060+0000", 2024-09-17T22:41:29.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1299999999999999e-07 2024-09-17T22:41:29.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523064+0000", 2024-09-17T22:41:29.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0489999999999997e-06 2024-09-17T22:41:29.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523074+0000", 2024-09-17T22:41:29.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0319999999999999e-05 2024-09-17T22:41:29.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523081+0000", 2024-09-17T22:41:29.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0790000000000003e-06 2024-09-17T22:41:29.855 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.855 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.855 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.855 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35623 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.856 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.536812+0000", 2024-09-17T22:41:29.856 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0045601419999999997, 2024-09-17T22:41:29.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0096062070000000003, 2024-09-17T22:41:29.856 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.856 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.856 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.856 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.856 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.856 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.856 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35623 2024-09-17T22:41:29.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.856 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536812+0000", 2024-09-17T22:41:29.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536812+0000", 2024-09-17T22:41:29.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536813+0000", 2024-09-17T22:41:29.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6199999999999998e-07 2024-09-17T22:41:29.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536815+0000", 2024-09-17T22:41:29.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1500000000000002e-06 2024-09-17T22:41:29.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536823+0000", 2024-09-17T22:41:29.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360000000000004e-06 2024-09-17T22:41:29.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536827+0000", 2024-09-17T22:41:29.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8130000000000003e-06 2024-09-17T22:41:29.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.858 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.859 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35125 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.859 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521275+0000", 2024-09-17T22:41:29.859 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020097833999999998, 2024-09-17T22:41:29.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025194197000000002, 2024-09-17T22:41:29.859 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.859 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.859 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.859 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35125 2024-09-17T22:41:29.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.859 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521275+0000", 2024-09-17T22:41:29.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521275+0000", 2024-09-17T22:41:29.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521275+0000", 2024-09-17T22:41:29.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6899999999999998e-07 2024-09-17T22:41:29.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521277+0000", 2024-09-17T22:41:29.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0389999999999999e-06 2024-09-17T22:41:29.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521284+0000", 2024-09-17T22:41:29.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4289999999999998e-06 2024-09-17T22:41:29.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521289+0000", 2024-09-17T22:41:29.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0180000000000001e-06 2024-09-17T22:41:29.862 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.862 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.862 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.862 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35318 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.862 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.523094+0000", 2024-09-17T22:41:29.862 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018278566, 2024-09-17T22:41:29.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023431982000000001, 2024-09-17T22:41:29.863 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.863 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.863 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.863 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.863 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.863 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.863 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35318 2024-09-17T22:41:29.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.863 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523094+0000", 2024-09-17T22:41:29.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523094+0000", 2024-09-17T22:41:29.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523095+0000", 2024-09-17T22:41:29.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:41:29.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523097+0000", 2024-09-17T22:41:29.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3499999999999999e-06 2024-09-17T22:41:29.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523105+0000", 2024-09-17T22:41:29.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8970000000000008e-06 2024-09-17T22:41:29.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523110+0000", 2024-09-17T22:41:29.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4840000000000003e-06 2024-09-17T22:41:29.865 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.865 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.865 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.866 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35636 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.866 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.536834+0000", 2024-09-17T22:41:29.866 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0045386360000000004, 2024-09-17T22:41:29.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0097458679999999995, 2024-09-17T22:41:29.866 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.866 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.866 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.866 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35636 2024-09-17T22:41:29.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.866 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536834+0000", 2024-09-17T22:41:29.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536834+0000", 2024-09-17T22:41:29.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536834+0000", 2024-09-17T22:41:29.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:41:29.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536836+0000", 2024-09-17T22:41:29.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1040000000000001e-06 2024-09-17T22:41:29.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536843+0000", 2024-09-17T22:41:29.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1940000000000003e-06 2024-09-17T22:41:29.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536847+0000", 2024-09-17T22:41:29.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5870000000000003e-06 2024-09-17T22:41:29.869 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.869 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.869 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.869 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35139 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.869 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521295+0000", 2024-09-17T22:41:29.869 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020077727, 2024-09-17T22:41:29.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025337374999999999, 2024-09-17T22:41:29.869 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.870 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.870 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.870 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.870 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.870 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.870 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35139 2024-09-17T22:41:29.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.870 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521295+0000", 2024-09-17T22:41:29.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521295+0000", 2024-09-17T22:41:29.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521295+0000", 2024-09-17T22:41:29.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:41:29.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521297+0000", 2024-09-17T22:41:29.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9910000000000001e-06 2024-09-17T22:41:29.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521304+0000", 2024-09-17T22:41:29.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3130000000000004e-06 2024-09-17T22:41:29.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521309+0000", 2024-09-17T22:41:29.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0490000000000003e-06 2024-09-17T22:41:29.872 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.872 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.872 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.873 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35331 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.873 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.523119+0000", 2024-09-17T22:41:29.873 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018254040999999999, 2024-09-17T22:41:29.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023564081000000001, 2024-09-17T22:41:29.873 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.873 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.873 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.873 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.873 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.873 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.873 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35331 2024-09-17T22:41:29.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.873 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523119+0000", 2024-09-17T22:41:29.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523119+0000", 2024-09-17T22:41:29.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523119+0000", 2024-09-17T22:41:29.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6699999999999999e-07 2024-09-17T22:41:29.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523121+0000", 2024-09-17T22:41:29.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.199e-06 2024-09-17T22:41:29.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523129+0000", 2024-09-17T22:41:29.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3869999999999997e-06 2024-09-17T22:41:29.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523134+0000", 2024-09-17T22:41:29.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0699999999999997e-06 2024-09-17T22:41:29.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.875 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.876 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.876 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35637 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.876 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.536854+0000", 2024-09-17T22:41:29.876 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.004518996, 2024-09-17T22:41:29.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0098800799999999994, 2024-09-17T22:41:29.876 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.876 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.876 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.876 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35637 2024-09-17T22:41:29.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.877 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536854+0000", 2024-09-17T22:41:29.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536854+0000", 2024-09-17T22:41:29.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536854+0000", 2024-09-17T22:41:29.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7099999999999997e-07 2024-09-17T22:41:29.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536856+0000", 2024-09-17T22:41:29.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1780000000000002e-06 2024-09-17T22:41:29.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536864+0000", 2024-09-17T22:41:29.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4159999999999998e-06 2024-09-17T22:41:29.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536869+0000", 2024-09-17T22:41:29.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9760000000000003e-06 2024-09-17T22:41:29.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.879 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.879 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35140 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.879 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521315+0000", 2024-09-17T22:41:29.879 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020057512999999999, 2024-09-17T22:41:29.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025474310999999999, 2024-09-17T22:41:29.879 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.879 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.880 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.880 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35140 2024-09-17T22:41:29.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.880 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521315+0000", 2024-09-17T22:41:29.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521315+0000", 2024-09-17T22:41:29.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521316+0000", 2024-09-17T22:41:29.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8700000000000001e-07 2024-09-17T22:41:29.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521322+0000", 2024-09-17T22:41:29.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.567e-06 2024-09-17T22:41:29.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521329+0000", 2024-09-17T22:41:29.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9550000000000003e-06 2024-09-17T22:41:29.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521334+0000", 2024-09-17T22:41:29.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.215e-06 2024-09-17T22:41:29.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.882 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.882 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35332 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.883 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.523140+0000", 2024-09-17T22:41:29.883 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018232155999999999, 2024-09-17T22:41:29.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023711137, 2024-09-17T22:41:29.883 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.883 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.883 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.883 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35332 2024-09-17T22:41:29.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.883 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523140+0000", 2024-09-17T22:41:29.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523140+0000", 2024-09-17T22:41:29.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523141+0000", 2024-09-17T22:41:29.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1600000000000002e-07 2024-09-17T22:41:29.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523143+0000", 2024-09-17T22:41:29.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0449999999999999e-06 2024-09-17T22:41:29.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523150+0000", 2024-09-17T22:41:29.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2570000000000001e-06 2024-09-17T22:41:29.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523155+0000", 2024-09-17T22:41:29.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8110000000000002e-06 2024-09-17T22:41:29.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.886 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.886 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35654 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.886 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.536874+0000", 2024-09-17T22:41:29.886 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0044986239999999997, 2024-09-17T22:41:29.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01002928, 2024-09-17T22:41:29.886 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.886 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.886 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.886 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.887 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.887 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.887 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35654 2024-09-17T22:41:29.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.887 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536874+0000", 2024-09-17T22:41:29.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536874+0000", 2024-09-17T22:41:29.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536874+0000", 2024-09-17T22:41:29.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.39e-07 2024-09-17T22:41:29.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536876+0000", 2024-09-17T22:41:29.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.114e-06 2024-09-17T22:41:29.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536882+0000", 2024-09-17T22:41:29.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7470000000000002e-06 2024-09-17T22:41:29.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536887+0000", 2024-09-17T22:41:29.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6120000000000001e-06 2024-09-17T22:41:29.889 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.889 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.889 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.889 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35143 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.890 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521341+0000", 2024-09-17T22:41:29.890 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020031469999999999, 2024-09-17T22:41:29.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025626224999999999, 2024-09-17T22:41:29.890 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.890 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.890 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.890 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35143 2024-09-17T22:41:29.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.890 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521341+0000", 2024-09-17T22:41:29.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521341+0000", 2024-09-17T22:41:29.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521342+0000", 2024-09-17T22:41:29.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.65e-07 2024-09-17T22:41:29.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521344+0000", 2024-09-17T22:41:29.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0080000000000001e-06 2024-09-17T22:41:29.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521350+0000", 2024-09-17T22:41:29.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3620000000000002e-06 2024-09-17T22:41:29.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521354+0000", 2024-09-17T22:41:29.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4179999999999999e-06 2024-09-17T22:41:29.893 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.893 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.893 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.893 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35346 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.893 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.523165+0000", 2024-09-17T22:41:29.893 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018207336000000001, 2024-09-17T22:41:29.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023854424999999999, 2024-09-17T22:41:29.893 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.893 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.893 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.894 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.894 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.894 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35346 2024-09-17T22:41:29.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.894 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523165+0000", 2024-09-17T22:41:29.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523165+0000", 2024-09-17T22:41:29.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523166+0000", 2024-09-17T22:41:29.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8599999999999998e-07 2024-09-17T22:41:29.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523168+0000", 2024-09-17T22:41:29.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2089999999999999e-06 2024-09-17T22:41:29.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523175+0000", 2024-09-17T22:41:29.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.922e-06 2024-09-17T22:41:29.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523180+0000", 2024-09-17T22:41:29.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9400000000000001e-06 2024-09-17T22:41:29.896 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.896 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.896 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.896 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35655 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.896 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.536893+0000", 2024-09-17T22:41:29.896 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0044792210000000002, 2024-09-17T22:41:29.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010194692999999999, 2024-09-17T22:41:29.897 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.897 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.897 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.899 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.899 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35655 2024-09-17T22:41:29.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.899 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536893+0000", 2024-09-17T22:41:29.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536893+0000", 2024-09-17T22:41:29.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536894+0000", 2024-09-17T22:41:29.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1899999999999998e-07 2024-09-17T22:41:29.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536896+0000", 2024-09-17T22:41:29.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.956e-06 2024-09-17T22:41:29.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536902+0000", 2024-09-17T22:41:29.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3040000000000001e-06 2024-09-17T22:41:29.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.536906+0000", 2024-09-17T22:41:29.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2370000000000003e-06 2024-09-17T22:41:29.901 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.901 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.901 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.901 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35157 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.901 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521361+0000", 2024-09-17T22:41:29.901 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020012124999999999, 2024-09-17T22:41:29.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025785184999999999, 2024-09-17T22:41:29.902 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.902 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.902 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.902 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35157 2024-09-17T22:41:29.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.902 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521361+0000", 2024-09-17T22:41:29.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521361+0000", 2024-09-17T22:41:29.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521361+0000", 2024-09-17T22:41:29.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.77e-07 2024-09-17T22:41:29.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521363+0000", 2024-09-17T22:41:29.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8929999999999999e-06 2024-09-17T22:41:29.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521369+0000", 2024-09-17T22:41:29.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1410000000000002e-06 2024-09-17T22:41:29.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521374+0000", 2024-09-17T22:41:29.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9629999999999997e-06 2024-09-17T22:41:29.904 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.904 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.905 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35368 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.905 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.523229+0000", 2024-09-17T22:41:29.905 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018144029999999998, 2024-09-17T22:41:29.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023981189999999999, 2024-09-17T22:41:29.905 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.905 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.905 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.905 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.905 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.906 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.906 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35368 2024-09-17T22:41:29.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.906 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523229+0000", 2024-09-17T22:41:29.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523229+0000", 2024-09-17T22:41:29.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523229+0000", 2024-09-17T22:41:29.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.13e-07 2024-09-17T22:41:29.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523232+0000", 2024-09-17T22:41:29.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.117e-06 2024-09-17T22:41:29.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523242+0000", 2024-09-17T22:41:29.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6339999999999996e-06 2024-09-17T22:41:29.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523248+0000", 2024-09-17T22:41:29.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7780000000000004e-06 2024-09-17T22:41:29.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.908 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.908 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35668 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.908 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540034+0000", 2024-09-17T22:41:29.908 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001338491, 2024-09-17T22:41:29.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072261679999999998, 2024-09-17T22:41:29.908 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.909 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.909 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.909 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35668 2024-09-17T22:41:29.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.909 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540034+0000", 2024-09-17T22:41:29.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540034+0000", 2024-09-17T22:41:29.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540035+0000", 2024-09-17T22:41:29.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4900000000000005e-07 2024-09-17T22:41:29.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540040+0000", 2024-09-17T22:41:29.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9400000000000001e-06 2024-09-17T22:41:29.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540062+0000", 2024-09-17T22:41:29.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1872e-05 2024-09-17T22:41:29.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540069+0000", 2024-09-17T22:41:29.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3139999999999998e-06 2024-09-17T22:41:29.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.911 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.911 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35158 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.911 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521380+0000", 2024-09-17T22:41:29.912 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019992395, 2024-09-17T22:41:29.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025932598000000001, 2024-09-17T22:41:29.912 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.912 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.912 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.912 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35158 2024-09-17T22:41:29.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.912 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521380+0000", 2024-09-17T22:41:29.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521380+0000", 2024-09-17T22:41:29.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521381+0000", 2024-09-17T22:41:29.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000002e-07 2024-09-17T22:41:29.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521385+0000", 2024-09-17T22:41:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-09-17T22:41:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521392+0000", 2024-09-17T22:41:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5930000000000002e-06 2024-09-17T22:41:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521397+0000", 2024-09-17T22:41:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9599999999999999e-06 2024-09-17T22:41:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35369 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.523257+0000", 2024-09-17T22:41:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018115839000000002, 2024-09-17T22:41:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024112855999999998, 2024-09-17T22:41:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35369 2024-09-17T22:41:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523257+0000", 2024-09-17T22:41:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523257+0000", 2024-09-17T22:41:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523257+0000", 2024-09-17T22:41:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4700000000000002e-07 2024-09-17T22:41:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523259+0000", 2024-09-17T22:41:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2249999999999999e-06 2024-09-17T22:41:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523267+0000", 2024-09-17T22:41:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6680000000000001e-06 2024-09-17T22:41:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.523272+0000", 2024-09-17T22:41:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9869999999999999e-06 2024-09-17T22:41:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35669 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540083+0000", 2024-09-17T22:41:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0012892890000000001, 2024-09-17T22:41:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073372530000000002, 2024-09-17T22:41:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35669 2024-09-17T22:41:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540083+0000", 2024-09-17T22:41:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540083+0000", 2024-09-17T22:41:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540084+0000", 2024-09-17T22:41:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5799999999999999e-07 2024-09-17T22:41:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540087+0000", 2024-09-17T22:41:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7180000000000001e-06 2024-09-17T22:41:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540094+0000", 2024-09-17T22:41:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2869999999999999e-06 2024-09-17T22:41:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540099+0000", 2024-09-17T22:41:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2360000000000004e-06 2024-09-17T22:41:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35171 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521403+0000", 2024-09-17T22:41:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019969321000000002, 2024-09-17T22:41:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026068133, 2024-09-17T22:41:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.923 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35171 2024-09-17T22:41:29.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.923 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521403+0000", 2024-09-17T22:41:29.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521403+0000", 2024-09-17T22:41:29.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521404+0000", 2024-09-17T22:41:29.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:41:29.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521405+0000", 2024-09-17T22:41:29.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.725e-06 2024-09-17T22:41:29.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521411+0000", 2024-09-17T22:41:29.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0140000000000004e-06 2024-09-17T22:41:29.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521416+0000", 2024-09-17T22:41:29.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6439999999999999e-06 2024-09-17T22:41:29.925 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.925 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.925 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.925 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35389 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.926 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.524070+0000", 2024-09-17T22:41:29.926 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.017303123, 2024-09-17T22:41:29.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023442357, 2024-09-17T22:41:29.926 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.926 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.926 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.926 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.926 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.926 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.926 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35389 2024-09-17T22:41:29.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.926 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.524070+0000", 2024-09-17T22:41:29.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.524070+0000", 2024-09-17T22:41:29.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.524070+0000", 2024-09-17T22:41:29.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2600000000000001e-07 2024-09-17T22:41:29.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.524074+0000", 2024-09-17T22:41:29.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8920000000000002e-06 2024-09-17T22:41:29.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.524084+0000", 2024-09-17T22:41:29.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8030000000000008e-06 2024-09-17T22:41:29.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.524091+0000", 2024-09-17T22:41:29.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8109999999999997e-06 2024-09-17T22:41:29.929 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.929 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.929 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.929 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35686 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.929 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540107+0000", 2024-09-17T22:41:29.929 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001266133, 2024-09-17T22:41:29.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0074319670000000003, 2024-09-17T22:41:29.929 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.929 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.929 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.930 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.930 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.930 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.930 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35686 2024-09-17T22:41:29.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.930 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540107+0000", 2024-09-17T22:41:29.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540107+0000", 2024-09-17T22:41:29.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540107+0000", 2024-09-17T22:41:29.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:41:29.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540109+0000", 2024-09-17T22:41:29.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1610000000000001e-06 2024-09-17T22:41:29.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540115+0000", 2024-09-17T22:41:29.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1920000000000002e-06 2024-09-17T22:41:29.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540120+0000", 2024-09-17T22:41:29.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7589999999999997e-06 2024-09-17T22:41:29.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.932 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.932 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35172 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.932 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.521422+0000", 2024-09-17T22:41:29.933 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019950147000000001, 2024-09-17T22:41:29.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026167319000000001, 2024-09-17T22:41:29.933 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.933 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.933 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.933 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35172 2024-09-17T22:41:29.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.933 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521422+0000", 2024-09-17T22:41:29.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521422+0000", 2024-09-17T22:41:29.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521423+0000", 2024-09-17T22:41:29.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4499999999999998e-07 2024-09-17T22:41:29.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521425+0000", 2024-09-17T22:41:29.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9269999999999999e-06 2024-09-17T22:41:29.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521431+0000", 2024-09-17T22:41:29.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1940000000000003e-06 2024-09-17T22:41:29.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.521436+0000", 2024-09-17T22:41:29.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1359999999999996e-06 2024-09-17T22:41:29.935 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.935 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.936 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.936 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35390 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.936 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.524104+0000", 2024-09-17T22:41:29.936 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.017269018000000001, 2024-09-17T22:41:29.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02353713, 2024-09-17T22:41:29.936 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.936 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.936 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.936 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35390 2024-09-17T22:41:29.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.937 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.524104+0000", 2024-09-17T22:41:29.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.524104+0000", 2024-09-17T22:41:29.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.524104+0000", 2024-09-17T22:41:29.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:41:29.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.524107+0000", 2024-09-17T22:41:29.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.633e-06 2024-09-17T22:41:29.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.524115+0000", 2024-09-17T22:41:29.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5010000000000004e-06 2024-09-17T22:41:29.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.524121+0000", 2024-09-17T22:41:29.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7540000000000003e-06 2024-09-17T22:41:29.939 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.939 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.939 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.939 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:35687 3.a 3.7a329d4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:29.939 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.540126+0000", 2024-09-17T22:41:29.939 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001246629, 2024-09-17T22:41:29.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0075663170000000004, 2024-09-17T22:41:29.939 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:29.939 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:29.939 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:29.939 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:29.939 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:29.939 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:29.940 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 35687 2024-09-17T22:41:29.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.940 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:29.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:29.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540126+0000", 2024-09-17T22:41:29.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:29.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540126+0000", 2024-09-17T22:41:29.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:29.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:29.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540126+0000", 2024-09-17T22:41:29.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:41:29.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:29.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540129+0000", 2024-09-17T22:41:29.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1399999999999998e-06 2024-09-17T22:41:29.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:29.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540134+0000", 2024-09-17T22:41:29.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9200000000000001e-06 2024-09-17T22:41:29.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:29.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:29.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:29.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.540139+0000", 2024-09-17T22:41:29.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-17T22:41:29.942 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.942 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:29.942 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.942 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:29.942 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:41:29.942 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 115 2024-09-17T22:41:29.942 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:29.943 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:41:29.945 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:29.945 INFO:teuthology.orchestra.run.smithi115.stdout:299 write 0xc7ba304 thru 0xc7c87ff (0xe4fc bytes) 2024-09-17T22:41:29.945 INFO:teuthology.orchestra.run.smithi115.stdout:300 write 0xbb9aa6e thru 0xbba01cd (0x5760 bytes) 2024-09-17T22:41:29.945 INFO:teuthology.orchestra.run.smithi115.stdout:302 read 0x4489235 thru 0x4498eb9 (0xfc85 bytes) 2024-09-17T22:41:29.945 INFO:teuthology.orchestra.run.smithi115.stdout:303 writesame 0x31f877d thru 0x31fe9f0 data_size 0x2(0x6274 bytes) 2024-09-17T22:41:29.945 INFO:teuthology.orchestra.run.smithi115.stdout:305 read 0x2669f1a thru 0x267154f (0x7636 bytes) 2024-09-17T22:41:29.945 INFO:teuthology.orchestra.run.smithi115.stdout:306 punch from 0x2b27b3d to 0x2b27d0f, (0x1d2 bytes) 2024-09-17T22:41:29.945 INFO:teuthology.orchestra.run.smithi115.stdout:309 writesame 0x950d99a thru 0x9511471 data_size 0x1c(0x3ad8 bytes) 2024-09-17T22:41:29.945 INFO:teuthology.orchestra.run.smithi115.stdout:310 writesame 0xd98c746 thru 0xd9973ee data_size 0x1(0xaca9 bytes) 2024-09-17T22:41:29.945 INFO:teuthology.orchestra.run.smithi115.stdout:311 read 0x95647b thru 0x95ceba (0x6a40 bytes) 2024-09-17T22:41:29.946 INFO:teuthology.orchestra.run.smithi115.stdout:312 write 0x46b2076 thru 0x46b83b3 (0x633e bytes) 2024-09-17T22:41:29.946 INFO:teuthology.orchestra.run.smithi115.stdout:313 writesame 0x3fe66d3 thru 0x3ff37b4 data_size 0x1(0xd0e2 bytes) 2024-09-17T22:41:29.946 INFO:teuthology.orchestra.run.smithi115.stdout:314 write 0xbbff75c thru 0xbc09d65 (0xa60a bytes) 2024-09-17T22:41:29.946 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:29.946 INFO:teuthology.orchestra.run.smithi115.stdout:317 read 0x1660287 thru 0x1664c69 (0x49e3 bytes) 2024-09-17T22:41:29.946 INFO:teuthology.orchestra.run.smithi115.stdout:318 writesame 0xd71f thru 0x196a7 data_size 0x1(0xbf89 bytes) 2024-09-17T22:41:29.946 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:29.946 INFO:teuthology.orchestra.run.smithi115.stdout:320 write 0x8dea2b0 thru 0x8df9b8e (0xf8df bytes) 2024-09-17T22:41:29.946 INFO:teuthology.orchestra.run.smithi115.stdout:321 read 0xa9173e7 thru 0xa92518e (0xdda8 bytes) 2024-09-17T22:41:29.946 INFO:teuthology.orchestra.run.smithi115.stdout:322 write 0x475f4fb thru 0x476261a (0x3120 bytes) 2024-09-17T22:41:29.946 INFO:teuthology.orchestra.run.smithi115.stdout:truncating to largest ever: 0xe681169 2024-09-17T22:41:29.946 INFO:teuthology.orchestra.run.smithi115.stdout:323 trunc from 0xd9973ef to 0xe681169 2024-09-17T22:41:29.947 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:29.895+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:29.947 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:29.947 INFO:teuthology.orchestra.run.smithi115.stdout:326 read 0xc3f78cd thru 0xc3fa85f (0x2f93 bytes) 2024-09-17T22:41:29.948 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:29.948 INFO:teuthology.orchestra.run.smithi115.stdout:328 read 0x6548119 thru 0x6554fce (0xceb6 bytes) 2024-09-17T22:41:29.948 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:29.948 INFO:teuthology.orchestra.run.smithi115.stdout:333 trunc from 0xe681169 to 0xba0e292 2024-09-17T22:41:29.948 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:29.948 INFO:teuthology.orchestra.run.smithi115.stdout:335 writesame 0x60336f7 thru 0x6039625 data_size 0x7(0x5f2f bytes) 2024-09-17T22:41:29.948 INFO:teuthology.orchestra.run.smithi115.stdout:336 write 0xe8bbd32 thru 0xe8c470b (0x89da bytes) 2024-09-17T22:41:29.949 INFO:teuthology.orchestra.run.smithi115.stdout:337 punch from 0x6c3274 to 0x6ccb95, (0x9921 bytes) 2024-09-17T22:41:29.950 INFO:teuthology.orchestra.run.smithi115.stdout:338 flatten 2024-09-17T22:41:29.999 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:29.996+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:30.099 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:30.097+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:30.118 INFO:teuthology.orchestra.run.smithi115.stdout:340 read 0xb5de41 thru 0xb60dc8 (0x2f88 bytes) 2024-09-17T22:41:30.119 INFO:teuthology.orchestra.run.smithi115.stdout:342 writesame 0x2fd9c6a thru 0x2fddbed data_size 0xa(0x3f84 bytes) 2024-09-17T22:41:30.131 INFO:teuthology.orchestra.run.smithi115.stdout:344 writesame 0xee1c08f thru 0xee1fd7d data_size 0x1(0x3cef bytes) 2024-09-17T22:41:30.133 INFO:teuthology.orchestra.run.smithi115.stdout:345 write 0x5cd8b42 thru 0x5ce5164 (0xc623 bytes) 2024-09-17T22:41:30.138 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:30.138 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:30.139 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:41:30.139 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:41:30.139 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:30.143 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:30.143 INFO:teuthology.orchestra.run.smithi115.stdout:347 read 0x3c1e783 thru 0x3c2c478 (0xdcf6 bytes) 2024-09-17T22:41:30.143 INFO:teuthology.orchestra.run.smithi115.stdout:349 punch from 0x79853ed to 0x7986a97, (0x16aa bytes) 2024-09-17T22:41:30.143 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:30.144 INFO:teuthology.orchestra.run.smithi115.stdout:353 read 0x1d98d5f thru 0x1da80c0 (0xf362 bytes) 2024-09-17T22:41:30.144 INFO:teuthology.orchestra.run.smithi115.stdout:354 read 0x341d786 thru 0x3428f46 (0xb7c1 bytes) 2024-09-17T22:41:30.144 INFO:teuthology.orchestra.run.smithi115.stdout:355 writesame 0xfc334c thru 0xfca4db data_size 0x5c(0x7190 bytes) 2024-09-17T22:41:30.148 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:41:30.149 INFO:teuthology.orchestra.run.smithi115.stdout:356 read 0x911ffc thru 0x913f6b (0x1f70 bytes) 2024-09-17T22:41:30.150 INFO:teuthology.orchestra.run.smithi115.stdout:357 read 0xc04a90b thru 0xc04d9dc (0x30d2 bytes) 2024-09-17T22:41:30.150 INFO:teuthology.orchestra.run.smithi115.stdout:359 read 0xc320c37 thru 0xc323472 (0x283c bytes) 2024-09-17T22:41:30.150 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:30.150 INFO:teuthology.orchestra.run.smithi115.stdout:361 write 0x2c7fe89 thru 0x2c8cf6c (0xd0e4 bytes) 2024-09-17T22:41:30.156 INFO:teuthology.orchestra.run.smithi115.stdout:362 writesame 0xa5fdb63 thru 0xa603e89 data_size 0x1(0x6327 bytes) 2024-09-17T22:41:30.178 INFO:teuthology.orchestra.run.smithi115.stdout:363 trunc from 0xee1fd7e to 0x4070968 2024-09-17T22:41:30.198 INFO:teuthology.orchestra.run.smithi115.stdout:364 punch from 0x941b10 to 0x9481bf, (0x66af bytes) 2024-09-17T22:41:30.198 INFO:teuthology.orchestra.run.smithi115.stdout:366 read 0x18ae65c thru 0x18ba543 (0xbee8 bytes) 2024-09-17T22:41:30.199 INFO:teuthology.orchestra.run.smithi115.stdout:368 read 0x1e643a3 thru 0x1e66ea8 (0x2b06 bytes) 2024-09-17T22:41:30.199 INFO:teuthology.orchestra.run.smithi115.stdout:369 punch from 0x19a2c62 to 0x19a4358, (0x16f6 bytes) 2024-09-17T22:41:30.199 INFO:teuthology.orchestra.run.smithi115.stdout:370 read 0x34969bb thru 0x349ea20 (0x8066 bytes) 2024-09-17T22:41:30.199 INFO:teuthology.orchestra.run.smithi115.stdout:371 read 0x164203c thru 0x1644a22 (0x29e7 bytes) 2024-09-17T22:41:30.200 INFO:teuthology.orchestra.run.smithi115.stdout:373 punch from 0x367fb58 to 0x368f155, (0xf5fd bytes) 2024-09-17T22:41:30.200 INFO:teuthology.orchestra.run.smithi115.stdout:375 trunc from 0x4070968 to 0x43030f9 2024-09-17T22:41:30.200 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:30.197+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:30.221 INFO:teuthology.orchestra.run.smithi115.stdout:376 write 0xeafa623 thru 0xeafe06e (0x3a4c bytes) 2024-09-17T22:41:30.225 INFO:teuthology.orchestra.run.smithi115.stdout:377 write 0x4e1027e thru 0x4e1aec1 (0xac44 bytes) 2024-09-17T22:41:30.229 INFO:teuthology.orchestra.run.smithi115.stdout:379 read 0xea87d3a thru 0xea8c8a7 (0x4b6e bytes) 2024-09-17T22:41:30.242 INFO:teuthology.orchestra.run.smithi115.stdout:381 trunc from 0xeafe06f to 0x418eb1c 2024-09-17T22:41:30.267 INFO:teuthology.orchestra.run.smithi115.stdout:382 trunc from 0x418eb1c to 0xe60de3d 2024-09-17T22:41:30.272 INFO:teuthology.orchestra.run.smithi115.stdout:383 write 0x35c86c7 thru 0x35d6be4 (0xe51e bytes) 2024-09-17T22:41:30.273 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:30.273 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:41:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:41:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:41:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4229.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-17T22:41:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:41:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 20.875543048000001, 2024-09-17T22:41:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25708877200000002, 2024-09-17T22:41:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-17T22:41:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.274 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4229", 2024-09-17T22:41:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/3080427038", 2024-09-17T22:41:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2 2024-09-17T22:41:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:41:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:41:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394611+0000", 2024-09-17T22:41:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3149999999999999e-06 2024-09-17T22:41:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394619+0000", 2024-09-17T22:41:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240000000000003e-06 2024-09-17T22:41:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394632+0000", 2024-09-17T22:41:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2921e-05 2024-09-17T22:41:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394639+0000", 2024-09-17T22:41:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-09-17T22:41:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394703+0000", 2024-09-17T22:41:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4407999999999997e-05 2024-09-17T22:41:30.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:41:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394711+0000", 2024-09-17T22:41:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5939999999999996e-06 2024-09-17T22:41:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651545+0000", 2024-09-17T22:41:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25683409200000001 2024-09-17T22:41:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651698+0000", 2024-09-17T22:41:30.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000153678 2024-09-17T22:41:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38736 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:41:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.49294704299999997, 2024-09-17T22:41:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051395418999999998, 2024-09-17T22:41:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:30.279 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38736 2024-09-17T22:41:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:41:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:41:30.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:41:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6899999999999998e-07 2024-09-17T22:41:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777212+0000", 2024-09-17T22:41:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1240000000000002e-06 2024-09-17T22:41:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777219+0000", 2024-09-17T22:41:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5409999999999997e-06 2024-09-17T22:41:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777224+0000", 2024-09-17T22:41:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:41:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.826131+0000", 2024-09-17T22:41:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048907369999999999 2024-09-17T22:41:30.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828457+0000", 2024-09-17T22:41:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023256209999999999 2024-09-17T22:41:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828601+0000", 2024-09-17T22:41:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014438799999999999 2024-09-17T22:41:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38752 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 673792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:41:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.49292297200000001, 2024-09-17T22:41:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051582301999999997, 2024-09-17T22:41:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38752 2024-09-17T22:41:30.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:41:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:41:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:41:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3000000000000001e-07 2024-09-17T22:41:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777232+0000", 2024-09-17T22:41:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9709999999999998e-06 2024-09-17T22:41:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777238+0000", 2024-09-17T22:41:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7209999999999999e-06 2024-09-17T22:41:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777243+0000", 2024-09-17T22:41:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5210000000000004e-06 2024-09-17T22:41:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828633+0000", 2024-09-17T22:41:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051390915000000002 2024-09-17T22:41:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828711+0000", 2024-09-17T22:41:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7474999999999999e-05 2024-09-17T22:41:30.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828812+0000", 2024-09-17T22:41:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010116900000000001 2024-09-17T22:41:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38753 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:41:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.49290369899999997, 2024-09-17T22:41:30.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051770444999999998, 2024-09-17T22:41:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38753 2024-09-17T22:41:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:41:30.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:41:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:41:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:41:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777251+0000", 2024-09-17T22:41:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8059999999999999e-06 2024-09-17T22:41:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777257+0000", 2024-09-17T22:41:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2480000000000001e-06 2024-09-17T22:41:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777262+0000", 2024-09-17T22:41:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000004e-06 2024-09-17T22:41:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828842+0000", 2024-09-17T22:41:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051580203999999998 2024-09-17T22:41:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828917+0000", 2024-09-17T22:41:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4980000000000004e-05 2024-09-17T22:41:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829019+0000", 2024-09-17T22:41:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102248 2024-09-17T22:41:30.292 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38766 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:41:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.492884714, 2024-09-17T22:41:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051958839999999999, 2024-09-17T22:41:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:30.293 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38766 2024-09-17T22:41:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:41:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:41:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777269+0000", 2024-09-17T22:41:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:41:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777271+0000", 2024-09-17T22:41:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-17T22:41:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777276+0000", 2024-09-17T22:41:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:41:30.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777281+0000", 2024-09-17T22:41:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:41:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829055+0000", 2024-09-17T22:41:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051773570999999997 2024-09-17T22:41:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:30.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829128+0000", 2024-09-17T22:41:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3131000000000002e-05 2024-09-17T22:41:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829227+0000", 2024-09-17T22:41:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8800000000000003e-05 2024-09-17T22:41:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38767 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.297 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:41:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.492864731, 2024-09-17T22:41:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137018, 2024-09-17T22:41:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38767 2024-09-17T22:41:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:41:30.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:41:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:41:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:41:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777290+0000", 2024-09-17T22:41:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.88e-06 2024-09-17T22:41:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777296+0000", 2024-09-17T22:41:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-09-17T22:41:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777300+0000", 2024-09-17T22:41:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4510000000000002e-06 2024-09-17T22:41:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829257+0000", 2024-09-17T22:41:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051956975000000002 2024-09-17T22:41:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829328+0000", 2024-09-17T22:41:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1026999999999996e-05 2024-09-17T22:41:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829425+0000", 2024-09-17T22:41:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.679e-05 2024-09-17T22:41:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.301 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38784 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 675840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:41:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.492846277, 2024-09-17T22:41:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052344522999999997, 2024-09-17T22:41:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.302 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38784 2024-09-17T22:41:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:41:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:41:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777307+0000", 2024-09-17T22:41:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:41:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777309+0000", 2024-09-17T22:41:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.063e-06 2024-09-17T22:41:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777314+0000", 2024-09-17T22:41:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-17T22:41:30.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777319+0000", 2024-09-17T22:41:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999996e-06 2024-09-17T22:41:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829456+0000", 2024-09-17T22:41:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137151 2024-09-17T22:41:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:30.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829527+0000", 2024-09-17T22:41:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1588000000000006e-05 2024-09-17T22:41:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829651+0000", 2024-09-17T22:41:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012365 2024-09-17T22:41:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38785 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 749568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:41:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.49282816200000001, 2024-09-17T22:41:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052546572999999999, 2024-09-17T22:41:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38785 2024-09-17T22:41:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.307 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:41:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:41:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:41:30.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:41:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777327+0000", 2024-09-17T22:41:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:41:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777333+0000", 2024-09-17T22:41:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7309999999999998e-06 2024-09-17T22:41:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777337+0000", 2024-09-17T22:41:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:41:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829686+0000", 2024-09-17T22:41:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052349159999999999 2024-09-17T22:41:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829766+0000", 2024-09-17T22:41:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9326999999999995e-05 2024-09-17T22:41:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829871+0000", 2024-09-17T22:41:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000105466 2024-09-17T22:41:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38798 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:41:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.49219455400000001, 2024-09-17T22:41:30.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052117564999999998, 2024-09-17T22:41:30.312 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.312 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.312 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:30.312 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.312 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:30.312 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.312 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38798 2024-09-17T22:41:30.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.312 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:41:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:41:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777959+0000", 2024-09-17T22:41:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9599999999999999e-07 2024-09-17T22:41:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777963+0000", 2024-09-17T22:41:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2660000000000003e-06 2024-09-17T22:41:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777971+0000", 2024-09-17T22:41:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3489999999999992e-06 2024-09-17T22:41:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777978+0000", 2024-09-17T22:41:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:41:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829906+0000", 2024-09-17T22:41:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051927635999999999 2024-09-17T22:41:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829986+0000", 2024-09-17T22:41:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0444000000000004e-05 2024-09-17T22:41:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830076+0000", 2024-09-17T22:41:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9473000000000002e-05 2024-09-17T22:41:30.315 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38799 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 882688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:41:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.49216311499999998, 2024-09-17T22:41:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052359606000000003, 2024-09-17T22:41:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:30.316 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38799 2024-09-17T22:41:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:41:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:41:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:41:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8299999999999997e-07 2024-09-17T22:41:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777992+0000", 2024-09-17T22:41:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.176e-06 2024-09-17T22:41:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777998+0000", 2024-09-17T22:41:30.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.173e-06 2024-09-17T22:41:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778003+0000", 2024-09-17T22:41:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6349999999999997e-06 2024-09-17T22:41:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830112+0000", 2024-09-17T22:41:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052109366999999997 2024-09-17T22:41:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:30.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830192+0000", 2024-09-17T22:41:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9814000000000005e-05 2024-09-17T22:41:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830349+0000", 2024-09-17T22:41:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015695800000000001 2024-09-17T22:41:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38814 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.320 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:41:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.491879711, 2024-09-17T22:41:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052274339000000003, 2024-09-17T22:41:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38814 2024-09-17T22:41:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:41:30.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:41:30.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:41:30.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:41:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778276+0000", 2024-09-17T22:41:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2029999999999999e-06 2024-09-17T22:41:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778282+0000", 2024-09-17T22:41:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0059999999999998e-06 2024-09-17T22:41:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778287+0000", 2024-09-17T22:41:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0039999999999999e-06 2024-09-17T22:41:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830382+0000", 2024-09-17T22:41:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052095379999999997 2024-09-17T22:41:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830455+0000", 2024-09-17T22:41:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3127000000000006e-05 2024-09-17T22:41:30.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830547+0000", 2024-09-17T22:41:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2207999999999995e-05 2024-09-17T22:41:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38828 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 163840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:41:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.49169730299999997, 2024-09-17T22:41:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052466039999999999, 2024-09-17T22:41:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38828 2024-09-17T22:41:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.326 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:41:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:41:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778456+0000", 2024-09-17T22:41:30.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7599999999999996e-07 2024-09-17T22:41:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778460+0000", 2024-09-17T22:41:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8229999999999997e-06 2024-09-17T22:41:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778470+0000", 2024-09-17T22:41:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8199999999999992e-06 2024-09-17T22:41:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778476+0000", 2024-09-17T22:41:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2589999999999997e-06 2024-09-17T22:41:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830578+0000", 2024-09-17T22:41:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052102273999999997 2024-09-17T22:41:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830667+0000", 2024-09-17T22:41:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8701000000000003e-05 2024-09-17T22:41:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830921+0000", 2024-09-17T22:41:30.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025438700000000002 2024-09-17T22:41:30.330 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.330 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.330 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.330 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38829 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.330 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:41:30.330 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.49166590599999999, 2024-09-17T22:41:30.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052648882000000001, 2024-09-17T22:41:30.330 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.330 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.330 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:30.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.331 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:30.331 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.331 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38829 2024-09-17T22:41:30.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.331 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:41:30.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:41:30.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:41:30.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7e-07 2024-09-17T22:41:30.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778490+0000", 2024-09-17T22:41:30.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:41:30.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778495+0000", 2024-09-17T22:41:30.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9329999999999998e-06 2024-09-17T22:41:30.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778500+0000", 2024-09-17T22:41:30.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9509999999999997e-06 2024-09-17T22:41:30.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830958+0000", 2024-09-17T22:41:30.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05245785 2024-09-17T22:41:30.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:30.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831051+0000", 2024-09-17T22:41:30.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2989e-05 2024-09-17T22:41:30.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831136+0000", 2024-09-17T22:41:30.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4448000000000002e-05 2024-09-17T22:41:30.334 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.334 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.334 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.335 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38891 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.335 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:41:30.335 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.48835384799999998, 2024-09-17T22:41:30.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050847934999999997, 2024-09-17T22:41:30.335 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.335 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.335 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:30.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:30.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.335 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38891 2024-09-17T22:41:30.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.336 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:41:30.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:41:30.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:41:30.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-09-17T22:41:30.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781802+0000", 2024-09-17T22:41:30.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:41:30.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781810+0000", 2024-09-17T22:41:30.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8019999999999992e-06 2024-09-17T22:41:30.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781816+0000", 2024-09-17T22:41:30.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.285e-06 2024-09-17T22:41:30.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832485+0000", 2024-09-17T22:41:30.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050668462999999997 2024-09-17T22:41:30.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:30.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832565+0000", 2024-09-17T22:41:30.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9692999999999997e-05 2024-09-17T22:41:30.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832647+0000", 2024-09-17T22:41:30.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2224000000000004e-05 2024-09-17T22:41:30.339 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.339 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.339 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.339 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38892 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.339 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:41:30.339 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.488325599, 2024-09-17T22:41:30.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050946120999999997, 2024-09-17T22:41:30.339 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.339 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.339 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:30.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:30.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.340 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38892 2024-09-17T22:41:30.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.340 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:41:30.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:41:30.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781828+0000", 2024-09-17T22:41:30.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5400000000000002e-07 2024-09-17T22:41:30.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781830+0000", 2024-09-17T22:41:30.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1959999999999998e-06 2024-09-17T22:41:30.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781837+0000", 2024-09-17T22:41:30.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8310000000000003e-06 2024-09-17T22:41:30.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781843+0000", 2024-09-17T22:41:30.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1e-06 2024-09-17T22:41:30.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832674+0000", 2024-09-17T22:41:30.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050831182000000003 2024-09-17T22:41:30.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:30.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832725+0000", 2024-09-17T22:41:30.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1588000000000001e-05 2024-09-17T22:41:30.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832773+0000", 2024-09-17T22:41:30.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7769999999999998e-05 2024-09-17T22:41:30.343 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.343 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.343 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.344 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38909 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 36864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.344 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:41:30.344 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.48830282899999999, 2024-09-17T22:41:30.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051046518999999999, 2024-09-17T22:41:30.344 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.344 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.344 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:30.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:30.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.344 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38909 2024-09-17T22:41:30.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.345 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:41:30.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:41:30.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:41:30.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:41:30.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781852+0000", 2024-09-17T22:41:30.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0590000000000001e-06 2024-09-17T22:41:30.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781859+0000", 2024-09-17T22:41:30.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3380000000000001e-06 2024-09-17T22:41:30.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781864+0000", 2024-09-17T22:41:30.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:41:30.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832804+0000", 2024-09-17T22:41:30.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050939614000000001 2024-09-17T22:41:30.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:30.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832845+0000", 2024-09-17T22:41:30.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1014999999999999e-05 2024-09-17T22:41:30.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832896+0000", 2024-09-17T22:41:30.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1836999999999999e-05 2024-09-17T22:41:30.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.348 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.348 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38910 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.348 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:41:30.348 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.48828251900000003, 2024-09-17T22:41:30.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051216818999999997, 2024-09-17T22:41:30.348 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.348 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.348 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:30.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:30.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.349 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38910 2024-09-17T22:41:30.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.349 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:41:30.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:41:30.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781871+0000", 2024-09-17T22:41:30.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:41:30.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781873+0000", 2024-09-17T22:41:30.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:41:30.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781879+0000", 2024-09-17T22:41:30.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0700000000000003e-06 2024-09-17T22:41:30.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781884+0000", 2024-09-17T22:41:30.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-09-17T22:41:30.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832932+0000", 2024-09-17T22:41:30.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051048219999999998 2024-09-17T22:41:30.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:30.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833001+0000", 2024-09-17T22:41:30.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8504999999999995e-05 2024-09-17T22:41:30.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833087+0000", 2024-09-17T22:41:30.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6464000000000004e-05 2024-09-17T22:41:30.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.352 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.353 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38923 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.353 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:41:30.353 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.48826219900000001, 2024-09-17T22:41:30.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051379573999999997, 2024-09-17T22:41:30.353 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.353 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.353 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:30.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:30.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.353 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38923 2024-09-17T22:41:30.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.354 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:41:30.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:41:30.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:41:30.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:41:30.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781893+0000", 2024-09-17T22:41:30.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-09-17T22:41:30.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781900+0000", 2024-09-17T22:41:30.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3860000000000004e-06 2024-09-17T22:41:30.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781904+0000", 2024-09-17T22:41:30.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9300000000000002e-06 2024-09-17T22:41:30.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833115+0000", 2024-09-17T22:41:30.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051210845999999997 2024-09-17T22:41:30.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:30.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833188+0000", 2024-09-17T22:41:30.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2276999999999999e-05 2024-09-17T22:41:30.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833270+0000", 2024-09-17T22:41:30.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2501000000000002e-05 2024-09-17T22:41:30.357 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.357 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.357 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.357 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38924 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.357 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.782651+0000", 2024-09-17T22:41:30.357 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.48750128399999998, 2024-09-17T22:41:30.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050794405000000001, 2024-09-17T22:41:30.358 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.358 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.358 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:30.358 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.358 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:30.358 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.358 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38924 2024-09-17T22:41:30.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.358 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782651+0000", 2024-09-17T22:41:30.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782651+0000", 2024-09-17T22:41:30.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782652+0000", 2024-09-17T22:41:30.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9699999999999996e-07 2024-09-17T22:41:30.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782659+0000", 2024-09-17T22:41:30.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7179999999999999e-06 2024-09-17T22:41:30.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782667+0000", 2024-09-17T22:41:30.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6959999999999994e-06 2024-09-17T22:41:30.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782674+0000", 2024-09-17T22:41:30.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5529999999999998e-06 2024-09-17T22:41:30.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833298+0000", 2024-09-17T22:41:30.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050623965999999999 2024-09-17T22:41:30.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:30.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833366+0000", 2024-09-17T22:41:30.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8303e-05 2024-09-17T22:41:30.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833446+0000", 2024-09-17T22:41:30.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9572000000000003e-05 2024-09-17T22:41:30.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.361 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.362 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38941 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.362 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:41:30.362 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.48746966400000002, 2024-09-17T22:41:30.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050969459000000002, 2024-09-17T22:41:30.362 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:30.362 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:30.362 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:30.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:30.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:30.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.362 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38941 2024-09-17T22:41:30.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.363 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:30.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:30.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:41:30.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:30.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:41:30.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:30.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:30.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782684+0000", 2024-09-17T22:41:30.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:41:30.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:30.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782686+0000", 2024-09-17T22:41:30.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.396e-06 2024-09-17T22:41:30.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:30.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782693+0000", 2024-09-17T22:41:30.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1999999999999997e-06 2024-09-17T22:41:30.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782699+0000", 2024-09-17T22:41:30.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7910000000000001e-06 2024-09-17T22:41:30.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:30.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833471+0000", 2024-09-17T22:41:30.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050772333000000003 2024-09-17T22:41:30.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:30.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833536+0000", 2024-09-17T22:41:30.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4380000000000004e-05 2024-09-17T22:41:30.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:30.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:30.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:30.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833653+0000", 2024-09-17T22:41:30.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011691799999999999 2024-09-17T22:41:30.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.366 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:30.366 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:30.366 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:30.367 INFO:teuthology.orchestra.run.smithi115.stdout:384 read 0x9c8391c thru 0x9c86b12 (0x31f7 bytes) 2024-09-17T22:41:30.368 INFO:teuthology.orchestra.run.smithi115.stdout:385 punch from 0xc1fbeab to 0xc1ffdc7, (0x3f1c bytes) 2024-09-17T22:41:30.368 INFO:teuthology.orchestra.run.smithi115.stdout:386 writesame 0x208d45f thru 0x2095550 data_size 0x5(0x80f2 bytes) 2024-09-17T22:41:30.369 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:41:30.370 INFO:teuthology.orchestra.run.smithi115.stdout:389 writesame 0x6451eb6 thru 0x645fc1a data_size 0x13(0xdd65 bytes) 2024-09-17T22:41:30.370 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:30.370 INFO:teuthology.orchestra.run.smithi115.stdout:393 read 0xc1edd1e thru 0xc1fa44f (0xc732 bytes) 2024-09-17T22:41:30.370 INFO:teuthology.orchestra.run.smithi115.stdout:394 read 0x1308b28 thru 0x1312468 (0x9941 bytes) 2024-09-17T22:41:30.370 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:30.370 INFO:teuthology.orchestra.run.smithi115.stdout:399 write 0xd24b7e3 thru 0xd2579bd (0xc1db bytes) 2024-09-17T22:41:30.370 INFO:teuthology.orchestra.run.smithi115.stdout:401 writesame 0x9169dd3 thru 0x9175666 data_size 0x2(0xb894 bytes) 2024-09-17T22:41:30.370 INFO:teuthology.orchestra.run.smithi115.stdout:402 punch from 0x206c6eb to 0x206db31, (0x1446 bytes) 2024-09-17T22:41:30.371 INFO:teuthology.orchestra.run.smithi115.stdout:403 writesame 0xe86be0f thru 0xe87a9b2 data_size 0x42(0xeba4 bytes) 2024-09-17T22:41:30.371 INFO:teuthology.orchestra.run.smithi115.stdout:404 write 0x956b184 thru 0x95786fb (0xd578 bytes) 2024-09-17T22:41:30.371 INFO:teuthology.orchestra.run.smithi115.stdout:405 read 0x88e2ee9 thru 0x88e38d1 (0x9e9 bytes) 2024-09-17T22:41:30.371 INFO:teuthology.orchestra.run.smithi115.stdout:406 read 0xa91a772 thru 0xa920afe (0x638d bytes) 2024-09-17T22:41:30.371 INFO:teuthology.orchestra.run.smithi115.stdout:408 clone 4 order 20 su 16384 sc 7 2024-09-17T22:41:30.371 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:30.365+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:30.469 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:30.466+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:30.474 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:30.475 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:30.475 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:41:30.475 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:30.483 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:41:30.569 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:30.567+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:30.627 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:30.627 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:30.627 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:30.627 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:30.627 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:30.635 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:41:30.676 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:30.674+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:30.771 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:30.769+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:30.780 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:30.781 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:30.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:30.781 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:30.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.781 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:637 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:30.781 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:24.660367+0000", 2024-09-17T22:41:30.781 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.1169241359999997, 2024-09-17T22:41:30.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027568698999999999, 2024-09-17T22:41:30.781 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.781 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.781 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:30.781 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.782 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.782 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.782 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 637 2024-09-17T22:41:30.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.782 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660367+0000", 2024-09-17T22:41:30.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660367+0000", 2024-09-17T22:41:30.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660368+0000", 2024-09-17T22:41:30.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1299999999999999e-07 2024-09-17T22:41:30.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660372+0000", 2024-09-17T22:41:30.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3819999999999997e-06 2024-09-17T22:41:30.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660382+0000", 2024-09-17T22:41:30.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8470000000000008e-06 2024-09-17T22:41:30.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660387+0000", 2024-09-17T22:41:30.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-17T22:41:30.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660429+0000", 2024-09-17T22:41:30.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2194000000000003e-05 2024-09-17T22:41:30.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660518+0000", 2024-09-17T22:41:30.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8572999999999994e-05 2024-09-17T22:41:30.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-17T22:41:30.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.664974+0000", 2024-09-17T22:41:30.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0044561469999999997 2024-09-17T22:41:30.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:30.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.687829+0000", 2024-09-17T22:41:30.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022854695000000001 2024-09-17T22:41:30.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:41:30.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.687850+0000", 2024-09-17T22:41:30.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0933000000000001e-05 2024-09-17T22:41:30.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:30.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.687885+0000", 2024-09-17T22:41:30.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5301e-05 2024-09-17T22:41:30.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.687936+0000", 2024-09-17T22:41:30.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0726999999999997e-05 2024-09-17T22:41:30.787 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.787 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.787 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.787 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6543 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5392384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:30.787 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476788+0000", 2024-09-17T22:41:30.787 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.3005035200000004, 2024-09-17T22:41:30.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024737467999999999, 2024-09-17T22:41:30.788 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.788 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.788 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:30.788 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.788 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.788 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.788 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6543 2024-09-17T22:41:30.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.788 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476788+0000", 2024-09-17T22:41:30.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476788+0000", 2024-09-17T22:41:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476788+0000", 2024-09-17T22:41:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6100000000000002e-07 2024-09-17T22:41:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476790+0000", 2024-09-17T22:41:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9570000000000001e-06 2024-09-17T22:41:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476796+0000", 2024-09-17T22:41:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9710000000000001e-06 2024-09-17T22:41:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476801+0000", 2024-09-17T22:41:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7129999999999996e-06 2024-09-17T22:41:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501329+0000", 2024-09-17T22:41:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024528378 2024-09-17T22:41:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501415+0000", 2024-09-17T22:41:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6184e-05 2024-09-17T22:41:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501525+0000", 2024-09-17T22:41:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000109904 2024-09-17T22:41:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.791 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6544 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5410816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476807+0000", 2024-09-17T22:41:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.3004844980000003, 2024-09-17T22:41:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024850173, 2024-09-17T22:41:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.792 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.793 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6544 2024-09-17T22:41:30.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.793 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476807+0000", 2024-09-17T22:41:30.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.793 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.793 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.793 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476807+0000", 2024-09-17T22:41:30.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476807+0000", 2024-09-17T22:41:30.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.58e-07 2024-09-17T22:41:30.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476810+0000", 2024-09-17T22:41:30.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7889999999999999e-06 2024-09-17T22:41:30.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476815+0000", 2024-09-17T22:41:30.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7299999999999996e-06 2024-09-17T22:41:30.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476819+0000", 2024-09-17T22:41:30.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6879999999999998e-06 2024-09-17T22:41:30.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501554+0000", 2024-09-17T22:41:30.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024734471000000001 2024-09-17T22:41:30.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501598+0000", 2024-09-17T22:41:30.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4257e-05 2024-09-17T22:41:30.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501657+0000", 2024-09-17T22:41:30.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8879999999999999e-05 2024-09-17T22:41:30.796 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.796 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.796 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.796 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6545 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5429248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:30.796 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476825+0000", 2024-09-17T22:41:30.796 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.3004658779999998, 2024-09-17T22:41:30.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024965472999999998, 2024-09-17T22:41:30.797 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.797 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.797 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:30.797 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.797 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.797 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.797 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6545 2024-09-17T22:41:30.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.797 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476825+0000", 2024-09-17T22:41:30.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476825+0000", 2024-09-17T22:41:30.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476826+0000", 2024-09-17T22:41:30.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4499999999999998e-07 2024-09-17T22:41:30.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476828+0000", 2024-09-17T22:41:30.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.88e-06 2024-09-17T22:41:30.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476835+0000", 2024-09-17T22:41:30.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4089999999999997e-06 2024-09-17T22:41:30.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476845+0000", 2024-09-17T22:41:30.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8660000000000001e-06 2024-09-17T22:41:30.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501693+0000", 2024-09-17T22:41:30.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024848282999999999 2024-09-17T22:41:30.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501736+0000", 2024-09-17T22:41:30.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3101e-05 2024-09-17T22:41:30.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501791+0000", 2024-09-17T22:41:30.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4589000000000002e-05 2024-09-17T22:41:30.800 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.801 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.801 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.801 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6546 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:30.801 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476854+0000", 2024-09-17T22:41:30.801 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.3004376420000003, 2024-09-17T22:41:30.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025123742000000001, 2024-09-17T22:41:30.801 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.801 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.801 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:30.802 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.802 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.802 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.802 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6546 2024-09-17T22:41:30.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.802 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476854+0000", 2024-09-17T22:41:30.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476854+0000", 2024-09-17T22:41:30.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476854+0000", 2024-09-17T22:41:30.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T22:41:30.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476856+0000", 2024-09-17T22:41:30.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4820000000000001e-06 2024-09-17T22:41:30.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476863+0000", 2024-09-17T22:41:30.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6749999999999996e-06 2024-09-17T22:41:30.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476868+0000", 2024-09-17T22:41:30.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0150000000000003e-06 2024-09-17T22:41:30.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501814+0000", 2024-09-17T22:41:30.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024946111 2024-09-17T22:41:30.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501888+0000", 2024-09-17T22:41:30.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4200000000000001e-05 2024-09-17T22:41:30.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501977+0000", 2024-09-17T22:41:30.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8799000000000001e-05 2024-09-17T22:41:30.805 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.805 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.805 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.806 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6547 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5466112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:30.806 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476875+0000", 2024-09-17T22:41:30.806 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.300416351, 2024-09-17T22:41:30.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025238633999999999, 2024-09-17T22:41:30.806 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.806 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.806 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:30.806 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.806 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.806 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.806 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6547 2024-09-17T22:41:30.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.807 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476875+0000", 2024-09-17T22:41:30.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476875+0000", 2024-09-17T22:41:30.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476875+0000", 2024-09-17T22:41:30.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:41:30.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476877+0000", 2024-09-17T22:41:30.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.075e-06 2024-09-17T22:41:30.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476883+0000", 2024-09-17T22:41:30.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.976e-06 2024-09-17T22:41:30.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476888+0000", 2024-09-17T22:41:30.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7430000000000002e-06 2024-09-17T22:41:30.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501974+0000", 2024-09-17T22:41:30.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025085547 2024-09-17T22:41:30.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502055+0000", 2024-09-17T22:41:30.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1484999999999997e-05 2024-09-17T22:41:30.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502113+0000", 2024-09-17T22:41:30.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8251999999999997e-05 2024-09-17T22:41:30.810 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.810 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.810 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.810 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6548 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5484544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:30.810 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476894+0000", 2024-09-17T22:41:30.810 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.3003969629999999, 2024-09-17T22:41:30.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025340366, 2024-09-17T22:41:30.811 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.811 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.811 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:30.811 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.811 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.811 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.811 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6548 2024-09-17T22:41:30.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.811 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476894+0000", 2024-09-17T22:41:30.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476894+0000", 2024-09-17T22:41:30.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476895+0000", 2024-09-17T22:41:30.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:41:30.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476897+0000", 2024-09-17T22:41:30.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9609999999999999e-06 2024-09-17T22:41:30.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476902+0000", 2024-09-17T22:41:30.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6969999999999998e-06 2024-09-17T22:41:30.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476908+0000", 2024-09-17T22:41:30.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3090000000000003e-06 2024-09-17T22:41:30.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502138+0000", 2024-09-17T22:41:30.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025230742 2024-09-17T22:41:30.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502181+0000", 2024-09-17T22:41:30.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2580999999999998e-05 2024-09-17T22:41:30.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502235+0000", 2024-09-17T22:41:30.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3684999999999998e-05 2024-09-17T22:41:30.815 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.815 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.815 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.815 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6549 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5502976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:30.815 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476914+0000", 2024-09-17T22:41:30.815 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.3003767740000001, 2024-09-17T22:41:30.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025541851000000001, 2024-09-17T22:41:30.815 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.815 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.815 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:30.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.816 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6549 2024-09-17T22:41:30.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.816 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476914+0000", 2024-09-17T22:41:30.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476914+0000", 2024-09-17T22:41:30.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476915+0000", 2024-09-17T22:41:30.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7899999999999999e-07 2024-09-17T22:41:30.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476917+0000", 2024-09-17T22:41:30.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.03e-06 2024-09-17T22:41:30.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476923+0000", 2024-09-17T22:41:30.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2210000000000002e-06 2024-09-17T22:41:30.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476928+0000", 2024-09-17T22:41:30.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4429999999999996e-06 2024-09-17T22:41:30.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502278+0000", 2024-09-17T22:41:30.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025350784000000001 2024-09-17T22:41:30.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502364+0000", 2024-09-17T22:41:30.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5779999999999995e-05 2024-09-17T22:41:30.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502456+0000", 2024-09-17T22:41:30.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2114000000000006e-05 2024-09-17T22:41:30.819 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.819 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.820 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.820 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6550 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5507072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:30.820 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476934+0000", 2024-09-17T22:41:30.820 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.3003574350000005, 2024-09-17T22:41:30.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025625060000000002, 2024-09-17T22:41:30.820 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.820 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.820 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:30.820 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.820 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.820 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.821 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6550 2024-09-17T22:41:30.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.821 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476934+0000", 2024-09-17T22:41:30.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476934+0000", 2024-09-17T22:41:30.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476934+0000", 2024-09-17T22:41:30.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:41:30.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476936+0000", 2024-09-17T22:41:30.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9479999999999998e-06 2024-09-17T22:41:30.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476942+0000", 2024-09-17T22:41:30.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.032e-06 2024-09-17T22:41:30.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476948+0000", 2024-09-17T22:41:30.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.57e-06 2024-09-17T22:41:30.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502474+0000", 2024-09-17T22:41:30.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025525808000000001 2024-09-17T22:41:30.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502510+0000", 2024-09-17T22:41:30.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6452000000000003e-05 2024-09-17T22:41:30.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502559+0000", 2024-09-17T22:41:30.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8828000000000001e-05 2024-09-17T22:41:30.824 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.824 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.824 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.825 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6551 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5525504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:30.825 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476954+0000", 2024-09-17T22:41:30.825 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.3003374140000004, 2024-09-17T22:41:30.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025815945, 2024-09-17T22:41:30.825 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.825 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.825 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:30.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.826 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6551 2024-09-17T22:41:30.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.826 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476954+0000", 2024-09-17T22:41:30.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476954+0000", 2024-09-17T22:41:30.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476954+0000", 2024-09-17T22:41:30.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1800000000000001e-07 2024-09-17T22:41:30.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476956+0000", 2024-09-17T22:41:30.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0810000000000001e-06 2024-09-17T22:41:30.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476974+0000", 2024-09-17T22:41:30.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8168e-05 2024-09-17T22:41:30.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476980+0000", 2024-09-17T22:41:30.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8200000000000002e-06 2024-09-17T22:41:30.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502595+0000", 2024-09-17T22:41:30.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025614865000000001 2024-09-17T22:41:30.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502675+0000", 2024-09-17T22:41:30.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:41:30.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502770+0000", 2024-09-17T22:41:30.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4789000000000003e-05 2024-09-17T22:41:30.829 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.829 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.829 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.830 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6552 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5543936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:30.830 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476988+0000", 2024-09-17T22:41:30.830 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.3003035809999997, 2024-09-17T22:41:30.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02590947, 2024-09-17T22:41:30.830 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.830 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.830 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:30.830 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.830 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.830 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.830 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6552 2024-09-17T22:41:30.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.831 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476988+0000", 2024-09-17T22:41:30.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476988+0000", 2024-09-17T22:41:30.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476988+0000", 2024-09-17T22:41:30.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T22:41:30.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476995+0000", 2024-09-17T22:41:30.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.533e-06 2024-09-17T22:41:30.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477001+0000", 2024-09-17T22:41:30.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6699999999999997e-06 2024-09-17T22:41:30.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477006+0000", 2024-09-17T22:41:30.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6970000000000001e-06 2024-09-17T22:41:30.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502798+0000", 2024-09-17T22:41:30.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025792234000000001 2024-09-17T22:41:30.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502835+0000", 2024-09-17T22:41:30.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7073999999999998e-05 2024-09-17T22:41:30.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502897+0000", 2024-09-17T22:41:30.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1828999999999994e-05 2024-09-17T22:41:30.834 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.834 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.834 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.834 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6553 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5562368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:30.834 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.477013+0000", 2024-09-17T22:41:30.834 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.3002786049999999, 2024-09-17T22:41:30.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026089772000000001, 2024-09-17T22:41:30.835 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.835 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.835 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:30.835 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.835 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.835 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.835 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6553 2024-09-17T22:41:30.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.835 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477013+0000", 2024-09-17T22:41:30.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477013+0000", 2024-09-17T22:41:30.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477013+0000", 2024-09-17T22:41:30.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:41:30.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477016+0000", 2024-09-17T22:41:30.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.207e-06 2024-09-17T22:41:30.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477023+0000", 2024-09-17T22:41:30.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.613e-06 2024-09-17T22:41:30.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477028+0000", 2024-09-17T22:41:30.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0669999999999999e-06 2024-09-17T22:41:30.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502933+0000", 2024-09-17T22:41:30.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025905076999999999 2024-09-17T22:41:30.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503010+0000", 2024-09-17T22:41:30.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6716000000000002e-05 2024-09-17T22:41:30.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503102+0000", 2024-09-17T22:41:30.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2651000000000005e-05 2024-09-17T22:41:30.839 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.839 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.839 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.839 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6554 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5580800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:30.839 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.477034+0000", 2024-09-17T22:41:30.840 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.3002573780000004, 2024-09-17T22:41:30.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026183898000000001, 2024-09-17T22:41:30.840 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.840 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.840 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:30.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.840 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6554 2024-09-17T22:41:30.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.840 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477034+0000", 2024-09-17T22:41:30.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477034+0000", 2024-09-17T22:41:30.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477034+0000", 2024-09-17T22:41:30.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6399999999999998e-07 2024-09-17T22:41:30.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477036+0000", 2024-09-17T22:41:30.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.889e-06 2024-09-17T22:41:30.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477042+0000", 2024-09-17T22:41:30.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7910000000000001e-06 2024-09-17T22:41:30.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477046+0000", 2024-09-17T22:41:30.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.561e-06 2024-09-17T22:41:30.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503130+0000", 2024-09-17T22:41:30.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026083168 2024-09-17T22:41:30.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503166+0000", 2024-09-17T22:41:30.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6879999999999999e-05 2024-09-17T22:41:30.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503218+0000", 2024-09-17T22:41:30.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1245000000000001e-05 2024-09-17T22:41:30.844 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.844 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.844 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.844 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6555 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5599232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:30.844 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.477053+0000", 2024-09-17T22:41:30.844 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.3002384859999996, 2024-09-17T22:41:30.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026368473, 2024-09-17T22:41:30.845 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.845 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.845 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:30.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.845 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6555 2024-09-17T22:41:30.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.845 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477053+0000", 2024-09-17T22:41:30.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477053+0000", 2024-09-17T22:41:30.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477053+0000", 2024-09-17T22:41:30.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6699999999999999e-07 2024-09-17T22:41:30.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477055+0000", 2024-09-17T22:41:30.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9120000000000001e-06 2024-09-17T22:41:30.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477061+0000", 2024-09-17T22:41:30.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6149999999999996e-06 2024-09-17T22:41:30.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477065+0000", 2024-09-17T22:41:30.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2080000000000002e-06 2024-09-17T22:41:30.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503252+0000", 2024-09-17T22:41:30.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026187430000000001 2024-09-17T22:41:30.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503330+0000", 2024-09-17T22:41:30.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7812000000000006e-05 2024-09-17T22:41:30.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503421+0000", 2024-09-17T22:41:30.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1129000000000001e-05 2024-09-17T22:41:30.849 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.849 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.849 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.849 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6556 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5617664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:30.849 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.477071+0000", 2024-09-17T22:41:30.849 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.3002200540000004, 2024-09-17T22:41:30.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026451061000000001, 2024-09-17T22:41:30.849 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.849 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.849 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:30.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.850 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.850 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6556 2024-09-17T22:41:30.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.850 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477071+0000", 2024-09-17T22:41:30.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477071+0000", 2024-09-17T22:41:30.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477072+0000", 2024-09-17T22:41:30.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3799999999999998e-07 2024-09-17T22:41:30.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477074+0000", 2024-09-17T22:41:30.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9870000000000002e-06 2024-09-17T22:41:30.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477080+0000", 2024-09-17T22:41:30.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3459999999999999e-06 2024-09-17T22:41:30.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477084+0000", 2024-09-17T22:41:30.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4880000000000001e-06 2024-09-17T22:41:30.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503438+0000", 2024-09-17T22:41:30.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026353225000000001 2024-09-17T22:41:30.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503473+0000", 2024-09-17T22:41:30.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5472000000000002e-05 2024-09-17T22:41:30.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503522+0000", 2024-09-17T22:41:30.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9104999999999998e-05 2024-09-17T22:41:30.853 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.853 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.853 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.853 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:33622 3.7 3:e206720e:::rbd_data.115fe0c30ef6.0000000000000042:5 [read 223232~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.853 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.501373+0000", 2024-09-17T22:41:30.854 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.2759178799999999, 2024-09-17T22:41:30.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024549422000000001, 2024-09-17T22:41:30.854 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.854 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.854 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:30.854 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.854 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.854 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.854 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 33622 2024-09-17T22:41:30.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.854 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501373+0000", 2024-09-17T22:41:30.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501373+0000", 2024-09-17T22:41:30.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501374+0000", 2024-09-17T22:41:30.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:41:30.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501380+0000", 2024-09-17T22:41:30.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1750000000000002e-06 2024-09-17T22:41:30.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501387+0000", 2024-09-17T22:41:30.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.939e-06 2024-09-17T22:41:30.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501393+0000", 2024-09-17T22:41:30.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8370000000000002e-06 2024-09-17T22:41:30.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.525744+0000", 2024-09-17T22:41:30.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024351482000000001 2024-09-17T22:41:30.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.525827+0000", 2024-09-17T22:41:30.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2437999999999999e-05 2024-09-17T22:41:30.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.525923+0000", 2024-09-17T22:41:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6057999999999994e-05 2024-09-17T22:41:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:33624 3.7 3:e206720e:::rbd_data.115fe0c30ef6.0000000000000042:5 [read 231424~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.501400+0000", 2024-09-17T22:41:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.2758911749999999, 2024-09-17T22:41:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024723893, 2024-09-17T22:41:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 33624 2024-09-17T22:41:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501400+0000", 2024-09-17T22:41:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501400+0000", 2024-09-17T22:41:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501400+0000", 2024-09-17T22:41:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8100000000000003e-07 2024-09-17T22:41:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501403+0000", 2024-09-17T22:41:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1500000000000002e-06 2024-09-17T22:41:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501409+0000", 2024-09-17T22:41:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3779999999999998e-06 2024-09-17T22:41:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501414+0000", 2024-09-17T22:41:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1139999999999997e-06 2024-09-17T22:41:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.525952+0000", 2024-09-17T22:41:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024537792999999999 2024-09-17T22:41:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526035+0000", 2024-09-17T22:41:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2948000000000006e-05 2024-09-17T22:41:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526124+0000", 2024-09-17T22:41:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9029000000000004e-05 2024-09-17T22:41:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:33625 3.7 3:e206720e:::rbd_data.115fe0c30ef6.0000000000000042:5 [read 245760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.501420+0000", 2024-09-17T22:41:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.2758711169999999, 2024-09-17T22:41:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024907489000000001, 2024-09-17T22:41:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 33625 2024-09-17T22:41:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501420+0000", 2024-09-17T22:41:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501420+0000", 2024-09-17T22:41:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501420+0000", 2024-09-17T22:41:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:41:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501428+0000", 2024-09-17T22:41:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6520000000000006e-06 2024-09-17T22:41:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501435+0000", 2024-09-17T22:41:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9229999999999996e-06 2024-09-17T22:41:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501440+0000", 2024-09-17T22:41:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3569999999999997e-06 2024-09-17T22:41:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526155+0000", 2024-09-17T22:41:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024715114999999999 2024-09-17T22:41:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526233+0000", 2024-09-17T22:41:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7560999999999998e-05 2024-09-17T22:41:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526328+0000", 2024-09-17T22:41:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4526999999999998e-05 2024-09-17T22:41:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:33626 3.7 3:e206720e:::rbd_data.115fe0c30ef6.0000000000000042:5 [read 253952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.501477+0000", 2024-09-17T22:41:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.275814427, 2024-09-17T22:41:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025046036000000001, 2024-09-17T22:41:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 33626 2024-09-17T22:41:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501477+0000", 2024-09-17T22:41:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501477+0000", 2024-09-17T22:41:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501478+0000", 2024-09-17T22:41:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0299999999999998e-07 2024-09-17T22:41:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501483+0000", 2024-09-17T22:41:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.519e-06 2024-09-17T22:41:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501492+0000", 2024-09-17T22:41:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4819999999999994e-06 2024-09-17T22:41:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501498+0000", 2024-09-17T22:41:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6780000000000003e-06 2024-09-17T22:41:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526356+0000", 2024-09-17T22:41:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024858203999999998 2024-09-17T22:41:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526427+0000", 2024-09-17T22:41:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1021999999999999e-05 2024-09-17T22:41:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526523+0000", 2024-09-17T22:41:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5328000000000006e-05 2024-09-17T22:41:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:33627 3.7 3:e206720e:::rbd_data.115fe0c30ef6.0000000000000042:5 [read 260096~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.501510+0000", 2024-09-17T22:41:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.275780868, 2024-09-17T22:41:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025207994000000001, 2024-09-17T22:41:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 33627 2024-09-17T22:41:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501510+0000", 2024-09-17T22:41:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501510+0000", 2024-09-17T22:41:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501511+0000", 2024-09-17T22:41:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3399999999999999e-07 2024-09-17T22:41:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501513+0000", 2024-09-17T22:41:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5009999999999999e-06 2024-09-17T22:41:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501520+0000", 2024-09-17T22:41:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7800000000000003e-06 2024-09-17T22:41:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501525+0000", 2024-09-17T22:41:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3399999999999997e-06 2024-09-17T22:41:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526550+0000", 2024-09-17T22:41:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025024066000000001 2024-09-17T22:41:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526623+0000", 2024-09-17T22:41:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3240999999999999e-05 2024-09-17T22:41:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526718+0000", 2024-09-17T22:41:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5532000000000006e-05 2024-09-17T22:41:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:30.876 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:30.878 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:41:30.878 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:30.875+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:30.978 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:30.976+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:30.980 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is set 2024-09-17T22:41:30.981 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:30.981 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:30.981 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:41:30.981 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:30.991 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:41:31.079 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:31.077+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:31.134 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:31.134 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:31.134 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:31.134 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:31.134 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:31.143 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:41:31.180 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:31.178+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:31.183 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-17T22:41:31.184 INFO:teuthology.orchestra.run.smithi115.stdout:409 clone 5 order 18 su 131072 sc 10 2024-09-17T22:41:31.187 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:41:31.248 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:31.248 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:31.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:31.248 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:31.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.249 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:1016 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:31.249 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:24.899766+0000", 2024-09-17T22:41:31.249 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.3454395129999996, 2024-09-17T22:41:31.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020466634000000001, 2024-09-17T22:41:31.249 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.249 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.249 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:31.249 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.249 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.249 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.249 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1016 2024-09-17T22:41:31.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.250 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899766+0000", 2024-09-17T22:41:31.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899766+0000", 2024-09-17T22:41:31.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899766+0000", 2024-09-17T22:41:31.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4799999999999997e-07 2024-09-17T22:41:31.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899771+0000", 2024-09-17T22:41:31.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8489999999999997e-06 2024-09-17T22:41:31.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899782+0000", 2024-09-17T22:41:31.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0702e-05 2024-09-17T22:41:31.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899787+0000", 2024-09-17T22:41:31.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1460000000000004e-06 2024-09-17T22:41:31.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899823+0000", 2024-09-17T22:41:31.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5478000000000002e-05 2024-09-17T22:41:31.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899878+0000", 2024-09-17T22:41:31.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5034999999999997e-05 2024-09-17T22:41:31.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T22:41:31.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.903518+0000", 2024-09-17T22:41:31.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036402240000000001 2024-09-17T22:41:31.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:31.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.920108+0000", 2024-09-17T22:41:31.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016589650000000001 2024-09-17T22:41:31.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:41:31.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.920127+0000", 2024-09-17T22:41:31.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9504000000000002e-05 2024-09-17T22:41:31.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:31.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.920180+0000", 2024-09-17T22:41:31.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3276999999999998e-05 2024-09-17T22:41:31.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.920232+0000", 2024-09-17T22:41:31.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2021000000000002e-05 2024-09-17T22:41:31.254 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.254 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.254 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.254 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6105 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 5564416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:31.254 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.471471+0000", 2024-09-17T22:41:31.254 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7737346220000001, 2024-09-17T22:41:31.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017349954000000001, 2024-09-17T22:41:31.255 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.255 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.255 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:31.255 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.255 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.255 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.255 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6105 2024-09-17T22:41:31.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.255 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471471+0000", 2024-09-17T22:41:31.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471471+0000", 2024-09-17T22:41:31.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471471+0000", 2024-09-17T22:41:31.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7599999999999998e-07 2024-09-17T22:41:31.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471481+0000", 2024-09-17T22:41:31.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0207000000000001e-05 2024-09-17T22:41:31.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471488+0000", 2024-09-17T22:41:31.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0700000000000001e-06 2024-09-17T22:41:31.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471493+0000", 2024-09-17T22:41:31.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1900000000000003e-06 2024-09-17T22:41:31.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.481097+0000", 2024-09-17T22:41:31.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0096031459999999999 2024-09-17T22:41:31.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.481170+0000", 2024-09-17T22:41:31.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3187000000000002e-05 2024-09-17T22:41:31.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.488821+0000", 2024-09-17T22:41:31.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0076507779999999996 2024-09-17T22:41:31.258 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.259 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.259 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.259 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35632 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2605056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:31.259 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.533932+0000", 2024-09-17T22:41:31.259 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.7112734469999999, 2024-09-17T22:41:31.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016583209000000002, 2024-09-17T22:41:31.259 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.259 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.259 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:31.259 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.259 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.259 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.260 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35632 2024-09-17T22:41:31.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.260 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.533932+0000", 2024-09-17T22:41:31.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.533932+0000", 2024-09-17T22:41:31.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.533932+0000", 2024-09-17T22:41:31.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.05e-07 2024-09-17T22:41:31.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.533933+0000", 2024-09-17T22:41:31.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6200000000000006e-07 2024-09-17T22:41:31.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.533936+0000", 2024-09-17T22:41:31.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9000000000000002e-06 2024-09-17T22:41:31.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.533939+0000", 2024-09-17T22:41:31.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6639999999999998e-06 2024-09-17T22:41:31.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.550351+0000", 2024-09-17T22:41:31.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016412663000000001 2024-09-17T22:41:31.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.550423+0000", 2024-09-17T22:41:31.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2175999999999995e-05 2024-09-17T22:41:31.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.550515+0000", 2024-09-17T22:41:31.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1638999999999995e-05 2024-09-17T22:41:31.263 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.263 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.263 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.263 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35695 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2553856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:31.263 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.536131+0000", 2024-09-17T22:41:31.263 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.7090746970000001, 2024-09-17T22:41:31.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.01647641, 2024-09-17T22:41:31.264 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.264 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.264 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:31.264 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.264 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.264 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.264 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35695 2024-09-17T22:41:31.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.264 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536131+0000", 2024-09-17T22:41:31.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536131+0000", 2024-09-17T22:41:31.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536131+0000", 2024-09-17T22:41:31.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.39e-07 2024-09-17T22:41:31.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536133+0000", 2024-09-17T22:41:31.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.95e-06 2024-09-17T22:41:31.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536138+0000", 2024-09-17T22:41:31.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1950000000000002e-06 2024-09-17T22:41:31.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536143+0000", 2024-09-17T22:41:31.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7779999999999998e-06 2024-09-17T22:41:31.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.552473+0000", 2024-09-17T22:41:31.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016330458999999999 2024-09-17T22:41:31.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.552517+0000", 2024-09-17T22:41:31.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3680999999999997e-05 2024-09-17T22:41:31.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.552607+0000", 2024-09-17T22:41:31.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0007999999999996e-05 2024-09-17T22:41:31.267 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.267 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.267 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.268 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35696 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:31.268 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.536148+0000", 2024-09-17T22:41:31.268 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.709056787, 2024-09-17T22:41:31.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016687067999999999, 2024-09-17T22:41:31.268 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.268 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.268 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:31.268 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.268 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.268 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.268 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35696 2024-09-17T22:41:31.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.269 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536148+0000", 2024-09-17T22:41:31.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536148+0000", 2024-09-17T22:41:31.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536149+0000", 2024-09-17T22:41:31.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:41:31.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536155+0000", 2024-09-17T22:41:31.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8690000000000001e-06 2024-09-17T22:41:31.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536161+0000", 2024-09-17T22:41:31.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7050000000000002e-06 2024-09-17T22:41:31.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536166+0000", 2024-09-17T22:41:31.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5909999999999998e-06 2024-09-17T22:41:31.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.552657+0000", 2024-09-17T22:41:31.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016490916000000001 2024-09-17T22:41:31.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.552741+0000", 2024-09-17T22:41:31.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3852999999999998e-05 2024-09-17T22:41:31.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.552836+0000", 2024-09-17T22:41:31.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4748999999999997e-05 2024-09-17T22:41:31.272 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.272 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.272 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.272 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35742 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2670592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:31.273 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.536998+0000", 2024-09-17T22:41:31.273 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.7082073209999999, 2024-09-17T22:41:31.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016453115000000001, 2024-09-17T22:41:31.273 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.273 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.273 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:31.273 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.273 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.273 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.273 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35742 2024-09-17T22:41:31.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.273 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536998+0000", 2024-09-17T22:41:31.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536998+0000", 2024-09-17T22:41:31.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536998+0000", 2024-09-17T22:41:31.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:41:31.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537000+0000", 2024-09-17T22:41:31.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1299999999999999e-06 2024-09-17T22:41:31.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537007+0000", 2024-09-17T22:41:31.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5289999999999997e-06 2024-09-17T22:41:31.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537012+0000", 2024-09-17T22:41:31.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2970000000000003e-06 2024-09-17T22:41:31.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553368+0000", 2024-09-17T22:41:31.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016356103 2024-09-17T22:41:31.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553404+0000", 2024-09-17T22:41:31.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5729000000000003e-05 2024-09-17T22:41:31.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.277 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.277 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553451+0000", 2024-09-17T22:41:31.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6935999999999999e-05 2024-09-17T22:41:31.277 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.277 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.277 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.277 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35743 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2744320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:31.277 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.537019+0000", 2024-09-17T22:41:31.277 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.708186486, 2024-09-17T22:41:31.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016542099000000001, 2024-09-17T22:41:31.277 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.277 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.277 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:31.278 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.278 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.278 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.278 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35743 2024-09-17T22:41:31.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.278 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537019+0000", 2024-09-17T22:41:31.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537019+0000", 2024-09-17T22:41:31.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537019+0000", 2024-09-17T22:41:31.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6300000000000001e-07 2024-09-17T22:41:31.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537021+0000", 2024-09-17T22:41:31.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9080000000000002e-06 2024-09-17T22:41:31.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537027+0000", 2024-09-17T22:41:31.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3090000000000001e-06 2024-09-17T22:41:31.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537033+0000", 2024-09-17T22:41:31.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4539999999999998e-06 2024-09-17T22:41:31.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553475+0000", 2024-09-17T22:41:31.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016442142999999999 2024-09-17T22:41:31.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553510+0000", 2024-09-17T22:41:31.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5030999999999997e-05 2024-09-17T22:41:31.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553561+0000", 2024-09-17T22:41:31.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0890999999999998e-05 2024-09-17T22:41:31.281 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.281 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.281 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.281 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35745 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2480128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:31.282 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.537054+0000", 2024-09-17T22:41:31.282 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.7081515139999999, 2024-09-17T22:41:31.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016621817000000001, 2024-09-17T22:41:31.282 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.282 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.282 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:31.282 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.282 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.282 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.282 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35745 2024-09-17T22:41:31.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.282 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537054+0000", 2024-09-17T22:41:31.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537054+0000", 2024-09-17T22:41:31.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537055+0000", 2024-09-17T22:41:31.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0400000000000005e-07 2024-09-17T22:41:31.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537059+0000", 2024-09-17T22:41:31.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9910000000000004e-06 2024-09-17T22:41:31.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537070+0000", 2024-09-17T22:41:31.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1398e-05 2024-09-17T22:41:31.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537077+0000", 2024-09-17T22:41:31.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4959999999999999e-06 2024-09-17T22:41:31.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553585+0000", 2024-09-17T22:41:31.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016508020000000002 2024-09-17T22:41:31.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553621+0000", 2024-09-17T22:41:31.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6013999999999998e-05 2024-09-17T22:41:31.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553676+0000", 2024-09-17T22:41:31.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4094000000000001e-05 2024-09-17T22:41:31.286 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.286 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.286 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.286 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35791 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2615296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:31.286 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.538661+0000", 2024-09-17T22:41:31.286 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.7065443390000001, 2024-09-17T22:41:31.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016411623, 2024-09-17T22:41:31.286 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.286 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.286 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:31.286 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.287 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35791 2024-09-17T22:41:31.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.287 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538661+0000", 2024-09-17T22:41:31.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538661+0000", 2024-09-17T22:41:31.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538661+0000", 2024-09-17T22:41:31.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3399999999999999e-07 2024-09-17T22:41:31.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538664+0000", 2024-09-17T22:41:31.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:41:31.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538685+0000", 2024-09-17T22:41:31.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1608999999999999e-05 2024-09-17T22:41:31.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538691+0000", 2024-09-17T22:41:31.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7629999999999997e-06 2024-09-17T22:41:31.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.554892+0000", 2024-09-17T22:41:31.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016200951000000002 2024-09-17T22:41:31.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.554962+0000", 2024-09-17T22:41:31.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9962999999999994e-05 2024-09-17T22:41:31.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555073+0000", 2024-09-17T22:41:31.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000110674 2024-09-17T22:41:31.290 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.290 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.290 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.290 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35805 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2674688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:31.290 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.538700+0000", 2024-09-17T22:41:31.290 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.7065057100000001, 2024-09-17T22:41:31.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016568598, 2024-09-17T22:41:31.291 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.291 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.291 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:31.291 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.291 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.291 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.291 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35805 2024-09-17T22:41:31.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.291 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.291 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.291 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538700+0000", 2024-09-17T22:41:31.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538700+0000", 2024-09-17T22:41:31.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538700+0000", 2024-09-17T22:41:31.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8500000000000002e-07 2024-09-17T22:41:31.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538702+0000", 2024-09-17T22:41:31.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.447e-06 2024-09-17T22:41:31.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538709+0000", 2024-09-17T22:41:31.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2530000000000001e-06 2024-09-17T22:41:31.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538714+0000", 2024-09-17T22:41:31.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8770000000000001e-06 2024-09-17T22:41:31.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555104+0000", 2024-09-17T22:41:31.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016390372 2024-09-17T22:41:31.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555168+0000", 2024-09-17T22:41:31.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3559000000000006e-05 2024-09-17T22:41:31.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555268+0000", 2024-09-17T22:41:31.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100605 2024-09-17T22:41:31.294 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.294 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.294 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.295 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35806 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2748416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:31.295 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.538719+0000", 2024-09-17T22:41:31.295 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.706485754, 2024-09-17T22:41:31.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016746647999999999, 2024-09-17T22:41:31.295 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.295 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.295 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:31.295 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.295 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.295 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.295 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35806 2024-09-17T22:41:31.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.295 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538719+0000", 2024-09-17T22:41:31.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538719+0000", 2024-09-17T22:41:31.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538720+0000", 2024-09-17T22:41:31.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:41:31.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538722+0000", 2024-09-17T22:41:31.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9429999999999999e-06 2024-09-17T22:41:31.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538727+0000", 2024-09-17T22:41:31.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6559999999999997e-06 2024-09-17T22:41:31.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538732+0000", 2024-09-17T22:41:31.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7949999999999998e-06 2024-09-17T22:41:31.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555297+0000", 2024-09-17T22:41:31.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016564861 2024-09-17T22:41:31.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555358+0000", 2024-09-17T22:41:31.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1177000000000001e-05 2024-09-17T22:41:31.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555466+0000", 2024-09-17T22:41:31.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010785000000000001 2024-09-17T22:41:31.299 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.299 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.299 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.299 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35809 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2410496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:31.299 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.538738+0000", 2024-09-17T22:41:31.299 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.70646736, 2024-09-17T22:41:31.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016923453000000001, 2024-09-17T22:41:31.299 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.299 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.299 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:31.299 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.299 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.299 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.300 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35809 2024-09-17T22:41:31.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.300 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538738+0000", 2024-09-17T22:41:31.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538738+0000", 2024-09-17T22:41:31.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538738+0000", 2024-09-17T22:41:31.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:41:31.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538740+0000", 2024-09-17T22:41:31.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9080000000000002e-06 2024-09-17T22:41:31.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538746+0000", 2024-09-17T22:41:31.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6330000000000001e-06 2024-09-17T22:41:31.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538750+0000", 2024-09-17T22:41:31.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5040000000000004e-06 2024-09-17T22:41:31.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555497+0000", 2024-09-17T22:41:31.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016747171000000002 2024-09-17T22:41:31.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555559+0000", 2024-09-17T22:41:31.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1822999999999994e-05 2024-09-17T22:41:31.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555661+0000", 2024-09-17T22:41:31.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010206 2024-09-17T22:41:31.303 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.303 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.303 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.303 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35810 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2484224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:31.303 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.538756+0000", 2024-09-17T22:41:31.303 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.706448875, 2024-09-17T22:41:31.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017104030999999999, 2024-09-17T22:41:31.303 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.304 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.304 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:31.304 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.304 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.304 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.304 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35810 2024-09-17T22:41:31.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.304 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538756+0000", 2024-09-17T22:41:31.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538756+0000", 2024-09-17T22:41:31.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538757+0000", 2024-09-17T22:41:31.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:41:31.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538759+0000", 2024-09-17T22:41:31.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.778e-06 2024-09-17T22:41:31.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538764+0000", 2024-09-17T22:41:31.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6840000000000001e-06 2024-09-17T22:41:31.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538771+0000", 2024-09-17T22:41:31.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5590000000000002e-06 2024-09-17T22:41:31.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555692+0000", 2024-09-17T22:41:31.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016921281 2024-09-17T22:41:31.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555752+0000", 2024-09-17T22:41:31.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0402000000000002e-05 2024-09-17T22:41:31.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555860+0000", 2024-09-17T22:41:31.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010796100000000001 2024-09-17T22:41:31.307 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.307 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.307 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.308 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35823 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2543616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:31.308 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.539339+0000", 2024-09-17T22:41:31.308 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.7058658580000001, 2024-09-17T22:41:31.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016657568000000001, 2024-09-17T22:41:31.308 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.308 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.308 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:31.308 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.308 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.308 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.308 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35823 2024-09-17T22:41:31.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.309 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539339+0000", 2024-09-17T22:41:31.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539339+0000", 2024-09-17T22:41:31.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539340+0000", 2024-09-17T22:41:31.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2900000000000002e-07 2024-09-17T22:41:31.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539344+0000", 2024-09-17T22:41:31.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0729999999999998e-06 2024-09-17T22:41:31.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539356+0000", 2024-09-17T22:41:31.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1484000000000001e-05 2024-09-17T22:41:31.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539360+0000", 2024-09-17T22:41:31.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7260000000000002e-06 2024-09-17T22:41:31.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555893+0000", 2024-09-17T22:41:31.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016532958 2024-09-17T22:41:31.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555941+0000", 2024-09-17T22:41:31.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7729999999999999e-05 2024-09-17T22:41:31.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555997+0000", 2024-09-17T22:41:31.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5767999999999999e-05 2024-09-17T22:41:31.312 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.312 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.312 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.312 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35824 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2617344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:31.313 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.539370+0000", 2024-09-17T22:41:31.313 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.7058354609999999, 2024-09-17T22:41:31.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016758449000000002, 2024-09-17T22:41:31.313 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.313 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.313 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:31.313 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.313 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.313 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.313 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35824 2024-09-17T22:41:31.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.313 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539370+0000", 2024-09-17T22:41:31.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539370+0000", 2024-09-17T22:41:31.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539370+0000", 2024-09-17T22:41:31.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3799999999999998e-07 2024-09-17T22:41:31.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539372+0000", 2024-09-17T22:41:31.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8160000000000001e-06 2024-09-17T22:41:31.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539376+0000", 2024-09-17T22:41:31.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.844e-06 2024-09-17T22:41:31.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539379+0000", 2024-09-17T22:41:31.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0910000000000001e-06 2024-09-17T22:41:31.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556031+0000", 2024-09-17T22:41:31.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016651750999999999 2024-09-17T22:41:31.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556072+0000", 2024-09-17T22:41:31.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1690999999999998e-05 2024-09-17T22:41:31.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556128+0000", 2024-09-17T22:41:31.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5918000000000002e-05 2024-09-17T22:41:31.317 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.317 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.317 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.317 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35837 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2676736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:31.318 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.539383+0000", 2024-09-17T22:41:31.318 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.705822492, 2024-09-17T22:41:31.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0168665, 2024-09-17T22:41:31.318 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.318 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.318 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:31.318 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.318 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.318 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.318 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35837 2024-09-17T22:41:31.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.318 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539383+0000", 2024-09-17T22:41:31.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539383+0000", 2024-09-17T22:41:31.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539383+0000", 2024-09-17T22:41:31.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2399999999999999e-07 2024-09-17T22:41:31.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539384+0000", 2024-09-17T22:41:31.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.305e-06 2024-09-17T22:41:31.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539387+0000", 2024-09-17T22:41:31.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0939999999999999e-06 2024-09-17T22:41:31.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539390+0000", 2024-09-17T22:41:31.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.678e-06 2024-09-17T22:41:31.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556152+0000", 2024-09-17T22:41:31.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016762353000000001 2024-09-17T22:41:31.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556193+0000", 2024-09-17T22:41:31.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0399999999999999e-05 2024-09-17T22:41:31.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556249+0000", 2024-09-17T22:41:31.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6446000000000001e-05 2024-09-17T22:41:31.322 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.322 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.322 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.322 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35838 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2750464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:31.322 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.539393+0000", 2024-09-17T22:41:31.322 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.7058119729999999, 2024-09-17T22:41:31.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017005407, 2024-09-17T22:41:31.322 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.322 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.322 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:31.323 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.323 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.323 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.323 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35838 2024-09-17T22:41:31.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.323 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539393+0000", 2024-09-17T22:41:31.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539393+0000", 2024-09-17T22:41:31.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539393+0000", 2024-09-17T22:41:31.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.05e-07 2024-09-17T22:41:31.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539395+0000", 2024-09-17T22:41:31.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.11e-06 2024-09-17T22:41:31.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539398+0000", 2024-09-17T22:41:31.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0240000000000002e-06 2024-09-17T22:41:31.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539401+0000", 2024-09-17T22:41:31.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1729999999999999e-06 2024-09-17T22:41:31.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556291+0000", 2024-09-17T22:41:31.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016890179000000002 2024-09-17T22:41:31.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556343+0000", 2024-09-17T22:41:31.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2380999999999999e-05 2024-09-17T22:41:31.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556399+0000", 2024-09-17T22:41:31.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5334999999999998e-05 2024-09-17T22:41:31.326 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.326 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.326 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.327 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35841 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2412544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:31.327 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.539404+0000", 2024-09-17T22:41:31.327 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.7058014240000001, 2024-09-17T22:41:31.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017168085999999999, 2024-09-17T22:41:31.327 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.327 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.327 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:31.327 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.327 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.327 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.327 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35841 2024-09-17T22:41:31.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.328 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539404+0000", 2024-09-17T22:41:31.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539404+0000", 2024-09-17T22:41:31.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539404+0000", 2024-09-17T22:41:31.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0599999999999999e-07 2024-09-17T22:41:31.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539407+0000", 2024-09-17T22:41:31.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3890000000000001e-06 2024-09-17T22:41:31.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539411+0000", 2024-09-17T22:41:31.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2009999999999999e-06 2024-09-17T22:41:31.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539413+0000", 2024-09-17T22:41:31.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6110000000000001e-06 2024-09-17T22:41:31.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556421+0000", 2024-09-17T22:41:31.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017008079999999998 2024-09-17T22:41:31.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556470+0000", 2024-09-17T22:41:31.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8987999999999999e-05 2024-09-17T22:41:31.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556572+0000", 2024-09-17T22:41:31.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101611 2024-09-17T22:41:31.331 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.331 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.331 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.331 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35845 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2486272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:31.331 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.539416+0000", 2024-09-17T22:41:31.331 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.7057887699999998, 2024-09-17T22:41:31.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017358905000000001, 2024-09-17T22:41:31.332 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.332 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.332 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:31.332 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.332 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.332 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.332 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35845 2024-09-17T22:41:31.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.332 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539416+0000", 2024-09-17T22:41:31.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539416+0000", 2024-09-17T22:41:31.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539417+0000", 2024-09-17T22:41:31.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3200000000000001e-07 2024-09-17T22:41:31.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539418+0000", 2024-09-17T22:41:31.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.094e-06 2024-09-17T22:41:31.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539421+0000", 2024-09-17T22:41:31.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0010000000000002e-06 2024-09-17T22:41:31.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539423+0000", 2024-09-17T22:41:31.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5919999999999999e-06 2024-09-17T22:41:31.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556602+0000", 2024-09-17T22:41:31.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017178729 2024-09-17T22:41:31.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556677+0000", 2024-09-17T22:41:31.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4825000000000003e-05 2024-09-17T22:41:31.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556775+0000", 2024-09-17T22:41:31.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8431999999999995e-05 2024-09-17T22:41:31.335 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.335 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.335 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.336 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35870 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2678784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:31.336 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.540812+0000", 2024-09-17T22:41:31.336 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 1.704393372, 2024-09-17T22:41:31.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016561679999999999, 2024-09-17T22:41:31.336 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:31.336 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:31.336 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:31.336 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:31.336 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:31.336 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:31.336 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35870 2024-09-17T22:41:31.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.337 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:31.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:31.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.540812+0000", 2024-09-17T22:41:31.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:31.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.540812+0000", 2024-09-17T22:41:31.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:31.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:31.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.540812+0000", 2024-09-17T22:41:31.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7300000000000001e-07 2024-09-17T22:41:31.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:31.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.540814+0000", 2024-09-17T22:41:31.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1610000000000001e-06 2024-09-17T22:41:31.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:31.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.540821+0000", 2024-09-17T22:41:31.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2249999999999997e-06 2024-09-17T22:41:31.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:31.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.540827+0000", 2024-09-17T22:41:31.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4540000000000004e-06 2024-09-17T22:41:31.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:31.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.557219+0000", 2024-09-17T22:41:31.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016392317 2024-09-17T22:41:31.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:31.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.557290+0000", 2024-09-17T22:41:31.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0239000000000003e-05 2024-09-17T22:41:31.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:31.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:31.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:31.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.557374+0000", 2024-09-17T22:41:31.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3811000000000001e-05 2024-09-17T22:41:31.340 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.340 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.340 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.340 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:31.340 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:31.340 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:31.341 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:41:31.341 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:41:31.341 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-17T22:41:31.341 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-17T22:41:31.342 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0: {} 2024-09-17T22:41:31.343 INFO:teuthology.orchestra.run.smithi115.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:41:31.343 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:41:31.343 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:41:31.344 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:41:31.344 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:31.339+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:31.344 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:41:31.345 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:41:31.355 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:41:31.355 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:41:31.372 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:41:31.372 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:41:31.389 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:41:31.389 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:41:31.406 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:41:31.406 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:41:31.422 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:41:31.422 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:41:31.477 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:31.475+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:31.483 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-17T22:41:31.487 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:31.487 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:31.496 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:41:31.567 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:31.565+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:31.598 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:31.598 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:31.607 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:41:31.650 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:31.648+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:31.656 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-17T22:41:31.710 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:31.711 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:31.719 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:41:31.744 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:31.742+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:31.830 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-17T22:41:31.863 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:31.863 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:31.870 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:31.868+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:31.871 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:41:31.946 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:31.944+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:31.975 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:31.975 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:31.985 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:41:32.007 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-17T22:41:32.039 INFO:teuthology.orchestra.run.smithi115.stdout:leaving image image_client.0-clone4 intact 2024-09-17T22:41:32.068 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:32.066+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:32.128 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:32.128 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:32.137 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:41:32.152 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-17T22:41:32.152 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:32.149+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:32.194 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-17T22:41:32.252 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:32.249+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:32.257 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:32.258 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:32.270 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:41:32.349 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-17T22:41:32.390 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:32.387+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:32.422 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:32.422 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:32.431 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:41:32.471 INFO:teuthology.orchestra.run.smithi115.stdout:410 write 0xea986ec thru 0xeaa0dff (0x8714 bytes) 2024-09-17T22:41:32.472 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:32.469+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:32.475 INFO:teuthology.orchestra.run.smithi115.stdout:411 read 0x1464294 thru 0x146d894 (0x9601 bytes) 2024-09-17T22:41:32.475 INFO:teuthology.orchestra.run.smithi115.stdout:412 read 0x9293e6b thru 0x929485b (0x9f1 bytes) 2024-09-17T22:41:32.476 INFO:teuthology.orchestra.run.smithi115.stdout:414 read 0x7090869 thru 0x7096614 (0x5dac bytes) 2024-09-17T22:41:32.476 INFO:teuthology.orchestra.run.smithi115.stdout:415 read 0x610efb0 thru 0x6118815 (0x9866 bytes) 2024-09-17T22:41:32.476 INFO:teuthology.orchestra.run.smithi115.stdout:416 read 0x14947fb thru 0x1494a3c (0x242 bytes) 2024-09-17T22:41:32.476 INFO:teuthology.orchestra.run.smithi115.stdout:417 write 0xdfa7164 thru 0xdfa898f (0x182c bytes) 2024-09-17T22:41:32.479 INFO:teuthology.orchestra.run.smithi115.stdout:418 write 0x2b744b4 thru 0x2b81a68 (0xd5b5 bytes) 2024-09-17T22:41:32.486 INFO:teuthology.orchestra.run.smithi115.stdout:419 read 0xd3e8c75 thru 0xd3ea12e (0x14ba bytes) 2024-09-17T22:41:32.486 INFO:teuthology.orchestra.run.smithi115.stdout:420 writesame 0x51bfe43 thru 0x51ced91 data_size 0x9(0xef4f bytes) 2024-09-17T22:41:32.493 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:32.493 INFO:teuthology.orchestra.run.smithi115.stdout:422 write 0x44a3cc1 thru 0x44ac51d (0x885d bytes) 2024-09-17T22:41:32.498 INFO:teuthology.orchestra.run.smithi115.stdout:423 write 0xb4cd887 thru 0xb4d3003 (0x577d bytes) 2024-09-17T22:41:32.501 INFO:teuthology.orchestra.run.smithi115.stdout:424 write 0x98cac36 thru 0x98caf5c (0x327 bytes) 2024-09-17T22:41:32.509 INFO:teuthology.orchestra.run.smithi115.stdout:427 trunc from 0xeaa0e00 to 0xbd4aeaa 2024-09-17T22:41:32.528 INFO:teuthology.orchestra.run.smithi115.stdout:428 writesame 0x6405797 thru 0x640b818 data_size 0xb(0x6082 bytes) 2024-09-17T22:41:32.533 INFO:teuthology.orchestra.run.smithi115.stdout:429 punch from 0xe0e46 to 0xe4ecf, (0x4089 bytes) 2024-09-17T22:41:32.533 INFO:teuthology.orchestra.run.smithi115.stdout:430 write 0x5ecabd3 thru 0x5ed8e04 (0xe232 bytes) 2024-09-17T22:41:32.541 INFO:teuthology.orchestra.run.smithi115.stdout:431 writesame 0xbe2a882 thru 0xbe322d5 data_size 0x2(0x7a54 bytes) 2024-09-17T22:41:32.541 INFO:teuthology.orchestra.run.smithi115.stdout:432 write 0x7f76fee thru 0x7f78640 (0x1653 bytes) 2024-09-17T22:41:32.545 INFO:teuthology.orchestra.run.smithi115.stdout:433 punch from 0x925ac6d to 0x9267a26, (0xcdb9 bytes) 2024-09-17T22:41:32.545 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:32.545 INFO:teuthology.orchestra.run.smithi115.stdout:436 write 0x3e3153d thru 0x3e390ad (0x7b71 bytes) 2024-09-17T22:41:32.547 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-17T22:41:32.565 INFO:teuthology.orchestra.run.smithi115.stdout:437 trunc from 0xbe322d6 to 0xb07aa 2024-09-17T22:41:32.565 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:32.562+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:32.600 INFO:teuthology.orchestra.run.smithi115.stdout:440 read 0x97d24 thru 0xa089a (0x8b77 bytes) 2024-09-17T22:41:32.600 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:32.604 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:32.604 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:32.617 INFO:teuthology.orchestra.run.smithi115.stdout:442 trunc from 0xb07aa to 0xe344f02 2024-09-17T22:41:32.619 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:41:32.633 INFO:teuthology.orchestra.run.smithi115.stdout:443 trunc from 0xe344f02 to 0x5ff8091 2024-09-17T22:41:32.649 INFO:teuthology.orchestra.run.smithi115.stdout:444 write 0x74908af thru 0x749e612 (0xdd64 bytes) 2024-09-17T22:41:32.653 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:32.650+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:32.666 INFO:teuthology.orchestra.run.smithi115.stdout:445 writesame 0xbb2c546 thru 0xbb3ad3c data_size 0x1(0xe7f7 bytes) 2024-09-17T22:41:32.666 INFO:teuthology.orchestra.run.smithi115.stdout:446 write 0x760c5f thru 0x76f601 (0xe9a3 bytes) 2024-09-17T22:41:32.670 INFO:teuthology.orchestra.run.smithi115.stdout:447 trunc from 0xbb3ad3d to 0xaefbf9d 2024-09-17T22:41:32.675 INFO:teuthology.orchestra.run.smithi115.stdout:448 writesame 0xf46238 thru 0xf4e518 data_size 0x1(0x82e1 bytes) 2024-09-17T22:41:32.693 INFO:teuthology.orchestra.run.smithi115.stdout:449 read 0x28a0fbc thru 0x28a41b5 (0x31fa bytes) 2024-09-17T22:41:32.701 INFO:teuthology.orchestra.run.smithi115.stdout:451 write 0xc2bad4d thru 0xc2c424e (0x9502 bytes) 2024-09-17T22:41:32.708 INFO:teuthology.orchestra.run.smithi115.stdout:452 read 0x5b747f4 thru 0x5b7864e (0x3e5b bytes) 2024-09-17T22:41:32.708 INFO:teuthology.orchestra.run.smithi115.stdout:453 write 0x16445ec thru 0x1654540 (0xff55 bytes) 2024-09-17T22:41:32.713 INFO:teuthology.orchestra.run.smithi115.stdout:455 read 0x3441698 thru 0x344177b (0xe4 bytes) 2024-09-17T22:41:32.725 INFO:teuthology.orchestra.run.smithi115.stdout:456 trunc from 0xc2c424f to 0x1925b90 2024-09-17T22:41:32.746 INFO:teuthology.orchestra.run.smithi115.stdout:458 trunc from 0x1925b90 to 0x96305ae 2024-09-17T22:41:32.753 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:32.751+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:32.756 INFO:teuthology.orchestra.run.smithi115.stdout:460 writesame 0x9e44ded thru 0x9e5072e data_size 0x17(0xb942 bytes) 2024-09-17T22:41:32.757 INFO:teuthology.orchestra.run.smithi115.stdout:462 punch from 0x44a1358 to 0x44a18dd, (0x585 bytes) 2024-09-17T22:41:32.757 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:32.757 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:32.757 INFO:teuthology.orchestra.run.smithi115.stdout:466 writesame 0x3f31b1e thru 0x3f3cbfc data_size 0x9(0xb0df bytes) 2024-09-17T22:41:32.760 INFO:teuthology.orchestra.run.smithi115.stdout:468 read 0x68f9f70 thru 0x6907fc6 (0xe057 bytes) 2024-09-17T22:41:32.761 INFO:teuthology.orchestra.run.smithi115.stdout:469 write 0x5b2a89e thru 0x5b3a4bf (0xfc22 bytes) 2024-09-17T22:41:32.764 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:32.764 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:32.764 INFO:teuthology.orchestra.run.smithi115.stdout:470 trunc from 0x9e5072f to 0x904d3da 2024-09-17T22:41:32.769 INFO:teuthology.orchestra.run.smithi115.stdout:471 read 0x827151b thru 0x827d9db (0xc4c1 bytes) 2024-09-17T22:41:32.769 INFO:teuthology.orchestra.run.smithi115.stdout:474 read 0x376bb96 thru 0x37723d8 (0x6843 bytes) 2024-09-17T22:41:32.769 INFO:teuthology.orchestra.run.smithi115.stdout:475 write 0x84e78a5 thru 0x84ec1f1 (0x494d bytes) 2024-09-17T22:41:32.772 INFO:teuthology.orchestra.run.smithi115.stdout:476 punch from 0x7a11a4e to 0x7a1ded4, (0xc486 bytes) 2024-09-17T22:41:32.773 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:32.773 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:32.773 INFO:teuthology.orchestra.run.smithi115.stdout:481 write 0x34c4314 thru 0x34c6226 (0x1f13 bytes) 2024-09-17T22:41:32.775 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:41:32.777 INFO:teuthology.orchestra.run.smithi115.stdout:482 write 0x79d3544 thru 0x79da07e (0x6b3b bytes) 2024-09-17T22:41:32.779 INFO:teuthology.orchestra.run.smithi115.stdout:483 read 0x34ee911 thru 0x34f0b42 (0x2232 bytes) 2024-09-17T22:41:32.786 INFO:teuthology.orchestra.run.smithi115.stdout:484 trunc from 0x904d3da to 0x359963a 2024-09-17T22:41:32.804 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:32.814 INFO:teuthology.orchestra.run.smithi115.stdout:486 write 0x6792630 thru 0x679ac33 (0x8604 bytes) 2024-09-17T22:41:32.818 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:32.818 INFO:teuthology.orchestra.run.smithi115.stdout:489 writesame 0x12d1c4b thru 0x12dc7d8 data_size 0x7(0xab8e bytes) 2024-09-17T22:41:32.823 INFO:teuthology.orchestra.run.smithi115.stdout:490 writesame 0x6745edf thru 0x67539ad data_size 0x11(0xdacf bytes) 2024-09-17T22:41:32.838 INFO:teuthology.orchestra.run.smithi115.stdout:491 write 0xb6f698b thru 0xb6ff4c1 (0x8b37 bytes) 2024-09-17T22:41:32.843 INFO:teuthology.orchestra.run.smithi115.stdout:492 read 0x382c441 thru 0x382c6f3 (0x2b3 bytes) 2024-09-17T22:41:32.845 INFO:teuthology.orchestra.run.smithi115.stdout:495 trunc from 0xb6ff4c2 to 0x977c8a2 2024-09-17T22:41:32.854 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:32.851+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:32.855 INFO:teuthology.orchestra.run.smithi115.stdout:496 read 0x84944a9 thru 0x84979e9 (0x3541 bytes) 2024-09-17T22:41:32.855 INFO:teuthology.orchestra.run.smithi115.stdout:497 punch from 0x67d2a8e to 0x67e26f4, (0xfc66 bytes) 2024-09-17T22:41:32.855 INFO:teuthology.orchestra.run.smithi115.stdout:498 write 0x4cc577f thru 0x4cc7f4e (0x27d0 bytes) 2024-09-17T22:41:32.857 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:32.862 INFO:teuthology.orchestra.run.smithi115.stdout:500 trunc from 0x977c8a2 to 0xd5ec0e2 2024-09-17T22:41:32.866 INFO:teuthology.orchestra.run.smithi115.stdout:501 punch from 0x6fbc0f to 0x7094b0, (0xd8a1 bytes) 2024-09-17T22:41:32.872 INFO:teuthology.orchestra.run.smithi115.stdout:503 trunc from 0xd5ec0e2 to 0x9953a57 2024-09-17T22:41:32.877 INFO:teuthology.orchestra.run.smithi115.stdout:504 read 0x5abfa2c thru 0x5ac9fc9 (0xa59e bytes) 2024-09-17T22:41:32.877 INFO:teuthology.orchestra.run.smithi115.stdout:505 punch from 0x77b334b to 0x77c2dd6, (0xfa8b bytes) 2024-09-17T22:41:32.877 INFO:teuthology.orchestra.run.smithi115.stdout:506 punch from 0x1b3e049 to 0x1b405b6, (0x256d bytes) 2024-09-17T22:41:32.877 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:32.877 INFO:teuthology.orchestra.run.smithi115.stdout:510 punch from 0x36f4b4c to 0x36ff349, (0xa7fd bytes) 2024-09-17T22:41:32.877 INFO:teuthology.orchestra.run.smithi115.stdout:511 read 0x6fcd0ee thru 0x6fda56f (0xd482 bytes) 2024-09-17T22:41:32.877 INFO:teuthology.orchestra.run.smithi115.stdout:514 trunc from 0x9953a57 to 0x9c6144b 2024-09-17T22:41:32.881 INFO:teuthology.orchestra.run.smithi115.stdout:515 clone 6 order 22 su 262144 sc 7 2024-09-17T22:41:32.896 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:32.896 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:32.904 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:41:32.955 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:32.953+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:33.000 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:41:33.055 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone5 from 0x9c6144b (overlap 0xb07aa) to 0x0 2024-09-17T22:41:33.056 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:33.053+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:33.058 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:33.059 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:33.070 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:41:33.123 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-17T22:41:33.156 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:33.153+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:33.203 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:33.203 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:33.222 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:41:33.265 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:33.263+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:33.358 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:33.355+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:33.404 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:33.404 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:33.419 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:41:33.437 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:41:33.461 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:33.458+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:33.560 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:33.557+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:33.570 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:33.570 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:33.579 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:41:33.587 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0: {} 2024-09-17T22:41:33.587 INFO:teuthology.orchestra.run.smithi115.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:41:33.606 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:41:33.606 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:41:33.622 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:41:33.622 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:41:33.639 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:41:33.639 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:41:33.655 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:41:33.656 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:41:33.656 INFO:teuthology.orchestra.run.smithi115.stdout:516 writesame 0xa5e42b thru 0xa63638 data_size 0x3(0x520e bytes) 2024-09-17T22:41:33.661 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:33.658+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:33.664 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:33.664 INFO:teuthology.orchestra.run.smithi115.stdout:518 read 0x6c158f6 thru 0x6c1abe8 (0x52f3 bytes) 2024-09-17T22:41:33.664 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:33.664 INFO:teuthology.orchestra.run.smithi115.stdout:521 writesame 0x35d9da2 thru 0x35e8a7f data_size 0x1(0xecde bytes) 2024-09-17T22:41:33.673 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:41:33.673 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:41:33.691 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:41:33.691 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:41:33.694 INFO:teuthology.orchestra.run.smithi115.stdout:523 read 0x8ae631f thru 0x8af1cd0 (0xb9b2 bytes) 2024-09-17T22:41:33.694 INFO:teuthology.orchestra.run.smithi115.stdout:524 write 0x5f7d997 thru 0x5f808be (0x2f28 bytes) 2024-09-17T22:41:33.695 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:33.695 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:33.695 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:41:33.695 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:33.704 INFO:teuthology.orchestra.run.smithi115.stdout:525 trunc from 0x9c6144b to 0x678c571 2024-09-17T22:41:33.705 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:41:33.710 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:41:33.710 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:41:33.723 INFO:teuthology.orchestra.run.smithi115.stdout:526 read 0x2cc6a94 thru 0x2ccaad5 (0x4042 bytes) 2024-09-17T22:41:33.723 INFO:teuthology.orchestra.run.smithi115.stdout:528 read 0x205a881 thru 0x205c66a (0x1dea bytes) 2024-09-17T22:41:33.723 INFO:teuthology.orchestra.run.smithi115.stdout:529 write 0x121cf60 thru 0x1223893 (0x6934 bytes) 2024-09-17T22:41:33.743 INFO:teuthology.orchestra.run.smithi115.stdout:530 write 0xdfac472 thru 0xdfae97b (0x250a bytes) 2024-09-17T22:41:33.746 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:33.749 INFO:teuthology.orchestra.run.smithi115.stdout:532 trunc from 0xdfae97c to 0xb80b017 2024-09-17T22:41:33.757 INFO:teuthology.orchestra.run.smithi115.stdout:533 write 0x2efce74 thru 0x2eff4ea (0x2677 bytes) 2024-09-17T22:41:33.759 INFO:teuthology.orchestra.run.smithi115.stdout:535 punch from 0x2354f26 to 0x236292f, (0xda09 bytes) 2024-09-17T22:41:33.759 INFO:teuthology.orchestra.run.smithi115.stdout:536 punch from 0xb2c3c35 to 0xb2c3e3f, (0x20a bytes) 2024-09-17T22:41:33.759 INFO:teuthology.orchestra.run.smithi115.stdout:537 read 0x6ed07fe thru 0x6ed920c (0x8a0f bytes) 2024-09-17T22:41:33.760 INFO:teuthology.orchestra.run.smithi115.stdout:538 read 0x1d53963 thru 0x1d58993 (0x5031 bytes) 2024-09-17T22:41:33.762 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:33.759+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:33.767 INFO:teuthology.orchestra.run.smithi115.stdout:540 trunc from 0xb80b017 to 0x518ea19 2024-09-17T22:41:33.799 INFO:teuthology.orchestra.run.smithi115.stdout:542 write 0x6f26986 thru 0x6f32fa4 (0xc61f bytes) 2024-09-17T22:41:33.802 INFO:teuthology.orchestra.run.smithi115.stdout:543 write 0x2b8320b thru 0x2b8db7e (0xa974 bytes) 2024-09-17T22:41:33.806 INFO:teuthology.orchestra.run.smithi115.stdout:545 read 0x69a1378 thru 0x69ad53b (0xc1c4 bytes) 2024-09-17T22:41:33.813 INFO:teuthology.orchestra.run.smithi115.stdout:546 writesame 0x89d3faf thru 0x89dbb5c data_size 0x9(0x7bae bytes) 2024-09-17T22:41:33.819 INFO:teuthology.orchestra.run.smithi115.stdout:547 trunc from 0x89dbb5d to 0xdc598d9 2024-09-17T22:41:33.825 INFO:teuthology.orchestra.run.smithi115.stdout:548 write 0x2c89576 thru 0x2c909bf (0x744a bytes) 2024-09-17T22:41:33.829 INFO:teuthology.orchestra.run.smithi115.stdout:549 read 0x40fa9e thru 0x417712 (0x7c75 bytes) 2024-09-17T22:41:33.829 INFO:teuthology.orchestra.run.smithi115.stdout:550 read 0x8727c4e thru 0x872aaee (0x2ea1 bytes) 2024-09-17T22:41:33.829 INFO:teuthology.orchestra.run.smithi115.stdout:553 read 0x3a0acd thru 0x3a2c77 (0x21ab bytes) 2024-09-17T22:41:33.829 INFO:teuthology.orchestra.run.smithi115.stdout:554 write 0x524bc28 thru 0x5251f53 (0x632c bytes) 2024-09-17T22:41:33.830 INFO:teuthology.orchestra.run.smithi115.stdout:555 punch from 0xa6521b0 to 0xa653abc, (0x190c bytes) 2024-09-17T22:41:33.830 INFO:teuthology.orchestra.run.smithi115.stdout:556 write 0x95d4c27 thru 0x95e2247 (0xd621 bytes) 2024-09-17T22:41:33.831 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:33.831 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:33.832 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:41:33.832 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:41:33.832 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:33.833 INFO:teuthology.orchestra.run.smithi115.stdout:557 punch from 0x67f42e5 to 0x67fd037, (0x8d52 bytes) 2024-09-17T22:41:33.833 INFO:teuthology.orchestra.run.smithi115.stdout:558 write 0xc47cec thru 0xc52bcd (0xaee2 bytes) 2024-09-17T22:41:33.837 INFO:teuthology.orchestra.run.smithi115.stdout:560 write 0x2f3b75 thru 0x2f904a (0x54d6 bytes) 2024-09-17T22:41:33.841 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:41:33.842 INFO:teuthology.orchestra.run.smithi115.stdout:561 writesame 0xd862528 thru 0xd86b2eb data_size 0x2b(0x8dc4 bytes) 2024-09-17T22:41:33.845 INFO:teuthology.orchestra.run.smithi115.stdout:562 punch from 0xd91b9d2 to 0xd92707d, (0xb6ab bytes) 2024-09-17T22:41:33.845 INFO:teuthology.orchestra.run.smithi115.stdout:564 read 0xa165973 thru 0xa166403 (0xa91 bytes) 2024-09-17T22:41:33.859 INFO:teuthology.orchestra.run.smithi115.stdout:565 trunc from 0xdc598d9 to 0x2140fc1 2024-09-17T22:41:33.863 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:33.860+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:33.887 INFO:teuthology.orchestra.run.smithi115.stdout:566 trunc from 0x2140fc1 to 0x904181c 2024-09-17T22:41:33.891 INFO:teuthology.orchestra.run.smithi115.stdout:567 read 0xab8bdb thru 0xabe38e (0x57b4 bytes) 2024-09-17T22:41:33.892 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:33.892 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:33.898 INFO:teuthology.orchestra.run.smithi115.stdout:573 write 0xa423ca3 thru 0xa42bcc5 (0x8023 bytes) 2024-09-17T22:41:33.901 INFO:teuthology.orchestra.run.smithi115.stdout:574 write 0x906cebf thru 0x90794f4 (0xc636 bytes) 2024-09-17T22:41:33.904 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:33.905 INFO:teuthology.orchestra.run.smithi115.stdout:576 write 0x6323f9f thru 0x6328d18 (0x4d7a bytes) 2024-09-17T22:41:33.915 INFO:teuthology.orchestra.run.smithi115.stdout:577 trunc from 0xa42bcc6 to 0x3b636a9 2024-09-17T22:41:33.924 INFO:teuthology.orchestra.run.smithi115.stdout:578 read 0x21abbb thru 0x222fe7 (0x842d bytes) 2024-09-17T22:41:33.924 INFO:teuthology.orchestra.run.smithi115.stdout:579 read 0x4a06a8 thru 0x4a395f (0x32b8 bytes) 2024-09-17T22:41:33.924 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:33.924 INFO:teuthology.orchestra.run.smithi115.stdout:581 read 0x334acce thru 0x3356e69 (0xc19c bytes) 2024-09-17T22:41:33.932 INFO:teuthology.orchestra.run.smithi115.stdout:582 trunc from 0x3b636a9 to 0x9af2fac 2024-09-17T22:41:33.937 INFO:teuthology.orchestra.run.smithi115.stdout:583 punch from 0x9a0915f to 0x9a0e190, (0x5031 bytes) 2024-09-17T22:41:33.937 INFO:teuthology.orchestra.run.smithi115.stdout:584 write 0x7f43766 thru 0x7f51e74 (0xe70f bytes) 2024-09-17T22:41:33.941 INFO:teuthology.orchestra.run.smithi115.stdout:585 read 0x7f2c6cc thru 0x7f35b9b (0x94d0 bytes) 2024-09-17T22:41:33.941 INFO:teuthology.orchestra.run.smithi115.stdout:586 write 0x7fb153c thru 0x7fb9d92 (0x8857 bytes) 2024-09-17T22:41:33.944 INFO:teuthology.orchestra.run.smithi115.stdout:587 write 0x7edb thru 0xc8b6 (0x49dc bytes) 2024-09-17T22:41:33.946 INFO:teuthology.orchestra.run.smithi115.stdout:588 punch from 0x740a7af to 0x740c8aa, (0x20fb bytes) 2024-09-17T22:41:33.946 INFO:teuthology.orchestra.run.smithi115.stdout:590 punch from 0x401810d to 0x40226be, (0xa5b1 bytes) 2024-09-17T22:41:33.946 INFO:teuthology.orchestra.run.smithi115.stdout:591 writesame 0x778fbf thru 0x7827b8 data_size 0x7(0x97fa bytes) 2024-09-17T22:41:33.952 INFO:teuthology.orchestra.run.smithi115.stdout:592 punch from 0x307a8f7 to 0x3086ecc, (0xc5d5 bytes) 2024-09-17T22:41:33.952 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:33.952 INFO:teuthology.orchestra.run.smithi115.stdout:594 punch from 0x14e9fce to 0x14f0706, (0x6738 bytes) 2024-09-17T22:41:33.952 INFO:teuthology.orchestra.run.smithi115.stdout:595 writesame 0x1c75357 thru 0x1c7abd8 data_size 0x1(0x5882 bytes) 2024-09-17T22:41:33.961 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:33.962 INFO:teuthology.orchestra.run.smithi115.stdout:597 read 0x1b2c22f thru 0x1b3717c (0xaf4e bytes) 2024-09-17T22:41:33.962 INFO:teuthology.orchestra.run.smithi115.stdout:598 read 0x3b26357 thru 0x3b27537 (0x11e1 bytes) 2024-09-17T22:41:33.962 INFO:teuthology.orchestra.run.smithi115.stdout:599 write 0x13fb974 thru 0x14068a2 (0xaf2f bytes) 2024-09-17T22:41:33.964 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:33.960+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:33.966 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:33.966 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:41:33.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:41:33.966 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:41:33.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4229.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-17T22:41:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:41:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.568407314000002, 2024-09-17T22:41:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25708877200000002, 2024-09-17T22:41:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-17T22:41:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4229", 2024-09-17T22:41:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/3080427038", 2024-09-17T22:41:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2 2024-09-17T22:41:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.967 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:41:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:41:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394611+0000", 2024-09-17T22:41:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3149999999999999e-06 2024-09-17T22:41:33.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394619+0000", 2024-09-17T22:41:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240000000000003e-06 2024-09-17T22:41:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394632+0000", 2024-09-17T22:41:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2921e-05 2024-09-17T22:41:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:33.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394639+0000", 2024-09-17T22:41:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-09-17T22:41:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394703+0000", 2024-09-17T22:41:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4407999999999997e-05 2024-09-17T22:41:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:41:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394711+0000", 2024-09-17T22:41:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5939999999999996e-06 2024-09-17T22:41:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651545+0000", 2024-09-17T22:41:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25683409200000001 2024-09-17T22:41:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651698+0000", 2024-09-17T22:41:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000153678 2024-09-17T22:41:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38736 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:41:33.971 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.185811309, 2024-09-17T22:41:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051395418999999998, 2024-09-17T22:41:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38736 2024-09-17T22:41:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:33.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:41:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:41:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:41:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6899999999999998e-07 2024-09-17T22:41:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:33.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777212+0000", 2024-09-17T22:41:33.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1240000000000002e-06 2024-09-17T22:41:33.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:33.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777219+0000", 2024-09-17T22:41:33.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5409999999999997e-06 2024-09-17T22:41:33.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:33.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777224+0000", 2024-09-17T22:41:33.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:41:33.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.826131+0000", 2024-09-17T22:41:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048907369999999999 2024-09-17T22:41:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828457+0000", 2024-09-17T22:41:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023256209999999999 2024-09-17T22:41:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828601+0000", 2024-09-17T22:41:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014438799999999999 2024-09-17T22:41:33.975 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38752 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 673792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:41:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1857872379999996, 2024-09-17T22:41:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051582301999999997, 2024-09-17T22:41:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:33.976 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38752 2024-09-17T22:41:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:41:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:41:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:41:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3000000000000001e-07 2024-09-17T22:41:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777232+0000", 2024-09-17T22:41:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9709999999999998e-06 2024-09-17T22:41:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777238+0000", 2024-09-17T22:41:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7209999999999999e-06 2024-09-17T22:41:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:33.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777243+0000", 2024-09-17T22:41:33.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5210000000000004e-06 2024-09-17T22:41:33.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:33.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828633+0000", 2024-09-17T22:41:33.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051390915000000002 2024-09-17T22:41:33.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:33.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828711+0000", 2024-09-17T22:41:33.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7474999999999999e-05 2024-09-17T22:41:33.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:33.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828812+0000", 2024-09-17T22:41:33.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010116900000000001 2024-09-17T22:41:33.980 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:33.980 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:33.980 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:33.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.980 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38753 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:33.980 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:41:33.980 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1857679650000001, 2024-09-17T22:41:33.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051770444999999998, 2024-09-17T22:41:33.980 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:33.980 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:33.980 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:33.980 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:33.981 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:33.981 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:33.981 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38753 2024-09-17T22:41:33.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.981 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:33.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:33.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:41:33.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:33.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:33.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:41:33.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:33.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:33.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:41:33.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:41:33.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:33.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777251+0000", 2024-09-17T22:41:33.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8059999999999999e-06 2024-09-17T22:41:33.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:33.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777257+0000", 2024-09-17T22:41:33.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2480000000000001e-06 2024-09-17T22:41:33.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:33.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777262+0000", 2024-09-17T22:41:33.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000004e-06 2024-09-17T22:41:33.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:33.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828842+0000", 2024-09-17T22:41:33.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051580203999999998 2024-09-17T22:41:33.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:33.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828917+0000", 2024-09-17T22:41:33.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4980000000000004e-05 2024-09-17T22:41:33.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:33.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829019+0000", 2024-09-17T22:41:33.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102248 2024-09-17T22:41:33.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:33.984 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:33.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:33.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.984 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38766 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:33.984 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:41:33.984 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1857489799999996, 2024-09-17T22:41:33.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051958839999999999, 2024-09-17T22:41:33.985 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:33.985 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:33.985 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:33.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:33.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:33.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:33.985 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38766 2024-09-17T22:41:33.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.985 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:33.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:33.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:41:33.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:33.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:33.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:41:33.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:33.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:33.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777269+0000", 2024-09-17T22:41:33.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:41:33.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:33.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777271+0000", 2024-09-17T22:41:33.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-17T22:41:33.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:33.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777276+0000", 2024-09-17T22:41:33.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:41:33.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:33.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777281+0000", 2024-09-17T22:41:33.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:41:33.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:33.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829055+0000", 2024-09-17T22:41:33.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051773570999999997 2024-09-17T22:41:33.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:33.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829128+0000", 2024-09-17T22:41:33.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3131000000000002e-05 2024-09-17T22:41:33.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:33.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829227+0000", 2024-09-17T22:41:33.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8800000000000003e-05 2024-09-17T22:41:33.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:33.988 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:33.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:33.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.989 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38767 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:33.989 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:41:33.989 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.185728997, 2024-09-17T22:41:33.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137018, 2024-09-17T22:41:33.989 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:33.989 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:33.989 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:33.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:33.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:33.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:33.989 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38767 2024-09-17T22:41:33.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.990 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:33.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:33.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:41:33.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:33.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:33.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:41:33.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:33.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:33.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:41:33.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:41:33.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:33.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777290+0000", 2024-09-17T22:41:33.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.88e-06 2024-09-17T22:41:33.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:33.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777296+0000", 2024-09-17T22:41:33.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-09-17T22:41:33.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:33.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777300+0000", 2024-09-17T22:41:33.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4510000000000002e-06 2024-09-17T22:41:33.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:33.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829257+0000", 2024-09-17T22:41:33.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051956975000000002 2024-09-17T22:41:33.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:33.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829328+0000", 2024-09-17T22:41:33.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1026999999999996e-05 2024-09-17T22:41:33.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:33.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829425+0000", 2024-09-17T22:41:33.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.679e-05 2024-09-17T22:41:33.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:33.993 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:33.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:33.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.993 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38784 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 675840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:33.993 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:41:33.993 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1857105429999999, 2024-09-17T22:41:33.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052344522999999997, 2024-09-17T22:41:33.993 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:33.993 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:33.993 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:33.993 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:33.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:33.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:33.994 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38784 2024-09-17T22:41:33.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.994 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:33.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:33.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:41:33.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:33.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:33.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:41:33.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:33.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:33.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777307+0000", 2024-09-17T22:41:33.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:41:33.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:33.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777309+0000", 2024-09-17T22:41:33.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.063e-06 2024-09-17T22:41:33.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:33.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777314+0000", 2024-09-17T22:41:33.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-17T22:41:33.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:33.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777319+0000", 2024-09-17T22:41:33.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999996e-06 2024-09-17T22:41:33.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:33.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829456+0000", 2024-09-17T22:41:33.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137151 2024-09-17T22:41:33.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:33.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829527+0000", 2024-09-17T22:41:33.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1588000000000006e-05 2024-09-17T22:41:33.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:33.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829651+0000", 2024-09-17T22:41:33.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012365 2024-09-17T22:41:33.997 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:33.997 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:33.997 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:33.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.997 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38785 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 749568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:33.997 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:41:33.998 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1856924280000003, 2024-09-17T22:41:33.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052546572999999999, 2024-09-17T22:41:33.998 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:33.998 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:33.998 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:33.998 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:33.998 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:33.998 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:33.998 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38785 2024-09-17T22:41:33.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.998 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:33.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:33.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:41:33.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:33.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:33.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:41:33.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:33.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:33.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:41:33.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:41:33.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:33.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:33.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:33.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777327+0000", 2024-09-17T22:41:33.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:41:34.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:34.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777333+0000", 2024-09-17T22:41:34.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7309999999999998e-06 2024-09-17T22:41:34.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777337+0000", 2024-09-17T22:41:34.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:41:34.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:34.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829686+0000", 2024-09-17T22:41:34.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052349159999999999 2024-09-17T22:41:34.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:34.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829766+0000", 2024-09-17T22:41:34.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9326999999999995e-05 2024-09-17T22:41:34.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:34.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829871+0000", 2024-09-17T22:41:34.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000105466 2024-09-17T22:41:34.001 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.001 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:34.001 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.002 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38798 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:34.002 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:41:34.002 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1850588200000001, 2024-09-17T22:41:34.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052117564999999998, 2024-09-17T22:41:34.002 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:34.002 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:34.002 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:34.002 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:34.002 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:34.002 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.002 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38798 2024-09-17T22:41:34.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.002 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:34.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:34.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:41:34.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:34.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:41:34.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:34.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777959+0000", 2024-09-17T22:41:34.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9599999999999999e-07 2024-09-17T22:41:34.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:34.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777963+0000", 2024-09-17T22:41:34.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2660000000000003e-06 2024-09-17T22:41:34.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:34.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777971+0000", 2024-09-17T22:41:34.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3489999999999992e-06 2024-09-17T22:41:34.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777978+0000", 2024-09-17T22:41:34.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:41:34.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:34.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829906+0000", 2024-09-17T22:41:34.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051927635999999999 2024-09-17T22:41:34.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:34.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829986+0000", 2024-09-17T22:41:34.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0444000000000004e-05 2024-09-17T22:41:34.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:34.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830076+0000", 2024-09-17T22:41:34.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9473000000000002e-05 2024-09-17T22:41:34.005 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.005 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:34.006 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.006 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38799 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 882688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:34.006 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:41:34.006 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1850273810000003, 2024-09-17T22:41:34.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052359606000000003, 2024-09-17T22:41:34.006 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:34.006 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:34.006 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:34.006 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:34.006 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:34.006 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.006 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38799 2024-09-17T22:41:34.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.006 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:34.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:34.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:41:34.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:34.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:41:34.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:34.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:41:34.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8299999999999997e-07 2024-09-17T22:41:34.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:34.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777992+0000", 2024-09-17T22:41:34.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.176e-06 2024-09-17T22:41:34.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:34.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777998+0000", 2024-09-17T22:41:34.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.173e-06 2024-09-17T22:41:34.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778003+0000", 2024-09-17T22:41:34.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6349999999999997e-06 2024-09-17T22:41:34.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:34.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830112+0000", 2024-09-17T22:41:34.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052109366999999997 2024-09-17T22:41:34.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:34.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830192+0000", 2024-09-17T22:41:34.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9814000000000005e-05 2024-09-17T22:41:34.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:34.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830349+0000", 2024-09-17T22:41:34.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015695800000000001 2024-09-17T22:41:34.010 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.010 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:34.010 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.010 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38814 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:34.010 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:41:34.010 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1847439770000001, 2024-09-17T22:41:34.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052274339000000003, 2024-09-17T22:41:34.010 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:34.010 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:34.010 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:34.010 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:34.010 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:34.010 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.011 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38814 2024-09-17T22:41:34.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.011 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:34.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:34.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:41:34.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:34.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:41:34.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:34.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:41:34.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:41:34.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:34.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778276+0000", 2024-09-17T22:41:34.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2029999999999999e-06 2024-09-17T22:41:34.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:34.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778282+0000", 2024-09-17T22:41:34.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0059999999999998e-06 2024-09-17T22:41:34.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778287+0000", 2024-09-17T22:41:34.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0039999999999999e-06 2024-09-17T22:41:34.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:34.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830382+0000", 2024-09-17T22:41:34.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052095379999999997 2024-09-17T22:41:34.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:34.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830455+0000", 2024-09-17T22:41:34.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3127000000000006e-05 2024-09-17T22:41:34.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:34.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830547+0000", 2024-09-17T22:41:34.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2207999999999995e-05 2024-09-17T22:41:34.014 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.014 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:34.014 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.014 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38828 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 163840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:34.014 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:41:34.014 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1845615690000004, 2024-09-17T22:41:34.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052466039999999999, 2024-09-17T22:41:34.014 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:34.014 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:34.014 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:34.014 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:34.015 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:34.015 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.015 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38828 2024-09-17T22:41:34.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.015 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:34.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:34.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:41:34.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:34.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:41:34.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:34.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778456+0000", 2024-09-17T22:41:34.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7599999999999996e-07 2024-09-17T22:41:34.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:34.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778460+0000", 2024-09-17T22:41:34.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8229999999999997e-06 2024-09-17T22:41:34.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:34.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778470+0000", 2024-09-17T22:41:34.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8199999999999992e-06 2024-09-17T22:41:34.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778476+0000", 2024-09-17T22:41:34.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2589999999999997e-06 2024-09-17T22:41:34.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:34.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830578+0000", 2024-09-17T22:41:34.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052102273999999997 2024-09-17T22:41:34.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:34.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830667+0000", 2024-09-17T22:41:34.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8701000000000003e-05 2024-09-17T22:41:34.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:34.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830921+0000", 2024-09-17T22:41:34.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025438700000000002 2024-09-17T22:41:34.018 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.018 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:34.018 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.018 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38829 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:34.018 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:41:34.018 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1845301719999997, 2024-09-17T22:41:34.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052648882000000001, 2024-09-17T22:41:34.019 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:34.019 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:34.019 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:34.019 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:34.019 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:34.019 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.019 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38829 2024-09-17T22:41:34.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.019 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:34.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:34.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:41:34.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:34.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:41:34.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:34.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:41:34.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7e-07 2024-09-17T22:41:34.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:34.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778490+0000", 2024-09-17T22:41:34.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:41:34.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:34.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778495+0000", 2024-09-17T22:41:34.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9329999999999998e-06 2024-09-17T22:41:34.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778500+0000", 2024-09-17T22:41:34.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9509999999999997e-06 2024-09-17T22:41:34.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:34.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830958+0000", 2024-09-17T22:41:34.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05245785 2024-09-17T22:41:34.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:34.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831051+0000", 2024-09-17T22:41:34.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2989e-05 2024-09-17T22:41:34.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:34.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831136+0000", 2024-09-17T22:41:34.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4448000000000002e-05 2024-09-17T22:41:34.022 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.022 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:34.022 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.022 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38891 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:34.022 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:41:34.023 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.181218114, 2024-09-17T22:41:34.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050847934999999997, 2024-09-17T22:41:34.023 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:34.023 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:34.023 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:34.023 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:34.023 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:34.023 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.023 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38891 2024-09-17T22:41:34.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.023 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:34.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:34.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:41:34.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:34.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:41:34.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:34.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:41:34.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-09-17T22:41:34.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:34.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781802+0000", 2024-09-17T22:41:34.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:41:34.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:34.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781810+0000", 2024-09-17T22:41:34.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8019999999999992e-06 2024-09-17T22:41:34.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781816+0000", 2024-09-17T22:41:34.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.285e-06 2024-09-17T22:41:34.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:34.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832485+0000", 2024-09-17T22:41:34.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050668462999999997 2024-09-17T22:41:34.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:34.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832565+0000", 2024-09-17T22:41:34.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9692999999999997e-05 2024-09-17T22:41:34.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:34.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832647+0000", 2024-09-17T22:41:34.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2224000000000004e-05 2024-09-17T22:41:34.026 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.026 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:34.026 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.027 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38892 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:34.027 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:41:34.027 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1811898650000003, 2024-09-17T22:41:34.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050946120999999997, 2024-09-17T22:41:34.027 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:34.027 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:34.027 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:34.027 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:34.027 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:34.027 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.027 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38892 2024-09-17T22:41:34.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.027 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:34.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:34.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:41:34.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:34.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:41:34.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:34.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781828+0000", 2024-09-17T22:41:34.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5400000000000002e-07 2024-09-17T22:41:34.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:34.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781830+0000", 2024-09-17T22:41:34.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1959999999999998e-06 2024-09-17T22:41:34.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:34.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781837+0000", 2024-09-17T22:41:34.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8310000000000003e-06 2024-09-17T22:41:34.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781843+0000", 2024-09-17T22:41:34.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1e-06 2024-09-17T22:41:34.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:34.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832674+0000", 2024-09-17T22:41:34.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050831182000000003 2024-09-17T22:41:34.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:34.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832725+0000", 2024-09-17T22:41:34.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1588000000000001e-05 2024-09-17T22:41:34.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:34.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832773+0000", 2024-09-17T22:41:34.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7769999999999998e-05 2024-09-17T22:41:34.030 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.030 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:34.031 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.031 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38909 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 36864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:34.031 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:41:34.031 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1811670950000002, 2024-09-17T22:41:34.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051046518999999999, 2024-09-17T22:41:34.031 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:34.031 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:34.031 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:34.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:34.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:34.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.031 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38909 2024-09-17T22:41:34.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.032 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:34.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:34.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:41:34.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:34.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:41:34.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:34.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:41:34.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:41:34.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:34.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781852+0000", 2024-09-17T22:41:34.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0590000000000001e-06 2024-09-17T22:41:34.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:34.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781859+0000", 2024-09-17T22:41:34.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3380000000000001e-06 2024-09-17T22:41:34.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781864+0000", 2024-09-17T22:41:34.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:41:34.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:34.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832804+0000", 2024-09-17T22:41:34.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050939614000000001 2024-09-17T22:41:34.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:34.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832845+0000", 2024-09-17T22:41:34.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1014999999999999e-05 2024-09-17T22:41:34.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:34.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832896+0000", 2024-09-17T22:41:34.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1836999999999999e-05 2024-09-17T22:41:34.035 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.035 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:34.035 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.035 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38910 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:34.035 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:41:34.035 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1811467850000001, 2024-09-17T22:41:34.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051216818999999997, 2024-09-17T22:41:34.035 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:34.035 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:34.035 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:34.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:34.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:34.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.035 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38910 2024-09-17T22:41:34.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.036 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:34.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:34.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:41:34.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:34.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:41:34.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:34.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781871+0000", 2024-09-17T22:41:34.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:41:34.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:34.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781873+0000", 2024-09-17T22:41:34.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:41:34.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:34.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781879+0000", 2024-09-17T22:41:34.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0700000000000003e-06 2024-09-17T22:41:34.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781884+0000", 2024-09-17T22:41:34.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-09-17T22:41:34.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:34.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832932+0000", 2024-09-17T22:41:34.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051048219999999998 2024-09-17T22:41:34.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:34.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833001+0000", 2024-09-17T22:41:34.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8504999999999995e-05 2024-09-17T22:41:34.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:34.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833087+0000", 2024-09-17T22:41:34.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6464000000000004e-05 2024-09-17T22:41:34.039 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.039 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:34.039 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.039 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38923 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:34.039 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:41:34.039 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1811264650000002, 2024-09-17T22:41:34.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051379573999999997, 2024-09-17T22:41:34.039 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:34.039 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:34.040 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:34.040 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:34.040 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:34.040 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.040 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38923 2024-09-17T22:41:34.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.040 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:34.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:34.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:41:34.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:34.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:41:34.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:34.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:41:34.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:41:34.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:34.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781893+0000", 2024-09-17T22:41:34.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-09-17T22:41:34.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:34.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781900+0000", 2024-09-17T22:41:34.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3860000000000004e-06 2024-09-17T22:41:34.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781904+0000", 2024-09-17T22:41:34.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9300000000000002e-06 2024-09-17T22:41:34.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:34.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833115+0000", 2024-09-17T22:41:34.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051210845999999997 2024-09-17T22:41:34.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:34.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833188+0000", 2024-09-17T22:41:34.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2276999999999999e-05 2024-09-17T22:41:34.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:34.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833270+0000", 2024-09-17T22:41:34.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2501000000000002e-05 2024-09-17T22:41:34.043 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.043 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:34.043 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.043 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38924 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:34.043 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.782651+0000", 2024-09-17T22:41:34.043 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1803655500000003, 2024-09-17T22:41:34.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050794405000000001, 2024-09-17T22:41:34.043 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:34.044 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:34.044 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:34.044 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:34.044 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:34.044 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.044 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38924 2024-09-17T22:41:34.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.044 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:34.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:34.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782651+0000", 2024-09-17T22:41:34.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:34.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782651+0000", 2024-09-17T22:41:34.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:34.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782652+0000", 2024-09-17T22:41:34.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9699999999999996e-07 2024-09-17T22:41:34.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:34.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782659+0000", 2024-09-17T22:41:34.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7179999999999999e-06 2024-09-17T22:41:34.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:34.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782667+0000", 2024-09-17T22:41:34.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6959999999999994e-06 2024-09-17T22:41:34.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782674+0000", 2024-09-17T22:41:34.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5529999999999998e-06 2024-09-17T22:41:34.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:34.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833298+0000", 2024-09-17T22:41:34.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050623965999999999 2024-09-17T22:41:34.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:34.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833366+0000", 2024-09-17T22:41:34.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8303e-05 2024-09-17T22:41:34.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:34.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833446+0000", 2024-09-17T22:41:34.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9572000000000003e-05 2024-09-17T22:41:34.047 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.047 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:34.047 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.047 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38941 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:34.048 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:41:34.048 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.1803339299999998, 2024-09-17T22:41:34.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050969459000000002, 2024-09-17T22:41:34.048 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:34.048 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:34.048 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:34.048 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:34.048 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:34.048 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.048 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38941 2024-09-17T22:41:34.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.048 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:34.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:34.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:41:34.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:34.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:41:34.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:34.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:34.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782684+0000", 2024-09-17T22:41:34.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:41:34.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:34.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782686+0000", 2024-09-17T22:41:34.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.396e-06 2024-09-17T22:41:34.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:34.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782693+0000", 2024-09-17T22:41:34.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1999999999999997e-06 2024-09-17T22:41:34.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782699+0000", 2024-09-17T22:41:34.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7910000000000001e-06 2024-09-17T22:41:34.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:34.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833471+0000", 2024-09-17T22:41:34.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050772333000000003 2024-09-17T22:41:34.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:34.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833536+0000", 2024-09-17T22:41:34.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4380000000000004e-05 2024-09-17T22:41:34.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:34.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:34.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:34.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833653+0000", 2024-09-17T22:41:34.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011691799999999999 2024-09-17T22:41:34.051 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.051 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:34.051 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.051 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:34.052 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:34.052 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:34.052 INFO:teuthology.orchestra.run.smithi115.stdout:600 trunc from 0x9af2fac to 0x3f268e 2024-09-17T22:41:34.053 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:41:34.053 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is set 2024-09-17T22:41:34.054 INFO:teuthology.orchestra.run.smithi115.stdout:601 writesame 0x8708444 thru 0x8709da0 data_size 0x1(0x195d bytes) 2024-09-17T22:41:34.054 INFO:teuthology.orchestra.run.smithi115.stdout:605 writesame 0xcc891c thru 0xcd8094 data_size 0x1(0xf779 bytes) 2024-09-17T22:41:34.054 INFO:teuthology.orchestra.run.smithi115.stdout:606 trunc from 0x8709da1 to 0x28e810b 2024-09-17T22:41:34.056 INFO:teuthology.orchestra.run.smithi115.stdout:607 writesame 0x345e389 thru 0x3460060 data_size 0x8(0x1cd8 bytes) 2024-09-17T22:41:34.066 INFO:teuthology.orchestra.run.smithi115.stdout:608 trunc from 0x3460061 to 0xb9e96f4 2024-09-17T22:41:34.071 INFO:teuthology.orchestra.run.smithi115.stdout:609 read 0x3aa9f89 thru 0x3ab2a1d (0x8a95 bytes) 2024-09-17T22:41:34.071 INFO:teuthology.orchestra.run.smithi115.stdout:612 writesame 0x6217ec9 thru 0x621885f data_size 0x1(0x997 bytes) 2024-09-17T22:41:34.083 INFO:teuthology.orchestra.run.smithi115.stdout:613 write 0xdc4a6ed thru 0xdc53c0b (0x951f bytes) 2024-09-17T22:41:34.087 INFO:teuthology.orchestra.run.smithi115.stdout:614 trunc from 0xdc53c0c to 0xcb1694b 2024-09-17T22:41:34.094 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:34.096 INFO:teuthology.orchestra.run.smithi115.stdout:617 trunc from 0xcb1694b to 0xb3009b1 2024-09-17T22:41:34.098 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:34.095+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:34.118 INFO:teuthology.orchestra.run.smithi115.stdout:618 trunc from 0xb3009b1 to 0x34d6da 2024-09-17T22:41:34.133 INFO:teuthology.orchestra.run.smithi115.stdout:619 read 0x13c712 thru 0x143481 (0x6d70 bytes) 2024-09-17T22:41:34.134 INFO:teuthology.orchestra.run.smithi115.stdout:620 punch from 0xb2910 to 0xbbc44, (0x9334 bytes) 2024-09-17T22:41:34.134 INFO:teuthology.orchestra.run.smithi115.stdout:621 read 0x2b115f thru 0x2b1b03 (0x9a5 bytes) 2024-09-17T22:41:34.147 INFO:teuthology.orchestra.run.smithi115.stdout:622 writesame 0x651cc78 thru 0x652ab40 data_size 0x3(0xdec9 bytes) 2024-09-17T22:41:34.157 INFO:teuthology.orchestra.run.smithi115.stdout:623 writesame 0xab2409b thru 0xab24de7 data_size 0x5(0xd4d bytes) 2024-09-17T22:41:34.157 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:34.157 INFO:teuthology.orchestra.run.smithi115.stdout:625 punch from 0x8eb01b to 0x8ee069, (0x304e bytes) 2024-09-17T22:41:34.157 INFO:teuthology.orchestra.run.smithi115.stdout:626 read 0xa10e0ea thru 0xa1124a6 (0x43bd bytes) 2024-09-17T22:41:34.157 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:34.157 INFO:teuthology.orchestra.run.smithi115.stdout:628 write 0x81a5e52 thru 0x81b19ca (0xbb79 bytes) 2024-09-17T22:41:34.160 INFO:teuthology.orchestra.run.smithi115.stdout:629 clone 7 order 24 su 65536 sc 4 2024-09-17T22:41:34.165 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:34.165 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:34.165 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:41:34.165 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:34.165 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:34.163+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:34.174 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:41:34.266 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:34.263+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:34.318 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:34.318 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:34.318 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:34.318 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:34.318 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:34.327 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:41:34.369 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:34.365+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:34.468 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:34.465+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:34.472 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:34.472 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:637 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:24.660367+0000", 2024-09-17T22:41:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.8089435480000002, 2024-09-17T22:41:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027568698999999999, 2024-09-17T22:41:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 637 2024-09-17T22:41:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.473 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660367+0000", 2024-09-17T22:41:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660367+0000", 2024-09-17T22:41:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660368+0000", 2024-09-17T22:41:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1299999999999999e-07 2024-09-17T22:41:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660372+0000", 2024-09-17T22:41:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3819999999999997e-06 2024-09-17T22:41:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660382+0000", 2024-09-17T22:41:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8470000000000008e-06 2024-09-17T22:41:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660387+0000", 2024-09-17T22:41:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-17T22:41:34.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660429+0000", 2024-09-17T22:41:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2194000000000003e-05 2024-09-17T22:41:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660518+0000", 2024-09-17T22:41:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8572999999999994e-05 2024-09-17T22:41:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-17T22:41:34.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.664974+0000", 2024-09-17T22:41:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0044561469999999997 2024-09-17T22:41:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.687829+0000", 2024-09-17T22:41:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022854695000000001 2024-09-17T22:41:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:41:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.687850+0000", 2024-09-17T22:41:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0933000000000001e-05 2024-09-17T22:41:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.687885+0000", 2024-09-17T22:41:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5301e-05 2024-09-17T22:41:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.687936+0000", 2024-09-17T22:41:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0726999999999997e-05 2024-09-17T22:41:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6543 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5392384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:34.478 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476788+0000", 2024-09-17T22:41:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.992522932, 2024-09-17T22:41:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024737467999999999, 2024-09-17T22:41:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6543 2024-09-17T22:41:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476788+0000", 2024-09-17T22:41:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476788+0000", 2024-09-17T22:41:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476788+0000", 2024-09-17T22:41:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6100000000000002e-07 2024-09-17T22:41:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476790+0000", 2024-09-17T22:41:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9570000000000001e-06 2024-09-17T22:41:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476796+0000", 2024-09-17T22:41:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9710000000000001e-06 2024-09-17T22:41:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476801+0000", 2024-09-17T22:41:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7129999999999996e-06 2024-09-17T22:41:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501329+0000", 2024-09-17T22:41:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024528378 2024-09-17T22:41:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501415+0000", 2024-09-17T22:41:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6184e-05 2024-09-17T22:41:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501525+0000", 2024-09-17T22:41:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000109904 2024-09-17T22:41:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6544 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5410816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476807+0000", 2024-09-17T22:41:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9925039099999999, 2024-09-17T22:41:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024850173, 2024-09-17T22:41:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6544 2024-09-17T22:41:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476807+0000", 2024-09-17T22:41:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476807+0000", 2024-09-17T22:41:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476807+0000", 2024-09-17T22:41:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.58e-07 2024-09-17T22:41:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476810+0000", 2024-09-17T22:41:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7889999999999999e-06 2024-09-17T22:41:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476815+0000", 2024-09-17T22:41:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7299999999999996e-06 2024-09-17T22:41:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476819+0000", 2024-09-17T22:41:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6879999999999998e-06 2024-09-17T22:41:34.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501554+0000", 2024-09-17T22:41:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024734471000000001 2024-09-17T22:41:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501598+0000", 2024-09-17T22:41:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4257e-05 2024-09-17T22:41:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501657+0000", 2024-09-17T22:41:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8879999999999999e-05 2024-09-17T22:41:34.486 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6545 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5429248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476825+0000", 2024-09-17T22:41:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9924852900000003, 2024-09-17T22:41:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024965472999999998, 2024-09-17T22:41:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6545 2024-09-17T22:41:34.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476825+0000", 2024-09-17T22:41:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476825+0000", 2024-09-17T22:41:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476826+0000", 2024-09-17T22:41:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4499999999999998e-07 2024-09-17T22:41:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476828+0000", 2024-09-17T22:41:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.88e-06 2024-09-17T22:41:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476835+0000", 2024-09-17T22:41:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4089999999999997e-06 2024-09-17T22:41:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476845+0000", 2024-09-17T22:41:34.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8660000000000001e-06 2024-09-17T22:41:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501693+0000", 2024-09-17T22:41:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024848282999999999 2024-09-17T22:41:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501736+0000", 2024-09-17T22:41:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3101e-05 2024-09-17T22:41:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501791+0000", 2024-09-17T22:41:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4589000000000002e-05 2024-09-17T22:41:34.490 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6546 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476854+0000", 2024-09-17T22:41:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9924570539999999, 2024-09-17T22:41:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025123742000000001, 2024-09-17T22:41:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.491 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6546 2024-09-17T22:41:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476854+0000", 2024-09-17T22:41:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476854+0000", 2024-09-17T22:41:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476854+0000", 2024-09-17T22:41:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T22:41:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476856+0000", 2024-09-17T22:41:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4820000000000001e-06 2024-09-17T22:41:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476863+0000", 2024-09-17T22:41:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6749999999999996e-06 2024-09-17T22:41:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476868+0000", 2024-09-17T22:41:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0150000000000003e-06 2024-09-17T22:41:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501814+0000", 2024-09-17T22:41:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024946111 2024-09-17T22:41:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501888+0000", 2024-09-17T22:41:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4200000000000001e-05 2024-09-17T22:41:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501977+0000", 2024-09-17T22:41:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8799000000000001e-05 2024-09-17T22:41:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6547 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5466112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476875+0000", 2024-09-17T22:41:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9924357629999996, 2024-09-17T22:41:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025238633999999999, 2024-09-17T22:41:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6547 2024-09-17T22:41:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476875+0000", 2024-09-17T22:41:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476875+0000", 2024-09-17T22:41:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476875+0000", 2024-09-17T22:41:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:41:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476877+0000", 2024-09-17T22:41:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.075e-06 2024-09-17T22:41:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476883+0000", 2024-09-17T22:41:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.976e-06 2024-09-17T22:41:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476888+0000", 2024-09-17T22:41:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7430000000000002e-06 2024-09-17T22:41:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501974+0000", 2024-09-17T22:41:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025085547 2024-09-17T22:41:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502055+0000", 2024-09-17T22:41:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1484999999999997e-05 2024-09-17T22:41:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502113+0000", 2024-09-17T22:41:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8251999999999997e-05 2024-09-17T22:41:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6548 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5484544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476894+0000", 2024-09-17T22:41:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9924163750000004, 2024-09-17T22:41:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025340366, 2024-09-17T22:41:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.499 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6548 2024-09-17T22:41:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476894+0000", 2024-09-17T22:41:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476894+0000", 2024-09-17T22:41:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476895+0000", 2024-09-17T22:41:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:41:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476897+0000", 2024-09-17T22:41:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9609999999999999e-06 2024-09-17T22:41:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476902+0000", 2024-09-17T22:41:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6969999999999998e-06 2024-09-17T22:41:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476908+0000", 2024-09-17T22:41:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3090000000000003e-06 2024-09-17T22:41:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502138+0000", 2024-09-17T22:41:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025230742 2024-09-17T22:41:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502181+0000", 2024-09-17T22:41:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2580999999999998e-05 2024-09-17T22:41:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502235+0000", 2024-09-17T22:41:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3684999999999998e-05 2024-09-17T22:41:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6549 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5502976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476914+0000", 2024-09-17T22:41:34.503 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9923961859999997, 2024-09-17T22:41:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025541851000000001, 2024-09-17T22:41:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6549 2024-09-17T22:41:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476914+0000", 2024-09-17T22:41:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476914+0000", 2024-09-17T22:41:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476915+0000", 2024-09-17T22:41:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7899999999999999e-07 2024-09-17T22:41:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476917+0000", 2024-09-17T22:41:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.03e-06 2024-09-17T22:41:34.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476923+0000", 2024-09-17T22:41:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2210000000000002e-06 2024-09-17T22:41:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476928+0000", 2024-09-17T22:41:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4429999999999996e-06 2024-09-17T22:41:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502278+0000", 2024-09-17T22:41:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025350784000000001 2024-09-17T22:41:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502364+0000", 2024-09-17T22:41:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5779999999999995e-05 2024-09-17T22:41:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502456+0000", 2024-09-17T22:41:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2114000000000006e-05 2024-09-17T22:41:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.507 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6550 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5507072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476934+0000", 2024-09-17T22:41:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9923768470000001, 2024-09-17T22:41:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025625060000000002, 2024-09-17T22:41:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6550 2024-09-17T22:41:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.508 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476934+0000", 2024-09-17T22:41:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476934+0000", 2024-09-17T22:41:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476934+0000", 2024-09-17T22:41:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:41:34.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476936+0000", 2024-09-17T22:41:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9479999999999998e-06 2024-09-17T22:41:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476942+0000", 2024-09-17T22:41:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.032e-06 2024-09-17T22:41:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476948+0000", 2024-09-17T22:41:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.57e-06 2024-09-17T22:41:34.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502474+0000", 2024-09-17T22:41:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025525808000000001 2024-09-17T22:41:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502510+0000", 2024-09-17T22:41:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6452000000000003e-05 2024-09-17T22:41:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502559+0000", 2024-09-17T22:41:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8828000000000001e-05 2024-09-17T22:41:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.511 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6551 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5525504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476954+0000", 2024-09-17T22:41:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.992356826, 2024-09-17T22:41:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025815945, 2024-09-17T22:41:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6551 2024-09-17T22:41:34.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476954+0000", 2024-09-17T22:41:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476954+0000", 2024-09-17T22:41:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476954+0000", 2024-09-17T22:41:34.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1800000000000001e-07 2024-09-17T22:41:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476956+0000", 2024-09-17T22:41:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0810000000000001e-06 2024-09-17T22:41:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476974+0000", 2024-09-17T22:41:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8168e-05 2024-09-17T22:41:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476980+0000", 2024-09-17T22:41:34.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8200000000000002e-06 2024-09-17T22:41:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502595+0000", 2024-09-17T22:41:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025614865000000001 2024-09-17T22:41:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502675+0000", 2024-09-17T22:41:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:41:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502770+0000", 2024-09-17T22:41:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4789000000000003e-05 2024-09-17T22:41:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6552 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5543936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476988+0000", 2024-09-17T22:41:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9923229930000002, 2024-09-17T22:41:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02590947, 2024-09-17T22:41:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6552 2024-09-17T22:41:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476988+0000", 2024-09-17T22:41:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476988+0000", 2024-09-17T22:41:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476988+0000", 2024-09-17T22:41:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T22:41:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476995+0000", 2024-09-17T22:41:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.533e-06 2024-09-17T22:41:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477001+0000", 2024-09-17T22:41:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6699999999999997e-06 2024-09-17T22:41:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477006+0000", 2024-09-17T22:41:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6970000000000001e-06 2024-09-17T22:41:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502798+0000", 2024-09-17T22:41:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025792234000000001 2024-09-17T22:41:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502835+0000", 2024-09-17T22:41:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7073999999999998e-05 2024-09-17T22:41:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502897+0000", 2024-09-17T22:41:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1828999999999994e-05 2024-09-17T22:41:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6553 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5562368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.477013+0000", 2024-09-17T22:41:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9922980169999995, 2024-09-17T22:41:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026089772000000001, 2024-09-17T22:41:34.520 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6553 2024-09-17T22:41:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477013+0000", 2024-09-17T22:41:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477013+0000", 2024-09-17T22:41:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477013+0000", 2024-09-17T22:41:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:41:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477016+0000", 2024-09-17T22:41:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.207e-06 2024-09-17T22:41:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477023+0000", 2024-09-17T22:41:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.613e-06 2024-09-17T22:41:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477028+0000", 2024-09-17T22:41:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0669999999999999e-06 2024-09-17T22:41:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502933+0000", 2024-09-17T22:41:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025905076999999999 2024-09-17T22:41:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503010+0000", 2024-09-17T22:41:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6716000000000002e-05 2024-09-17T22:41:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503102+0000", 2024-09-17T22:41:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2651000000000005e-05 2024-09-17T22:41:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6554 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5580800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:34.524 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.477034+0000", 2024-09-17T22:41:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.99227679, 2024-09-17T22:41:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026183898000000001, 2024-09-17T22:41:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6554 2024-09-17T22:41:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477034+0000", 2024-09-17T22:41:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477034+0000", 2024-09-17T22:41:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477034+0000", 2024-09-17T22:41:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6399999999999998e-07 2024-09-17T22:41:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477036+0000", 2024-09-17T22:41:34.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.889e-06 2024-09-17T22:41:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477042+0000", 2024-09-17T22:41:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7910000000000001e-06 2024-09-17T22:41:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477046+0000", 2024-09-17T22:41:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.561e-06 2024-09-17T22:41:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503130+0000", 2024-09-17T22:41:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026083168 2024-09-17T22:41:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503166+0000", 2024-09-17T22:41:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6879999999999999e-05 2024-09-17T22:41:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503218+0000", 2024-09-17T22:41:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1245000000000001e-05 2024-09-17T22:41:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6555 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5599232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.477053+0000", 2024-09-17T22:41:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9922578980000001, 2024-09-17T22:41:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026368473, 2024-09-17T22:41:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6555 2024-09-17T22:41:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477053+0000", 2024-09-17T22:41:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477053+0000", 2024-09-17T22:41:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477053+0000", 2024-09-17T22:41:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6699999999999999e-07 2024-09-17T22:41:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477055+0000", 2024-09-17T22:41:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9120000000000001e-06 2024-09-17T22:41:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477061+0000", 2024-09-17T22:41:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6149999999999996e-06 2024-09-17T22:41:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477065+0000", 2024-09-17T22:41:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2080000000000002e-06 2024-09-17T22:41:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503252+0000", 2024-09-17T22:41:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026187430000000001 2024-09-17T22:41:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503330+0000", 2024-09-17T22:41:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7812000000000006e-05 2024-09-17T22:41:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503421+0000", 2024-09-17T22:41:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1129000000000001e-05 2024-09-17T22:41:34.532 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6556 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5617664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.477071+0000", 2024-09-17T22:41:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.992239466, 2024-09-17T22:41:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026451061000000001, 2024-09-17T22:41:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6556 2024-09-17T22:41:34.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477071+0000", 2024-09-17T22:41:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477071+0000", 2024-09-17T22:41:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477072+0000", 2024-09-17T22:41:34.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3799999999999998e-07 2024-09-17T22:41:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477074+0000", 2024-09-17T22:41:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9870000000000002e-06 2024-09-17T22:41:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477080+0000", 2024-09-17T22:41:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3459999999999999e-06 2024-09-17T22:41:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477084+0000", 2024-09-17T22:41:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4880000000000001e-06 2024-09-17T22:41:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503438+0000", 2024-09-17T22:41:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026353225000000001 2024-09-17T22:41:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503473+0000", 2024-09-17T22:41:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5472000000000002e-05 2024-09-17T22:41:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503522+0000", 2024-09-17T22:41:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9104999999999998e-05 2024-09-17T22:41:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:33624 3.7 3:e206720e:::rbd_data.115fe0c30ef6.0000000000000042:5 [read 231424~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.501400+0000", 2024-09-17T22:41:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.9679105870000004, 2024-09-17T22:41:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024723893, 2024-09-17T22:41:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.537 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 33624 2024-09-17T22:41:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501400+0000", 2024-09-17T22:41:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501400+0000", 2024-09-17T22:41:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501400+0000", 2024-09-17T22:41:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8100000000000003e-07 2024-09-17T22:41:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501403+0000", 2024-09-17T22:41:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1500000000000002e-06 2024-09-17T22:41:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501409+0000", 2024-09-17T22:41:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3779999999999998e-06 2024-09-17T22:41:34.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501414+0000", 2024-09-17T22:41:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1139999999999997e-06 2024-09-17T22:41:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.525952+0000", 2024-09-17T22:41:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024537792999999999 2024-09-17T22:41:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526035+0000", 2024-09-17T22:41:34.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2948000000000006e-05 2024-09-17T22:41:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526124+0000", 2024-09-17T22:41:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9029000000000004e-05 2024-09-17T22:41:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:33625 3.7 3:e206720e:::rbd_data.115fe0c30ef6.0000000000000042:5 [read 245760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.501420+0000", 2024-09-17T22:41:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.9678905289999999, 2024-09-17T22:41:34.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024907489000000001, 2024-09-17T22:41:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 33625 2024-09-17T22:41:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501420+0000", 2024-09-17T22:41:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501420+0000", 2024-09-17T22:41:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501420+0000", 2024-09-17T22:41:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:41:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501428+0000", 2024-09-17T22:41:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6520000000000006e-06 2024-09-17T22:41:34.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501435+0000", 2024-09-17T22:41:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9229999999999996e-06 2024-09-17T22:41:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501440+0000", 2024-09-17T22:41:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3569999999999997e-06 2024-09-17T22:41:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526155+0000", 2024-09-17T22:41:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024715114999999999 2024-09-17T22:41:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526233+0000", 2024-09-17T22:41:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7560999999999998e-05 2024-09-17T22:41:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526328+0000", 2024-09-17T22:41:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4526999999999998e-05 2024-09-17T22:41:34.545 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:33626 3.7 3:e206720e:::rbd_data.115fe0c30ef6.0000000000000042:5 [read 253952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.501477+0000", 2024-09-17T22:41:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.9678338389999999, 2024-09-17T22:41:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025046036000000001, 2024-09-17T22:41:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.546 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 33626 2024-09-17T22:41:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501477+0000", 2024-09-17T22:41:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501477+0000", 2024-09-17T22:41:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501478+0000", 2024-09-17T22:41:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0299999999999998e-07 2024-09-17T22:41:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501483+0000", 2024-09-17T22:41:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.519e-06 2024-09-17T22:41:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501492+0000", 2024-09-17T22:41:34.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4819999999999994e-06 2024-09-17T22:41:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501498+0000", 2024-09-17T22:41:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6780000000000003e-06 2024-09-17T22:41:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526356+0000", 2024-09-17T22:41:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024858203999999998 2024-09-17T22:41:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526427+0000", 2024-09-17T22:41:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1021999999999999e-05 2024-09-17T22:41:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526523+0000", 2024-09-17T22:41:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5328000000000006e-05 2024-09-17T22:41:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:33627 3.7 3:e206720e:::rbd_data.115fe0c30ef6.0000000000000042:5 [read 260096~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.501510+0000", 2024-09-17T22:41:34.550 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.9678002799999996, 2024-09-17T22:41:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025207994000000001, 2024-09-17T22:41:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 33627 2024-09-17T22:41:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501510+0000", 2024-09-17T22:41:34.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501510+0000", 2024-09-17T22:41:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501511+0000", 2024-09-17T22:41:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3399999999999999e-07 2024-09-17T22:41:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501513+0000", 2024-09-17T22:41:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5009999999999999e-06 2024-09-17T22:41:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501520+0000", 2024-09-17T22:41:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7800000000000003e-06 2024-09-17T22:41:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501525+0000", 2024-09-17T22:41:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3399999999999997e-06 2024-09-17T22:41:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526550+0000", 2024-09-17T22:41:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025024066000000001 2024-09-17T22:41:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526623+0000", 2024-09-17T22:41:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3240999999999999e-05 2024-09-17T22:41:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526718+0000", 2024-09-17T22:41:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5532000000000006e-05 2024-09-17T22:41:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.554 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:51796 3.1b 3:dc9348e8:::rbd_data.115fe9ac8535.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T22:41:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:33.665030+0000", 2024-09-17T22:41:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.804281001, 2024-09-17T22:41:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027937322000000001, 2024-09-17T22:41:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 51796 2024-09-17T22:41:34.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.665030+0000", 2024-09-17T22:41:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.665030+0000", 2024-09-17T22:41:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.665030+0000", 2024-09-17T22:41:34.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.37e-07 2024-09-17T22:41:34.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:34.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.665034+0000", 2024-09-17T22:41:34.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8539999999999999e-06 2024-09-17T22:41:34.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:34.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.665043+0000", 2024-09-17T22:41:34.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6000000000000007e-06 2024-09-17T22:41:34.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:34.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.665047+0000", 2024-09-17T22:41:34.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4490000000000001e-06 2024-09-17T22:41:34.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:34.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.665095+0000", 2024-09-17T22:41:34.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7703000000000001e-05 2024-09-17T22:41:34.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:34.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.665180+0000", 2024-09-17T22:41:34.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4950000000000005e-05 2024-09-17T22:41:34.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-17T22:41:34.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.669629+0000", 2024-09-17T22:41:34.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0044492919999999997 2024-09-17T22:41:34.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:34.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.692861+0000", 2024-09-17T22:41:34.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023232222 2024-09-17T22:41:34.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:41:34.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.692874+0000", 2024-09-17T22:41:34.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2813000000000001e-05 2024-09-17T22:41:34.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:34.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.692911+0000", 2024-09-17T22:41:34.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7064999999999998e-05 2024-09-17T22:41:34.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:34.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:34.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:34.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.692967+0000", 2024-09-17T22:41:34.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5637000000000003e-05 2024-09-17T22:41:34.560 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.560 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.560 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.560 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:34.560 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:34.560 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:34.561 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:41:34.569 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:34.567+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:34.671 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:34.667+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:34.704 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:34.704 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:34.705 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:41:34.705 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:34.713 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:41:34.797 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:34.794+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:34.856 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:34.856 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:34.856 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:34.856 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:34.856 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:34.865 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:41:34.872 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:34.870+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:34.973 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:34.970+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:35.013 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:1016 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:35.013 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:24.899766+0000", 2024-09-17T22:41:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.110003058, 2024-09-17T22:41:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020466634000000001, 2024-09-17T22:41:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1016 2024-09-17T22:41:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899766+0000", 2024-09-17T22:41:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899766+0000", 2024-09-17T22:41:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899766+0000", 2024-09-17T22:41:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4799999999999997e-07 2024-09-17T22:41:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899771+0000", 2024-09-17T22:41:35.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8489999999999997e-06 2024-09-17T22:41:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899782+0000", 2024-09-17T22:41:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0702e-05 2024-09-17T22:41:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899787+0000", 2024-09-17T22:41:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1460000000000004e-06 2024-09-17T22:41:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899823+0000", 2024-09-17T22:41:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5478000000000002e-05 2024-09-17T22:41:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899878+0000", 2024-09-17T22:41:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5034999999999997e-05 2024-09-17T22:41:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T22:41:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.903518+0000", 2024-09-17T22:41:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036402240000000001 2024-09-17T22:41:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.920108+0000", 2024-09-17T22:41:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016589650000000001 2024-09-17T22:41:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:41:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.920127+0000", 2024-09-17T22:41:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9504000000000002e-05 2024-09-17T22:41:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.920180+0000", 2024-09-17T22:41:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3276999999999998e-05 2024-09-17T22:41:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.920232+0000", 2024-09-17T22:41:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2021000000000002e-05 2024-09-17T22:41:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6105 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 5564416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.471471+0000", 2024-09-17T22:41:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.5382981670000007, 2024-09-17T22:41:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017349954000000001, 2024-09-17T22:41:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6105 2024-09-17T22:41:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471471+0000", 2024-09-17T22:41:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471471+0000", 2024-09-17T22:41:35.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471471+0000", 2024-09-17T22:41:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7599999999999998e-07 2024-09-17T22:41:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471481+0000", 2024-09-17T22:41:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0207000000000001e-05 2024-09-17T22:41:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471488+0000", 2024-09-17T22:41:35.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0700000000000001e-06 2024-09-17T22:41:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471493+0000", 2024-09-17T22:41:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1900000000000003e-06 2024-09-17T22:41:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.481097+0000", 2024-09-17T22:41:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0096031459999999999 2024-09-17T22:41:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.481170+0000", 2024-09-17T22:41:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3187000000000002e-05 2024-09-17T22:41:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.488821+0000", 2024-09-17T22:41:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0076507779999999996 2024-09-17T22:41:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35632 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2605056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.023 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.533932+0000", 2024-09-17T22:41:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.4758369919999996, 2024-09-17T22:41:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016583209000000002, 2024-09-17T22:41:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35632 2024-09-17T22:41:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.533932+0000", 2024-09-17T22:41:35.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.533932+0000", 2024-09-17T22:41:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.533932+0000", 2024-09-17T22:41:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.05e-07 2024-09-17T22:41:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.533933+0000", 2024-09-17T22:41:35.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6200000000000006e-07 2024-09-17T22:41:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.533936+0000", 2024-09-17T22:41:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9000000000000002e-06 2024-09-17T22:41:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.533939+0000", 2024-09-17T22:41:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6639999999999998e-06 2024-09-17T22:41:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.550351+0000", 2024-09-17T22:41:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016412663000000001 2024-09-17T22:41:35.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.550423+0000", 2024-09-17T22:41:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2175999999999995e-05 2024-09-17T22:41:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.550515+0000", 2024-09-17T22:41:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1638999999999995e-05 2024-09-17T22:41:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35695 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2553856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.536131+0000", 2024-09-17T22:41:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.4736382419999998, 2024-09-17T22:41:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.01647641, 2024-09-17T22:41:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35695 2024-09-17T22:41:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536131+0000", 2024-09-17T22:41:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536131+0000", 2024-09-17T22:41:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536131+0000", 2024-09-17T22:41:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.39e-07 2024-09-17T22:41:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536133+0000", 2024-09-17T22:41:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.95e-06 2024-09-17T22:41:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536138+0000", 2024-09-17T22:41:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1950000000000002e-06 2024-09-17T22:41:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536143+0000", 2024-09-17T22:41:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7779999999999998e-06 2024-09-17T22:41:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.552473+0000", 2024-09-17T22:41:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016330458999999999 2024-09-17T22:41:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.552517+0000", 2024-09-17T22:41:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3680999999999997e-05 2024-09-17T22:41:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.552607+0000", 2024-09-17T22:41:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0007999999999996e-05 2024-09-17T22:41:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.031 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35696 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.536148+0000", 2024-09-17T22:41:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.4736203320000003, 2024-09-17T22:41:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016687067999999999, 2024-09-17T22:41:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35696 2024-09-17T22:41:35.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536148+0000", 2024-09-17T22:41:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536148+0000", 2024-09-17T22:41:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536149+0000", 2024-09-17T22:41:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:41:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536155+0000", 2024-09-17T22:41:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8690000000000001e-06 2024-09-17T22:41:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536161+0000", 2024-09-17T22:41:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7050000000000002e-06 2024-09-17T22:41:35.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536166+0000", 2024-09-17T22:41:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5909999999999998e-06 2024-09-17T22:41:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.552657+0000", 2024-09-17T22:41:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016490916000000001 2024-09-17T22:41:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.552741+0000", 2024-09-17T22:41:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3852999999999998e-05 2024-09-17T22:41:35.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.552836+0000", 2024-09-17T22:41:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4748999999999997e-05 2024-09-17T22:41:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35742 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2670592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.536998+0000", 2024-09-17T22:41:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.4727708660000003, 2024-09-17T22:41:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016453115000000001, 2024-09-17T22:41:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.036 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35742 2024-09-17T22:41:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536998+0000", 2024-09-17T22:41:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536998+0000", 2024-09-17T22:41:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536998+0000", 2024-09-17T22:41:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:41:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537000+0000", 2024-09-17T22:41:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1299999999999999e-06 2024-09-17T22:41:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537007+0000", 2024-09-17T22:41:35.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5289999999999997e-06 2024-09-17T22:41:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537012+0000", 2024-09-17T22:41:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2970000000000003e-06 2024-09-17T22:41:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553368+0000", 2024-09-17T22:41:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016356103 2024-09-17T22:41:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553404+0000", 2024-09-17T22:41:35.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5729000000000003e-05 2024-09-17T22:41:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553451+0000", 2024-09-17T22:41:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6935999999999999e-05 2024-09-17T22:41:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35743 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2744320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.537019+0000", 2024-09-17T22:41:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.4727500310000003, 2024-09-17T22:41:35.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016542099000000001, 2024-09-17T22:41:35.041 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.041 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.140 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.140 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.140 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.140 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.140 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35743 2024-09-17T22:41:35.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.140 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537019+0000", 2024-09-17T22:41:35.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537019+0000", 2024-09-17T22:41:35.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537019+0000", 2024-09-17T22:41:35.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6300000000000001e-07 2024-09-17T22:41:35.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537021+0000", 2024-09-17T22:41:35.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9080000000000002e-06 2024-09-17T22:41:35.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537027+0000", 2024-09-17T22:41:35.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3090000000000001e-06 2024-09-17T22:41:35.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537033+0000", 2024-09-17T22:41:35.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4539999999999998e-06 2024-09-17T22:41:35.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553475+0000", 2024-09-17T22:41:35.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016442142999999999 2024-09-17T22:41:35.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553510+0000", 2024-09-17T22:41:35.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5030999999999997e-05 2024-09-17T22:41:35.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553561+0000", 2024-09-17T22:41:35.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0890999999999998e-05 2024-09-17T22:41:35.144 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.144 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.144 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.144 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35745 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2480128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.144 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.537054+0000", 2024-09-17T22:41:35.144 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.4727150590000004, 2024-09-17T22:41:35.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016621817000000001, 2024-09-17T22:41:35.145 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.145 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.145 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.145 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.145 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.145 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.145 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35745 2024-09-17T22:41:35.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.145 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537054+0000", 2024-09-17T22:41:35.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537054+0000", 2024-09-17T22:41:35.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537055+0000", 2024-09-17T22:41:35.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0400000000000005e-07 2024-09-17T22:41:35.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537059+0000", 2024-09-17T22:41:35.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9910000000000004e-06 2024-09-17T22:41:35.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537070+0000", 2024-09-17T22:41:35.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1398e-05 2024-09-17T22:41:35.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537077+0000", 2024-09-17T22:41:35.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4959999999999999e-06 2024-09-17T22:41:35.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553585+0000", 2024-09-17T22:41:35.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016508020000000002 2024-09-17T22:41:35.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553621+0000", 2024-09-17T22:41:35.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6013999999999998e-05 2024-09-17T22:41:35.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553676+0000", 2024-09-17T22:41:35.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4094000000000001e-05 2024-09-17T22:41:35.148 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.148 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.148 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.148 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35791 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2615296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.149 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.538661+0000", 2024-09-17T22:41:35.149 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.4711078840000003, 2024-09-17T22:41:35.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016411623, 2024-09-17T22:41:35.149 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.149 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.149 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.149 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.149 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.149 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.149 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35791 2024-09-17T22:41:35.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.149 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538661+0000", 2024-09-17T22:41:35.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538661+0000", 2024-09-17T22:41:35.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538661+0000", 2024-09-17T22:41:35.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3399999999999999e-07 2024-09-17T22:41:35.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538664+0000", 2024-09-17T22:41:35.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:41:35.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538685+0000", 2024-09-17T22:41:35.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1608999999999999e-05 2024-09-17T22:41:35.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.151 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.151 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538691+0000", 2024-09-17T22:41:35.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7629999999999997e-06 2024-09-17T22:41:35.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.554892+0000", 2024-09-17T22:41:35.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016200951000000002 2024-09-17T22:41:35.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.554962+0000", 2024-09-17T22:41:35.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9962999999999994e-05 2024-09-17T22:41:35.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555073+0000", 2024-09-17T22:41:35.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000110674 2024-09-17T22:41:35.152 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.153 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.153 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.153 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35805 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2674688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.153 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.538700+0000", 2024-09-17T22:41:35.153 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.4710692549999997, 2024-09-17T22:41:35.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016568598, 2024-09-17T22:41:35.153 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.153 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.153 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.153 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.153 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.153 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.154 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35805 2024-09-17T22:41:35.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.154 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538700+0000", 2024-09-17T22:41:35.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538700+0000", 2024-09-17T22:41:35.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538700+0000", 2024-09-17T22:41:35.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8500000000000002e-07 2024-09-17T22:41:35.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538702+0000", 2024-09-17T22:41:35.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.447e-06 2024-09-17T22:41:35.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538709+0000", 2024-09-17T22:41:35.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2530000000000001e-06 2024-09-17T22:41:35.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538714+0000", 2024-09-17T22:41:35.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8770000000000001e-06 2024-09-17T22:41:35.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555104+0000", 2024-09-17T22:41:35.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016390372 2024-09-17T22:41:35.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555168+0000", 2024-09-17T22:41:35.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3559000000000006e-05 2024-09-17T22:41:35.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555268+0000", 2024-09-17T22:41:35.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100605 2024-09-17T22:41:35.157 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.157 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.157 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.157 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35806 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2748416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.157 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.538719+0000", 2024-09-17T22:41:35.157 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.4710492989999997, 2024-09-17T22:41:35.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016746647999999999, 2024-09-17T22:41:35.157 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.157 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.157 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.157 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.158 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.158 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.158 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35806 2024-09-17T22:41:35.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.158 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538719+0000", 2024-09-17T22:41:35.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538719+0000", 2024-09-17T22:41:35.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538720+0000", 2024-09-17T22:41:35.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:41:35.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538722+0000", 2024-09-17T22:41:35.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9429999999999999e-06 2024-09-17T22:41:35.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538727+0000", 2024-09-17T22:41:35.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6559999999999997e-06 2024-09-17T22:41:35.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538732+0000", 2024-09-17T22:41:35.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7949999999999998e-06 2024-09-17T22:41:35.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555297+0000", 2024-09-17T22:41:35.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016564861 2024-09-17T22:41:35.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555358+0000", 2024-09-17T22:41:35.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1177000000000001e-05 2024-09-17T22:41:35.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555466+0000", 2024-09-17T22:41:35.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010785000000000001 2024-09-17T22:41:35.161 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.161 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.161 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.161 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35809 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2410496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.161 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.538738+0000", 2024-09-17T22:41:35.161 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.4710309050000001, 2024-09-17T22:41:35.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016923453000000001, 2024-09-17T22:41:35.161 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.161 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.162 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.162 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.162 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.162 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.162 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35809 2024-09-17T22:41:35.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.162 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538738+0000", 2024-09-17T22:41:35.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538738+0000", 2024-09-17T22:41:35.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538738+0000", 2024-09-17T22:41:35.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:41:35.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538740+0000", 2024-09-17T22:41:35.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9080000000000002e-06 2024-09-17T22:41:35.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538746+0000", 2024-09-17T22:41:35.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6330000000000001e-06 2024-09-17T22:41:35.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538750+0000", 2024-09-17T22:41:35.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5040000000000004e-06 2024-09-17T22:41:35.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555497+0000", 2024-09-17T22:41:35.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016747171000000002 2024-09-17T22:41:35.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555559+0000", 2024-09-17T22:41:35.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1822999999999994e-05 2024-09-17T22:41:35.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555661+0000", 2024-09-17T22:41:35.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010206 2024-09-17T22:41:35.165 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.165 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.165 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.165 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35810 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2484224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.165 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.538756+0000", 2024-09-17T22:41:35.165 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.4710124200000001, 2024-09-17T22:41:35.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017104030999999999, 2024-09-17T22:41:35.166 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.166 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.166 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.166 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.166 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.166 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.166 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35810 2024-09-17T22:41:35.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.166 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538756+0000", 2024-09-17T22:41:35.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538756+0000", 2024-09-17T22:41:35.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538757+0000", 2024-09-17T22:41:35.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:41:35.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538759+0000", 2024-09-17T22:41:35.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.778e-06 2024-09-17T22:41:35.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538764+0000", 2024-09-17T22:41:35.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6840000000000001e-06 2024-09-17T22:41:35.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538771+0000", 2024-09-17T22:41:35.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5590000000000002e-06 2024-09-17T22:41:35.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555692+0000", 2024-09-17T22:41:35.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016921281 2024-09-17T22:41:35.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555752+0000", 2024-09-17T22:41:35.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0402000000000002e-05 2024-09-17T22:41:35.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555860+0000", 2024-09-17T22:41:35.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010796100000000001 2024-09-17T22:41:35.169 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.169 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.169 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.169 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35823 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2543616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.169 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.539339+0000", 2024-09-17T22:41:35.170 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.4704294029999998, 2024-09-17T22:41:35.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016657568000000001, 2024-09-17T22:41:35.170 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.170 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.170 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.170 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.170 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.170 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.170 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35823 2024-09-17T22:41:35.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.170 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539339+0000", 2024-09-17T22:41:35.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539339+0000", 2024-09-17T22:41:35.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539340+0000", 2024-09-17T22:41:35.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2900000000000002e-07 2024-09-17T22:41:35.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539344+0000", 2024-09-17T22:41:35.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0729999999999998e-06 2024-09-17T22:41:35.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539356+0000", 2024-09-17T22:41:35.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1484000000000001e-05 2024-09-17T22:41:35.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539360+0000", 2024-09-17T22:41:35.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7260000000000002e-06 2024-09-17T22:41:35.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555893+0000", 2024-09-17T22:41:35.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016532958 2024-09-17T22:41:35.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555941+0000", 2024-09-17T22:41:35.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7729999999999999e-05 2024-09-17T22:41:35.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555997+0000", 2024-09-17T22:41:35.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5767999999999999e-05 2024-09-17T22:41:35.173 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.173 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.173 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.174 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35824 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2617344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.174 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.539370+0000", 2024-09-17T22:41:35.174 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.4703990060000001, 2024-09-17T22:41:35.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016758449000000002, 2024-09-17T22:41:35.174 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.174 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.174 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.174 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.174 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.174 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.174 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35824 2024-09-17T22:41:35.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.174 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539370+0000", 2024-09-17T22:41:35.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539370+0000", 2024-09-17T22:41:35.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539370+0000", 2024-09-17T22:41:35.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3799999999999998e-07 2024-09-17T22:41:35.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539372+0000", 2024-09-17T22:41:35.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8160000000000001e-06 2024-09-17T22:41:35.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539376+0000", 2024-09-17T22:41:35.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.844e-06 2024-09-17T22:41:35.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539379+0000", 2024-09-17T22:41:35.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0910000000000001e-06 2024-09-17T22:41:35.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556031+0000", 2024-09-17T22:41:35.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016651750999999999 2024-09-17T22:41:35.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556072+0000", 2024-09-17T22:41:35.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1690999999999998e-05 2024-09-17T22:41:35.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556128+0000", 2024-09-17T22:41:35.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5918000000000002e-05 2024-09-17T22:41:35.177 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.177 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.177 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.178 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35837 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2676736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.178 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.539383+0000", 2024-09-17T22:41:35.178 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.4703860369999999, 2024-09-17T22:41:35.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0168665, 2024-09-17T22:41:35.178 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.178 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.178 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.178 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.178 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.178 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.178 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35837 2024-09-17T22:41:35.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.178 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539383+0000", 2024-09-17T22:41:35.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539383+0000", 2024-09-17T22:41:35.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539383+0000", 2024-09-17T22:41:35.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2399999999999999e-07 2024-09-17T22:41:35.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539384+0000", 2024-09-17T22:41:35.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.305e-06 2024-09-17T22:41:35.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539387+0000", 2024-09-17T22:41:35.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0939999999999999e-06 2024-09-17T22:41:35.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539390+0000", 2024-09-17T22:41:35.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.678e-06 2024-09-17T22:41:35.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556152+0000", 2024-09-17T22:41:35.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016762353000000001 2024-09-17T22:41:35.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556193+0000", 2024-09-17T22:41:35.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0399999999999999e-05 2024-09-17T22:41:35.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556249+0000", 2024-09-17T22:41:35.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6446000000000001e-05 2024-09-17T22:41:35.181 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.182 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.182 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.182 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35838 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2750464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.182 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.539393+0000", 2024-09-17T22:41:35.182 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.470375518, 2024-09-17T22:41:35.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017005407, 2024-09-17T22:41:35.182 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.182 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.182 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.182 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.182 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.182 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.183 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35838 2024-09-17T22:41:35.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.183 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539393+0000", 2024-09-17T22:41:35.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539393+0000", 2024-09-17T22:41:35.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539393+0000", 2024-09-17T22:41:35.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.05e-07 2024-09-17T22:41:35.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539395+0000", 2024-09-17T22:41:35.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.11e-06 2024-09-17T22:41:35.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539398+0000", 2024-09-17T22:41:35.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0240000000000002e-06 2024-09-17T22:41:35.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539401+0000", 2024-09-17T22:41:35.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1729999999999999e-06 2024-09-17T22:41:35.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556291+0000", 2024-09-17T22:41:35.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016890179000000002 2024-09-17T22:41:35.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556343+0000", 2024-09-17T22:41:35.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2380999999999999e-05 2024-09-17T22:41:35.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556399+0000", 2024-09-17T22:41:35.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5334999999999998e-05 2024-09-17T22:41:35.186 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.186 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.186 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.186 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35841 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2412544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.186 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.539404+0000", 2024-09-17T22:41:35.186 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.4703649690000002, 2024-09-17T22:41:35.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017168085999999999, 2024-09-17T22:41:35.186 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.186 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.186 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.186 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.186 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.187 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35841 2024-09-17T22:41:35.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.187 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.187 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539404+0000", 2024-09-17T22:41:35.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.187 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539404+0000", 2024-09-17T22:41:35.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539404+0000", 2024-09-17T22:41:35.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0599999999999999e-07 2024-09-17T22:41:35.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539407+0000", 2024-09-17T22:41:35.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3890000000000001e-06 2024-09-17T22:41:35.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539411+0000", 2024-09-17T22:41:35.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2009999999999999e-06 2024-09-17T22:41:35.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539413+0000", 2024-09-17T22:41:35.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6110000000000001e-06 2024-09-17T22:41:35.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556421+0000", 2024-09-17T22:41:35.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017008079999999998 2024-09-17T22:41:35.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556470+0000", 2024-09-17T22:41:35.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8987999999999999e-05 2024-09-17T22:41:35.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556572+0000", 2024-09-17T22:41:35.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101611 2024-09-17T22:41:35.190 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.190 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.190 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.190 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35845 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2486272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.190 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.539416+0000", 2024-09-17T22:41:35.190 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.4703523150000004, 2024-09-17T22:41:35.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017358905000000001, 2024-09-17T22:41:35.190 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.190 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.191 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.191 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.191 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.191 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.191 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35845 2024-09-17T22:41:35.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.191 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539416+0000", 2024-09-17T22:41:35.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539416+0000", 2024-09-17T22:41:35.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539417+0000", 2024-09-17T22:41:35.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3200000000000001e-07 2024-09-17T22:41:35.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539418+0000", 2024-09-17T22:41:35.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.094e-06 2024-09-17T22:41:35.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539421+0000", 2024-09-17T22:41:35.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0010000000000002e-06 2024-09-17T22:41:35.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539423+0000", 2024-09-17T22:41:35.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5919999999999999e-06 2024-09-17T22:41:35.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556602+0000", 2024-09-17T22:41:35.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017178729 2024-09-17T22:41:35.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556677+0000", 2024-09-17T22:41:35.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4825000000000003e-05 2024-09-17T22:41:35.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556775+0000", 2024-09-17T22:41:35.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8431999999999995e-05 2024-09-17T22:41:35.194 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.194 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.194 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.194 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35870 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2678784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.194 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.540812+0000", 2024-09-17T22:41:35.194 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.4689569169999999, 2024-09-17T22:41:35.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016561679999999999, 2024-09-17T22:41:35.195 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.195 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.195 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.195 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.195 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.195 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.195 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35870 2024-09-17T22:41:35.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.195 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.540812+0000", 2024-09-17T22:41:35.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.540812+0000", 2024-09-17T22:41:35.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.540812+0000", 2024-09-17T22:41:35.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7300000000000001e-07 2024-09-17T22:41:35.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.540814+0000", 2024-09-17T22:41:35.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1610000000000001e-06 2024-09-17T22:41:35.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.540821+0000", 2024-09-17T22:41:35.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2249999999999997e-06 2024-09-17T22:41:35.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.540827+0000", 2024-09-17T22:41:35.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4540000000000004e-06 2024-09-17T22:41:35.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.557219+0000", 2024-09-17T22:41:35.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016392317 2024-09-17T22:41:35.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.557290+0000", 2024-09-17T22:41:35.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0239000000000003e-05 2024-09-17T22:41:35.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.557374+0000", 2024-09-17T22:41:35.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3811000000000001e-05 2024-09-17T22:41:35.198 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.198 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.198 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.198 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.198 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.198 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:35.199 INFO:teuthology.orchestra.run.smithi115.stdout:flattening image image_client.0-clone6 2024-09-17T22:41:35.200 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-17T22:41:35.200 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:41:35.201 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:35.196+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:35.207 INFO:teuthology.orchestra.run.smithi115.stdout:631 write 0xda3c03b thru 0xda4a336 (0xe2fc bytes) 2024-09-17T22:41:35.213 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:35.214 INFO:teuthology.orchestra.run.smithi115.stdout:634 punch from 0x3fb0767 to 0x3fb1109, (0x9a2 bytes) 2024-09-17T22:41:35.214 INFO:teuthology.orchestra.run.smithi115.stdout:636 punch from 0x2f396fd to 0x2f3b483, (0x1d86 bytes) 2024-09-17T22:41:35.214 INFO:teuthology.orchestra.run.smithi115.stdout:638 write 0x2343ae1 thru 0x234c72b (0x8c4b bytes) 2024-09-17T22:41:35.225 INFO:teuthology.orchestra.run.smithi115.stdout:640 write 0x4696383 thru 0x4697b7f (0x17fd bytes) 2024-09-17T22:41:35.231 INFO:teuthology.orchestra.run.smithi115.stdout:641 punch from 0xaeac65c to 0xaeb4e26, (0x87ca bytes) 2024-09-17T22:41:35.237 INFO:teuthology.orchestra.run.smithi115.stdout:642 write 0xe85a224 thru 0xe869315 (0xf0f2 bytes) 2024-09-17T22:41:35.243 INFO:teuthology.orchestra.run.smithi115.stdout:643 read 0x4563bc5 thru 0x456815f (0x459b bytes) 2024-09-17T22:41:35.243 INFO:teuthology.orchestra.run.smithi115.stdout:644 read 0x75ce2bd thru 0x75d5a29 (0x776d bytes) 2024-09-17T22:41:35.243 INFO:teuthology.orchestra.run.smithi115.stdout:646 writesame 0xa7a4050 thru 0xa7af0dc data_size 0x1(0xb08d bytes) 2024-09-17T22:41:35.267 INFO:teuthology.orchestra.run.smithi115.stdout:647 writesame 0x8f2377 thru 0x8fde6e data_size 0x3e(0xbaf8 bytes) 2024-09-17T22:41:35.283 INFO:teuthology.orchestra.run.smithi115.stdout:648 write 0x6d32890 thru 0x6d3c470 (0x9be1 bytes) 2024-09-17T22:41:35.289 INFO:teuthology.orchestra.run.smithi115.stdout:650 read 0xecd49d thru 0xed96ec (0xc250 bytes) 2024-09-17T22:41:35.289 INFO:teuthology.orchestra.run.smithi115.stdout:651 read 0xb28d746 thru 0xb29a4af (0xcd6a bytes) 2024-09-17T22:41:35.289 INFO:teuthology.orchestra.run.smithi115.stdout:652 punch from 0xb0a1d4d to 0xb0a454e, (0x2801 bytes) 2024-09-17T22:41:35.290 INFO:teuthology.orchestra.run.smithi115.stdout:653 punch from 0x556737 to 0x5622ba, (0xbb83 bytes) 2024-09-17T22:41:35.290 INFO:teuthology.orchestra.run.smithi115.stdout:655 writesame 0x1caf2c0 thru 0x1cbdc18 data_size 0x29(0xe959 bytes) 2024-09-17T22:41:35.301 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:35.297+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:35.302 INFO:teuthology.orchestra.run.smithi115.stdout:656 read 0xbd5dc26 thru 0xbd60d07 (0x30e2 bytes) 2024-09-17T22:41:35.302 INFO:teuthology.orchestra.run.smithi115.stdout:657 write 0xe72e31e thru 0xe72f012 (0xcf5 bytes) 2024-09-17T22:41:35.303 INFO:teuthology.orchestra.run.smithi115.stdout:659 write 0x56a4704 thru 0x56aa779 (0x6076 bytes) 2024-09-17T22:41:35.308 INFO:teuthology.orchestra.run.smithi115.stdout:661 read 0x57684dc thru 0x57725e4 (0xa109 bytes) 2024-09-17T22:41:35.309 INFO:teuthology.orchestra.run.smithi115.stdout:662 write 0x8658854 thru 0x8665169 (0xc916 bytes) 2024-09-17T22:41:35.315 INFO:teuthology.orchestra.run.smithi115.stdout:663 write 0x8232cc3 thru 0x823cb3c (0x9e7a bytes) 2024-09-17T22:41:35.320 INFO:teuthology.orchestra.run.smithi115.stdout:664 writesame 0xca00588 thru 0xca02e27 data_size 0x14(0x28a0 bytes) 2024-09-17T22:41:35.321 INFO:teuthology.orchestra.run.smithi115.stdout:665 write 0x60df292 thru 0x60e2b35 (0x38a4 bytes) 2024-09-17T22:41:35.323 INFO:teuthology.orchestra.run.smithi115.stdout:666 punch from 0x549e98b to 0x54ade1a, (0xf48f bytes) 2024-09-17T22:41:35.323 INFO:teuthology.orchestra.run.smithi115.stdout:667 write 0x4ace38 thru 0x4b505b (0x8224 bytes) 2024-09-17T22:41:35.325 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:35.325 INFO:teuthology.orchestra.run.smithi115.stdout:669 write 0xd448398 thru 0xd44b4ac (0x3115 bytes) 2024-09-17T22:41:35.326 INFO:teuthology.orchestra.run.smithi115.stdout:670 write 0xc834a81 thru 0xc837a8a (0x300a bytes) 2024-09-17T22:41:35.327 INFO:teuthology.orchestra.run.smithi115.stdout:671 clone 8 order 19 su 4096 sc 15 2024-09-17T22:41:35.353 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:35.353 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:35.353 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:41:35.353 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:35.362 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:41:35.402 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:35.399+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:35.503 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:35.500+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:35.506 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:35.507 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:35.515 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:41:35.603 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:35.601+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:35.619 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:35.619 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:35.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:35.619 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:35.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.619 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38083 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.619 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.763873+0000", 2024-09-17T22:41:35.619 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.8518958980000004, 2024-09-17T22:41:35.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023344747999999999, 2024-09-17T22:41:35.619 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.619 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.619 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.620 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.620 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.620 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38083 2024-09-17T22:41:35.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.620 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763873+0000", 2024-09-17T22:41:35.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763873+0000", 2024-09-17T22:41:35.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763874+0000", 2024-09-17T22:41:35.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0899999999999997e-07 2024-09-17T22:41:35.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763875+0000", 2024-09-17T22:41:35.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8530000000000001e-06 2024-09-17T22:41:35.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763881+0000", 2024-09-17T22:41:35.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4879999999999998e-06 2024-09-17T22:41:35.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763886+0000", 2024-09-17T22:41:35.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.587e-06 2024-09-17T22:41:35.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787057+0000", 2024-09-17T22:41:35.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023170611000000001 2024-09-17T22:41:35.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787127+0000", 2024-09-17T22:41:35.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0259000000000006e-05 2024-09-17T22:41:35.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787218+0000", 2024-09-17T22:41:35.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0641000000000002e-05 2024-09-17T22:41:35.623 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.623 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.623 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.623 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38105 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.623 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.763938+0000", 2024-09-17T22:41:35.623 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.8518310910000002, 2024-09-17T22:41:35.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023471315999999999, 2024-09-17T22:41:35.624 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.624 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.624 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.624 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.624 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.624 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.624 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38105 2024-09-17T22:41:35.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.624 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763938+0000", 2024-09-17T22:41:35.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763938+0000", 2024-09-17T22:41:35.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763939+0000", 2024-09-17T22:41:35.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:41:35.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763941+0000", 2024-09-17T22:41:35.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9489999999999999e-06 2024-09-17T22:41:35.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763946+0000", 2024-09-17T22:41:35.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9139999999999996e-06 2024-09-17T22:41:35.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763951+0000", 2024-09-17T22:41:35.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6589999999999998e-06 2024-09-17T22:41:35.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787249+0000", 2024-09-17T22:41:35.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023297989000000002 2024-09-17T22:41:35.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787319+0000", 2024-09-17T22:41:35.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9836e-05 2024-09-17T22:41:35.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787409+0000", 2024-09-17T22:41:35.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0457000000000005e-05 2024-09-17T22:41:35.627 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.627 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.627 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.628 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38107 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.628 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:35.628 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.8518123019999999, 2024-09-17T22:41:35.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023669550000000001, 2024-09-17T22:41:35.628 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.628 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.628 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.628 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.628 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.628 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.628 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38107 2024-09-17T22:41:35.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.628 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:35.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:35.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:35.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:41:35.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763959+0000", 2024-09-17T22:41:35.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.849e-06 2024-09-17T22:41:35.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763965+0000", 2024-09-17T22:41:35.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6110000000000001e-06 2024-09-17T22:41:35.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763969+0000", 2024-09-17T22:41:35.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-06 2024-09-17T22:41:35.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787451+0000", 2024-09-17T22:41:35.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023482281000000001 2024-09-17T22:41:35.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787516+0000", 2024-09-17T22:41:35.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4930000000000003e-05 2024-09-17T22:41:35.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787626+0000", 2024-09-17T22:41:35.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011000700000000001 2024-09-17T22:41:35.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.631 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.632 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38141 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.632 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764408+0000", 2024-09-17T22:41:35.632 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.8513610700000003, 2024-09-17T22:41:35.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023422268, 2024-09-17T22:41:35.632 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.632 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.632 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.632 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38141 2024-09-17T22:41:35.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.632 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764408+0000", 2024-09-17T22:41:35.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764408+0000", 2024-09-17T22:41:35.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764409+0000", 2024-09-17T22:41:35.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:41:35.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764412+0000", 2024-09-17T22:41:35.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4470000000000002e-06 2024-09-17T22:41:35.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764421+0000", 2024-09-17T22:41:35.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7299999999999994e-06 2024-09-17T22:41:35.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764427+0000", 2024-09-17T22:41:35.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.392e-06 2024-09-17T22:41:35.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787669+0000", 2024-09-17T22:41:35.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023241385 2024-09-17T22:41:35.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787743+0000", 2024-09-17T22:41:35.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4260999999999999e-05 2024-09-17T22:41:35.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787830+0000", 2024-09-17T22:41:35.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7398999999999994e-05 2024-09-17T22:41:35.635 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.636 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.636 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.636 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38143 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.636 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:35.636 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.8513311420000003, 2024-09-17T22:41:35.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023565691, 2024-09-17T22:41:35.636 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.636 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.636 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.636 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.636 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.636 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.636 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38143 2024-09-17T22:41:35.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.637 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:35.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:35.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:35.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6199999999999998e-07 2024-09-17T22:41:35.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764441+0000", 2024-09-17T22:41:35.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1179999999999999e-06 2024-09-17T22:41:35.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764447+0000", 2024-09-17T22:41:35.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1940000000000003e-06 2024-09-17T22:41:35.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764452+0000", 2024-09-17T22:41:35.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9150000000000004e-06 2024-09-17T22:41:35.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787881+0000", 2024-09-17T22:41:35.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023429141000000001 2024-09-17T22:41:35.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787947+0000", 2024-09-17T22:41:35.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5902999999999996e-05 2024-09-17T22:41:35.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788004+0000", 2024-09-17T22:41:35.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6957999999999999e-05 2024-09-17T22:41:35.640 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.640 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.640 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.640 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38150 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.640 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:35.640 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.85131114, 2024-09-17T22:41:35.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023741399, 2024-09-17T22:41:35.640 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.640 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.640 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.640 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.640 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.641 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.641 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38150 2024-09-17T22:41:35.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.641 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:35.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:35.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:35.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8299999999999998e-07 2024-09-17T22:41:35.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764460+0000", 2024-09-17T22:41:35.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.855e-06 2024-09-17T22:41:35.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764466+0000", 2024-09-17T22:41:35.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8200000000000002e-06 2024-09-17T22:41:35.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764471+0000", 2024-09-17T22:41:35.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8319999999999996e-06 2024-09-17T22:41:35.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788032+0000", 2024-09-17T22:41:35.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023561480999999999 2024-09-17T22:41:35.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788106+0000", 2024-09-17T22:41:35.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3406999999999997e-05 2024-09-17T22:41:35.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788199+0000", 2024-09-17T22:41:35.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3621000000000004e-05 2024-09-17T22:41:35.644 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.644 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.644 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.644 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38152 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 83968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.644 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:35.644 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.8512924230000003, 2024-09-17T22:41:35.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023859726000000001, 2024-09-17T22:41:35.644 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.644 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.644 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.644 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.645 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.645 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.645 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38152 2024-09-17T22:41:35.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.645 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:35.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:35.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:35.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3000000000000001e-07 2024-09-17T22:41:35.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764479+0000", 2024-09-17T22:41:35.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8920000000000001e-06 2024-09-17T22:41:35.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764485+0000", 2024-09-17T22:41:35.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:41:35.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764490+0000", 2024-09-17T22:41:35.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6890000000000003e-06 2024-09-17T22:41:35.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788232+0000", 2024-09-17T22:41:35.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023742452000000001 2024-09-17T22:41:35.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788276+0000", 2024-09-17T22:41:35.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3689000000000001e-05 2024-09-17T22:41:35.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788336+0000", 2024-09-17T22:41:35.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0581999999999997e-05 2024-09-17T22:41:35.648 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.648 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.648 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.648 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38184 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 161792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.648 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764589+0000", 2024-09-17T22:41:35.648 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.8511800300000001, 2024-09-17T22:41:35.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023956801, 2024-09-17T22:41:35.649 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.649 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.649 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.649 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.649 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.649 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.649 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38184 2024-09-17T22:41:35.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.649 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764589+0000", 2024-09-17T22:41:35.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764589+0000", 2024-09-17T22:41:35.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764590+0000", 2024-09-17T22:41:35.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7800000000000001e-07 2024-09-17T22:41:35.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764592+0000", 2024-09-17T22:41:35.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6929999999999999e-06 2024-09-17T22:41:35.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764600+0000", 2024-09-17T22:41:35.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1140000000000006e-06 2024-09-17T22:41:35.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764606+0000", 2024-09-17T22:41:35.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6200000000000004e-06 2024-09-17T22:41:35.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788374+0000", 2024-09-17T22:41:35.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023768083999999998 2024-09-17T22:41:35.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788447+0000", 2024-09-17T22:41:35.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2922000000000004e-05 2024-09-17T22:41:35.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788546+0000", 2024-09-17T22:41:35.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8789999999999994e-05 2024-09-17T22:41:35.652 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.652 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.652 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.653 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38186 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.653 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:35.653 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.8511550799999998, 2024-09-17T22:41:35.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024156486000000001, 2024-09-17T22:41:35.653 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.653 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.653 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.653 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.653 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.653 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.653 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38186 2024-09-17T22:41:35.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.654 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:35.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:35.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:35.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:41:35.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764621+0000", 2024-09-17T22:41:35.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6409999999999996e-06 2024-09-17T22:41:35.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764628+0000", 2024-09-17T22:41:35.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0360000000000001e-06 2024-09-17T22:41:35.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764633+0000", 2024-09-17T22:41:35.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0300000000000001e-06 2024-09-17T22:41:35.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788594+0000", 2024-09-17T22:41:35.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023960921999999999 2024-09-17T22:41:35.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788676+0000", 2024-09-17T22:41:35.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2123e-05 2024-09-17T22:41:35.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788771+0000", 2024-09-17T22:41:35.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4284999999999996e-05 2024-09-17T22:41:35.657 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.657 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.657 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.657 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38218 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 350208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.657 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764706+0000", 2024-09-17T22:41:35.657 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.8510628269999998, 2024-09-17T22:41:35.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024324732000000002, 2024-09-17T22:41:35.657 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.657 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.657 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.657 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.657 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.657 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.658 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38218 2024-09-17T22:41:35.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.658 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764706+0000", 2024-09-17T22:41:35.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764706+0000", 2024-09-17T22:41:35.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764707+0000", 2024-09-17T22:41:35.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7199999999999999e-07 2024-09-17T22:41:35.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764710+0000", 2024-09-17T22:41:35.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6819999999999999e-06 2024-09-17T22:41:35.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764716+0000", 2024-09-17T22:41:35.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7900000000000002e-06 2024-09-17T22:41:35.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764722+0000", 2024-09-17T22:41:35.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2560000000000002e-06 2024-09-17T22:41:35.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788867+0000", 2024-09-17T22:41:35.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024144982999999998 2024-09-17T22:41:35.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788941+0000", 2024-09-17T22:41:35.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4529000000000004e-05 2024-09-17T22:41:35.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789031+0000", 2024-09-17T22:41:35.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9920000000000006e-05 2024-09-17T22:41:35.661 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.661 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.661 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.661 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38222 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.661 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764765+0000", 2024-09-17T22:41:35.661 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.8510039730000001, 2024-09-17T22:41:35.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024443910999999999, 2024-09-17T22:41:35.661 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.661 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.661 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.662 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.662 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38222 2024-09-17T22:41:35.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.662 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764765+0000", 2024-09-17T22:41:35.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764765+0000", 2024-09-17T22:41:35.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764766+0000", 2024-09-17T22:41:35.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8999999999999996e-07 2024-09-17T22:41:35.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764769+0000", 2024-09-17T22:41:35.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7659999999999999e-06 2024-09-17T22:41:35.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764776+0000", 2024-09-17T22:41:35.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4229999999999999e-06 2024-09-17T22:41:35.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764783+0000", 2024-09-17T22:41:35.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9600000000000003e-06 2024-09-17T22:41:35.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789060+0000", 2024-09-17T22:41:35.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024277343 2024-09-17T22:41:35.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789129+0000", 2024-09-17T22:41:35.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8367999999999993e-05 2024-09-17T22:41:35.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789209+0000", 2024-09-17T22:41:35.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0461000000000001e-05 2024-09-17T22:41:35.665 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.665 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.665 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.665 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38671 3.12 3:4a5b92ac:::rbd_data.115fe0c30ef6.0000000000000011:5 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.665 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778865+0000", 2024-09-17T22:41:35.665 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.8369039130000004, 2024-09-17T22:41:35.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023227351, 2024-09-17T22:41:35.665 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.665 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.665 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.666 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.666 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38671 2024-09-17T22:41:35.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.666 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778865+0000", 2024-09-17T22:41:35.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778865+0000", 2024-09-17T22:41:35.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778866+0000", 2024-09-17T22:41:35.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0900000000000002e-07 2024-09-17T22:41:35.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778868+0000", 2024-09-17T22:41:35.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4339999999999999e-06 2024-09-17T22:41:35.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778875+0000", 2024-09-17T22:41:35.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4559999999999997e-06 2024-09-17T22:41:35.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778880+0000", 2024-09-17T22:41:35.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2689999999999999e-06 2024-09-17T22:41:35.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.801978+0000", 2024-09-17T22:41:35.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023098225999999999 2024-09-17T22:41:35.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802035+0000", 2024-09-17T22:41:35.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7123999999999997e-05 2024-09-17T22:41:35.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802093+0000", 2024-09-17T22:41:35.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7432999999999997e-05 2024-09-17T22:41:35.669 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.669 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.669 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.669 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38673 3.12 3:4a5b92ac:::rbd_data.115fe0c30ef6.0000000000000011:5 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.669 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778886+0000", 2024-09-17T22:41:35.669 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.8368827650000004, 2024-09-17T22:41:35.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023389259999999999, 2024-09-17T22:41:35.669 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.669 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.669 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.670 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.670 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.670 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.670 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38673 2024-09-17T22:41:35.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.670 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778886+0000", 2024-09-17T22:41:35.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778886+0000", 2024-09-17T22:41:35.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778887+0000", 2024-09-17T22:41:35.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:41:35.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778889+0000", 2024-09-17T22:41:35.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9379999999999999e-06 2024-09-17T22:41:35.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778895+0000", 2024-09-17T22:41:35.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0360000000000003e-06 2024-09-17T22:41:35.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778900+0000", 2024-09-17T22:41:35.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1220000000000003e-06 2024-09-17T22:41:35.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802126+0000", 2024-09-17T22:41:35.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023226607 2024-09-17T22:41:35.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802208+0000", 2024-09-17T22:41:35.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1157999999999998e-05 2024-09-17T22:41:35.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802276+0000", 2024-09-17T22:41:35.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8122999999999998e-05 2024-09-17T22:41:35.673 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.673 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.673 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.673 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38675 3.12 3:4a5b92ac:::rbd_data.115fe0c30ef6.0000000000000011:5 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.673 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:35.673 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.8368475599999998, 2024-09-17T22:41:35.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023484221999999999, 2024-09-17T22:41:35.673 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.673 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.673 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.674 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.674 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.674 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.674 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38675 2024-09-17T22:41:35.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.674 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:35.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:35.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:35.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9999999999999997e-07 2024-09-17T22:41:35.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778924+0000", 2024-09-17T22:41:35.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3039999999999999e-06 2024-09-17T22:41:35.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778931+0000", 2024-09-17T22:41:35.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9519999999999997e-06 2024-09-17T22:41:35.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778937+0000", 2024-09-17T22:41:35.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3639999999999998e-06 2024-09-17T22:41:35.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802300+0000", 2024-09-17T22:41:35.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023363544 2024-09-17T22:41:35.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802347+0000", 2024-09-17T22:41:35.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6897999999999997e-05 2024-09-17T22:41:35.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802406+0000", 2024-09-17T22:41:35.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8560000000000002e-05 2024-09-17T22:41:35.677 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.677 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.677 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.677 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38679 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.677 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778945+0000", 2024-09-17T22:41:35.677 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.8368239439999998, 2024-09-17T22:41:35.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023578015000000001, 2024-09-17T22:41:35.677 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.677 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.678 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.678 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.678 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.678 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.678 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38679 2024-09-17T22:41:35.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.678 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778945+0000", 2024-09-17T22:41:35.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778945+0000", 2024-09-17T22:41:35.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778946+0000", 2024-09-17T22:41:35.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.44e-07 2024-09-17T22:41:35.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778948+0000", 2024-09-17T22:41:35.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-17T22:41:35.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778955+0000", 2024-09-17T22:41:35.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.528e-06 2024-09-17T22:41:35.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778960+0000", 2024-09-17T22:41:35.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1919999999999996e-06 2024-09-17T22:41:35.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802429+0000", 2024-09-17T22:41:35.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023469561999999999 2024-09-17T22:41:35.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802470+0000", 2024-09-17T22:41:35.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1068000000000001e-05 2024-09-17T22:41:35.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802523+0000", 2024-09-17T22:41:35.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2852999999999997e-05 2024-09-17T22:41:35.681 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.681 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.681 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.681 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38680 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.681 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778966+0000", 2024-09-17T22:41:35.681 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.8368029860000004, 2024-09-17T22:41:35.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023670668999999998, 2024-09-17T22:41:35.681 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.681 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.682 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.682 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.682 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.682 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.682 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38680 2024-09-17T22:41:35.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.682 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778966+0000", 2024-09-17T22:41:35.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778966+0000", 2024-09-17T22:41:35.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778967+0000", 2024-09-17T22:41:35.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.89e-07 2024-09-17T22:41:35.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778969+0000", 2024-09-17T22:41:35.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0219999999999999e-06 2024-09-17T22:41:35.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778975+0000", 2024-09-17T22:41:35.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3779999999999998e-06 2024-09-17T22:41:35.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778980+0000", 2024-09-17T22:41:35.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0699999999999997e-06 2024-09-17T22:41:35.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802545+0000", 2024-09-17T22:41:35.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023565358000000002 2024-09-17T22:41:35.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802585+0000", 2024-09-17T22:41:35.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0136999999999999e-05 2024-09-17T22:41:35.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802637+0000", 2024-09-17T22:41:35.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1314999999999999e-05 2024-09-17T22:41:35.685 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.685 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.685 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.685 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38697 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 675840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.685 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.779074+0000", 2024-09-17T22:41:35.685 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.8366947329999999, 2024-09-17T22:41:35.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023676402999999999, 2024-09-17T22:41:35.685 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.686 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.686 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.686 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.686 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.686 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.686 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38697 2024-09-17T22:41:35.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.686 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779074+0000", 2024-09-17T22:41:35.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779074+0000", 2024-09-17T22:41:35.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779075+0000", 2024-09-17T22:41:35.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:41:35.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779077+0000", 2024-09-17T22:41:35.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8810000000000001e-06 2024-09-17T22:41:35.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779084+0000", 2024-09-17T22:41:35.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.691e-06 2024-09-17T22:41:35.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779088+0000", 2024-09-17T22:41:35.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6970000000000001e-06 2024-09-17T22:41:35.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802659+0000", 2024-09-17T22:41:35.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023571214 2024-09-17T22:41:35.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802699+0000", 2024-09-17T22:41:35.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9762000000000003e-05 2024-09-17T22:41:35.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802751+0000", 2024-09-17T22:41:35.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1572e-05 2024-09-17T22:41:35.689 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.689 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.689 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.689 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38698 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 749568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.689 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.779094+0000", 2024-09-17T22:41:35.689 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.8366747960000005, 2024-09-17T22:41:35.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023778845999999999, 2024-09-17T22:41:35.690 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.690 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.690 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.690 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38698 2024-09-17T22:41:35.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.690 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779094+0000", 2024-09-17T22:41:35.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779094+0000", 2024-09-17T22:41:35.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779095+0000", 2024-09-17T22:41:35.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8799999999999998e-07 2024-09-17T22:41:35.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779097+0000", 2024-09-17T22:41:35.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8530000000000001e-06 2024-09-17T22:41:35.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779103+0000", 2024-09-17T22:41:35.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0410000000000003e-06 2024-09-17T22:41:35.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779107+0000", 2024-09-17T22:41:35.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T22:41:35.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802774+0000", 2024-09-17T22:41:35.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023666348 2024-09-17T22:41:35.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802814+0000", 2024-09-17T22:41:35.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0173000000000003e-05 2024-09-17T22:41:35.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802873+0000", 2024-09-17T22:41:35.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9484999999999997e-05 2024-09-17T22:41:35.693 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.693 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.693 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.693 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38825 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.693 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.781521+0000", 2024-09-17T22:41:35.693 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.8342477290000003, 2024-09-17T22:41:35.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023214104999999999, 2024-09-17T22:41:35.694 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.694 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.694 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:35.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.694 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38825 2024-09-17T22:41:35.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.694 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781521+0000", 2024-09-17T22:41:35.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781521+0000", 2024-09-17T22:41:35.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781522+0000", 2024-09-17T22:41:35.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.53e-07 2024-09-17T22:41:35.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781524+0000", 2024-09-17T22:41:35.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0389999999999999e-06 2024-09-17T22:41:35.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781530+0000", 2024-09-17T22:41:35.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8209999999999998e-06 2024-09-17T22:41:35.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781534+0000", 2024-09-17T22:41:35.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6419999999999998e-06 2024-09-17T22:41:35.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804570+0000", 2024-09-17T22:41:35.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023035951999999998 2024-09-17T22:41:35.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:35.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804645+0000", 2024-09-17T22:41:35.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4696000000000004e-05 2024-09-17T22:41:35.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804736+0000", 2024-09-17T22:41:35.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0601999999999998e-05 2024-09-17T22:41:35.697 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.697 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.697 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.697 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38843 3.12 3.c4845672 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:35.697 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:35.697 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.8341255189999996, 2024-09-17T22:41:35.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023236265999999998, 2024-09-17T22:41:35.698 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:35.698 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:35.698 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:35.698 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:35.698 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:35.698 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:35.698 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38843 2024-09-17T22:41:35.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.698 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:35.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:35.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:35.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:35.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:35.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:35.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:35.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:35.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:41:35.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:35.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781646+0000", 2024-09-17T22:41:35.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9290000000000001e-06 2024-09-17T22:41:35.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:35.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781652+0000", 2024-09-17T22:41:35.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7389999999999996e-06 2024-09-17T22:41:35.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:35.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781656+0000", 2024-09-17T22:41:35.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3830000000000002e-06 2024-09-17T22:41:35.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:35.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804871+0000", 2024-09-17T22:41:35.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023215257999999999 2024-09-17T22:41:35.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:35.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:35.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:35.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804880+0000", 2024-09-17T22:41:35.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5769999999999994e-06 2024-09-17T22:41:35.701 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.701 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.701 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.701 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:35.701 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:35.701 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:35.702 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:41:35.704 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:35.701+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:35.723 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:41:35.804 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:35.801+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:35.805 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:35.805 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:35.805 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:41:35.805 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:35.814 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:41:35.842 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0: {} 2024-09-17T22:41:35.842 INFO:teuthology.orchestra.run.smithi115.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:41:35.858 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:41:35.859 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:41:35.874 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:41:35.874 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:41:35.890 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:41:35.890 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:41:35.905 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:35.902+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:35.908 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:41:35.908 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:41:35.925 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:41:35.926 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:41:35.942 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:41:35.942 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:41:35.958 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:41:35.958 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:41:35.960 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:35.960 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:35.969 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:41:36.006 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:36.004+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:36.052 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone7 from 0xe869316 (overlap 0xab24de8) to 0x88ea4b9 2024-09-17T22:41:36.055 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:41:36.107 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:36.104+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:36.116 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:36.116 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:36.124 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:41:36.179 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-17T22:41:36.208 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:36.204+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:36.258 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:36.258 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:36.268 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:41:36.308 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:36.306+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:36.371 INFO:teuthology.orchestra.run.smithi115.stdout:672 write 0x2d1ad70 thru 0x2d2181f (0x6ab0 bytes) 2024-09-17T22:41:36.380 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:36.380 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:36.393 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:41:36.409 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:36.405+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:36.504 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:36.504 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:36.510 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:36.507+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:36.512 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:41:36.611 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:36.608+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:36.614 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:36.614 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:36.623 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:41:36.646 INFO:teuthology.orchestra.run.smithi115.stdout:673 write 0x356c4d5 thru 0x356d353 (0xe7f bytes) 2024-09-17T22:41:36.671 INFO:teuthology.orchestra.run.smithi115.stdout:674 trunc from 0xe869316 to 0x920e755 2024-09-17T22:41:36.712 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:36.709+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:36.784 INFO:teuthology.orchestra.run.smithi115.stdout:676 writesame 0x990179f thru 0x9903386 data_size 0x26(0x1be8 bytes) 2024-09-17T22:41:36.784 INFO:teuthology.orchestra.run.smithi115.stdout:677 punch from 0x4781ecf to 0x4786fcf, (0x5100 bytes) 2024-09-17T22:41:36.784 INFO:teuthology.orchestra.run.smithi115.stdout:678 read 0x1da1b05 thru 0x1da6b39 (0x5035 bytes) 2024-09-17T22:41:36.792 INFO:teuthology.orchestra.run.smithi115.stdout:681 trunc from 0x9903387 to 0x44f549f 2024-09-17T22:41:36.799 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:36.799 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:36.812 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:41:36.853 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:36.851+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:36.913 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:36.910+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:36.923 INFO:teuthology.orchestra.run.smithi115.stdout:682 trunc from 0x44f549f to 0x688785b 2024-09-17T22:41:36.928 INFO:teuthology.orchestra.run.smithi115.stdout:683 read 0x72921b thru 0x734643 (0xb429 bytes) 2024-09-17T22:41:36.930 INFO:teuthology.orchestra.run.smithi115.stdout:684 write 0x66dcb45 thru 0x66e5fc4 (0x9480 bytes) 2024-09-17T22:41:36.947 INFO:teuthology.orchestra.run.smithi115.stdout:685 read 0x415f1cc thru 0x416cf44 (0xdd79 bytes) 2024-09-17T22:41:36.949 INFO:teuthology.orchestra.run.smithi115.stdout:686 clone 9 order 20 su 32768 sc 11 2024-09-17T22:41:36.984 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:36.984 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:36.993 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:41:37.014 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is unset 2024-09-17T22:41:37.014 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:37.011+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:37.075 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone8 from 0x688785b (overlap 0x44f549f) to 0x295ff92 2024-09-17T22:41:37.109 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:37.109 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:37.118 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:41:37.122 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:37.112+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:37.215 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:37.211+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:37.288 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:37.289 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:37.302 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:41:37.345 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:37.343+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:37.416 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:37.412+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:37.467 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:37.468 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:37.480 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:41:37.517 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:37.514+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:37.555 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-17T22:41:37.618 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:37.614+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:37.651 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:37.652 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:37.661 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:41:37.723 INFO:teuthology.orchestra.run.smithi115.stdout:687 write 0x40eb0cd thru 0x40f5808 (0xa73c bytes) 2024-09-17T22:41:37.724 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:37.721+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:37.738 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:41:37.738 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-17T22:41:37.771 INFO:teuthology.orchestra.run.smithi115.stdout:688 write 0xde52f64 thru 0xde5707c (0x4119 bytes) 2024-09-17T22:41:37.780 INFO:teuthology.orchestra.run.smithi115.stdout:689 trunc from 0xde5707d to 0x936b47e 2024-09-17T22:41:37.790 INFO:teuthology.orchestra.run.smithi115.stdout:690 writesame 0x1826559 thru 0x18313e4 data_size 0x2(0xae8c bytes) 2024-09-17T22:41:37.819 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:37.816+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:37.846 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:37.846 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:37.853 INFO:teuthology.orchestra.run.smithi115.stdout:691 clone 10 order 18 su 131072 sc 9 2024-09-17T22:41:37.856 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:41:37.920 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:37.917+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:37.970 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:41:38.000 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:38.001 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:38.010 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:41:38.051 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:38.049+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:38.080 INFO:teuthology.orchestra.run.smithi115.stdout:leaving image image_client.0-clone9 intact 2024-09-17T22:41:38.102 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0: {} 2024-09-17T22:41:38.102 INFO:teuthology.orchestra.run.smithi115.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:41:38.116 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:38.116 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:38.119 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:41:38.120 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:41:38.121 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:38.117+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:38.124 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:41:38.136 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:41:38.136 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:41:38.153 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:41:38.154 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:41:38.176 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:41:38.176 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:41:38.197 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:41:38.197 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:41:38.216 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:41:38.217 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:41:38.218 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-17T22:41:38.221 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:38.218+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:38.238 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:41:38.238 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:41:38.271 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:38.271 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:38.271 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:38.271 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:38.271 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:38.280 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:41:38.322 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:38.319+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:38.423 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:38.420+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:38.427 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:637 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:24.660367+0000", 2024-09-17T22:41:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.763350018000001, 2024-09-17T22:41:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027568698999999999, 2024-09-17T22:41:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.427 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 637 2024-09-17T22:41:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660367+0000", 2024-09-17T22:41:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660367+0000", 2024-09-17T22:41:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660368+0000", 2024-09-17T22:41:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1299999999999999e-07 2024-09-17T22:41:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660372+0000", 2024-09-17T22:41:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3819999999999997e-06 2024-09-17T22:41:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660382+0000", 2024-09-17T22:41:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8470000000000008e-06 2024-09-17T22:41:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660387+0000", 2024-09-17T22:41:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0869999999999997e-06 2024-09-17T22:41:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660429+0000", 2024-09-17T22:41:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2194000000000003e-05 2024-09-17T22:41:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.660518+0000", 2024-09-17T22:41:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8572999999999994e-05 2024-09-17T22:41:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 1", 2024-09-17T22:41:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.664974+0000", 2024-09-17T22:41:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0044561469999999997 2024-09-17T22:41:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.687829+0000", 2024-09-17T22:41:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022854695000000001 2024-09-17T22:41:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:41:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.687850+0000", 2024-09-17T22:41:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0933000000000001e-05 2024-09-17T22:41:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.687885+0000", 2024-09-17T22:41:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5301e-05 2024-09-17T22:41:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.687936+0000", 2024-09-17T22:41:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0726999999999997e-05 2024-09-17T22:41:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6543 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5392384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476788+0000", 2024-09-17T22:41:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 11.946929402, 2024-09-17T22:41:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024737467999999999, 2024-09-17T22:41:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.433 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6543 2024-09-17T22:41:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476788+0000", 2024-09-17T22:41:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476788+0000", 2024-09-17T22:41:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476788+0000", 2024-09-17T22:41:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6100000000000002e-07 2024-09-17T22:41:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476790+0000", 2024-09-17T22:41:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9570000000000001e-06 2024-09-17T22:41:38.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476796+0000", 2024-09-17T22:41:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9710000000000001e-06 2024-09-17T22:41:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476801+0000", 2024-09-17T22:41:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7129999999999996e-06 2024-09-17T22:41:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501329+0000", 2024-09-17T22:41:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024528378 2024-09-17T22:41:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501415+0000", 2024-09-17T22:41:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6184e-05 2024-09-17T22:41:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501525+0000", 2024-09-17T22:41:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000109904 2024-09-17T22:41:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6544 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5410816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476807+0000", 2024-09-17T22:41:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 11.94691038, 2024-09-17T22:41:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024850173, 2024-09-17T22:41:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6544 2024-09-17T22:41:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.438 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476807+0000", 2024-09-17T22:41:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476807+0000", 2024-09-17T22:41:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476807+0000", 2024-09-17T22:41:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.58e-07 2024-09-17T22:41:38.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476810+0000", 2024-09-17T22:41:38.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7889999999999999e-06 2024-09-17T22:41:38.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476815+0000", 2024-09-17T22:41:38.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7299999999999996e-06 2024-09-17T22:41:38.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476819+0000", 2024-09-17T22:41:38.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6879999999999998e-06 2024-09-17T22:41:38.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501554+0000", 2024-09-17T22:41:38.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024734471000000001 2024-09-17T22:41:38.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501598+0000", 2024-09-17T22:41:38.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4257e-05 2024-09-17T22:41:38.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501657+0000", 2024-09-17T22:41:38.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8879999999999999e-05 2024-09-17T22:41:38.442 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.442 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.442 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.442 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6545 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5429248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:38.442 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476825+0000", 2024-09-17T22:41:38.442 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 11.94689176, 2024-09-17T22:41:38.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024965472999999998, 2024-09-17T22:41:38.442 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.442 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.442 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.443 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6545 2024-09-17T22:41:38.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.443 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476825+0000", 2024-09-17T22:41:38.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476825+0000", 2024-09-17T22:41:38.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476826+0000", 2024-09-17T22:41:38.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4499999999999998e-07 2024-09-17T22:41:38.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476828+0000", 2024-09-17T22:41:38.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.88e-06 2024-09-17T22:41:38.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476835+0000", 2024-09-17T22:41:38.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4089999999999997e-06 2024-09-17T22:41:38.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476845+0000", 2024-09-17T22:41:38.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8660000000000001e-06 2024-09-17T22:41:38.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501693+0000", 2024-09-17T22:41:38.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024848282999999999 2024-09-17T22:41:38.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501736+0000", 2024-09-17T22:41:38.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3101e-05 2024-09-17T22:41:38.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501791+0000", 2024-09-17T22:41:38.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4589000000000002e-05 2024-09-17T22:41:38.446 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.446 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.446 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.446 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6546 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5447680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:38.446 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476854+0000", 2024-09-17T22:41:38.446 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 11.946863523999999, 2024-09-17T22:41:38.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025123742000000001, 2024-09-17T22:41:38.447 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.447 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.447 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.447 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.447 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.447 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.447 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6546 2024-09-17T22:41:38.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.447 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476854+0000", 2024-09-17T22:41:38.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476854+0000", 2024-09-17T22:41:38.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476854+0000", 2024-09-17T22:41:38.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T22:41:38.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476856+0000", 2024-09-17T22:41:38.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4820000000000001e-06 2024-09-17T22:41:38.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476863+0000", 2024-09-17T22:41:38.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6749999999999996e-06 2024-09-17T22:41:38.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476868+0000", 2024-09-17T22:41:38.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0150000000000003e-06 2024-09-17T22:41:38.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501814+0000", 2024-09-17T22:41:38.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024946111 2024-09-17T22:41:38.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501888+0000", 2024-09-17T22:41:38.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4200000000000001e-05 2024-09-17T22:41:38.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501977+0000", 2024-09-17T22:41:38.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8799000000000001e-05 2024-09-17T22:41:38.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.451 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.451 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6547 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5466112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:38.451 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476875+0000", 2024-09-17T22:41:38.451 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 11.946842233, 2024-09-17T22:41:38.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025238633999999999, 2024-09-17T22:41:38.451 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.451 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.451 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.452 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6547 2024-09-17T22:41:38.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.452 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476875+0000", 2024-09-17T22:41:38.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476875+0000", 2024-09-17T22:41:38.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476875+0000", 2024-09-17T22:41:38.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:41:38.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476877+0000", 2024-09-17T22:41:38.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.075e-06 2024-09-17T22:41:38.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476883+0000", 2024-09-17T22:41:38.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.976e-06 2024-09-17T22:41:38.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476888+0000", 2024-09-17T22:41:38.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7430000000000002e-06 2024-09-17T22:41:38.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.501974+0000", 2024-09-17T22:41:38.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025085547 2024-09-17T22:41:38.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502055+0000", 2024-09-17T22:41:38.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1484999999999997e-05 2024-09-17T22:41:38.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502113+0000", 2024-09-17T22:41:38.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8251999999999997e-05 2024-09-17T22:41:38.456 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.456 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.456 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.456 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6548 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5484544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:38.456 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476894+0000", 2024-09-17T22:41:38.456 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 11.946822845, 2024-09-17T22:41:38.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025340366, 2024-09-17T22:41:38.456 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.456 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.456 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.456 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.457 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6548 2024-09-17T22:41:38.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.457 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476894+0000", 2024-09-17T22:41:38.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476894+0000", 2024-09-17T22:41:38.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476895+0000", 2024-09-17T22:41:38.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:41:38.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476897+0000", 2024-09-17T22:41:38.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9609999999999999e-06 2024-09-17T22:41:38.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476902+0000", 2024-09-17T22:41:38.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6969999999999998e-06 2024-09-17T22:41:38.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476908+0000", 2024-09-17T22:41:38.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3090000000000003e-06 2024-09-17T22:41:38.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502138+0000", 2024-09-17T22:41:38.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025230742 2024-09-17T22:41:38.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502181+0000", 2024-09-17T22:41:38.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2580999999999998e-05 2024-09-17T22:41:38.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502235+0000", 2024-09-17T22:41:38.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3684999999999998e-05 2024-09-17T22:41:38.460 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.460 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.460 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.461 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6549 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5502976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:38.461 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476914+0000", 2024-09-17T22:41:38.461 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 11.946802655999999, 2024-09-17T22:41:38.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025541851000000001, 2024-09-17T22:41:38.461 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.461 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.461 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.462 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6549 2024-09-17T22:41:38.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.462 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476914+0000", 2024-09-17T22:41:38.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476914+0000", 2024-09-17T22:41:38.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476915+0000", 2024-09-17T22:41:38.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7899999999999999e-07 2024-09-17T22:41:38.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476917+0000", 2024-09-17T22:41:38.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.03e-06 2024-09-17T22:41:38.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476923+0000", 2024-09-17T22:41:38.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2210000000000002e-06 2024-09-17T22:41:38.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476928+0000", 2024-09-17T22:41:38.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4429999999999996e-06 2024-09-17T22:41:38.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502278+0000", 2024-09-17T22:41:38.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025350784000000001 2024-09-17T22:41:38.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502364+0000", 2024-09-17T22:41:38.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5779999999999995e-05 2024-09-17T22:41:38.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502456+0000", 2024-09-17T22:41:38.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2114000000000006e-05 2024-09-17T22:41:38.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.465 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.466 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6550 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5507072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:38.466 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476934+0000", 2024-09-17T22:41:38.466 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 11.946783317, 2024-09-17T22:41:38.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025625060000000002, 2024-09-17T22:41:38.466 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.466 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.466 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.466 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.466 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.466 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.466 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6550 2024-09-17T22:41:38.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.467 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476934+0000", 2024-09-17T22:41:38.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476934+0000", 2024-09-17T22:41:38.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476934+0000", 2024-09-17T22:41:38.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2199999999999999e-07 2024-09-17T22:41:38.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476936+0000", 2024-09-17T22:41:38.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9479999999999998e-06 2024-09-17T22:41:38.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476942+0000", 2024-09-17T22:41:38.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.032e-06 2024-09-17T22:41:38.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476948+0000", 2024-09-17T22:41:38.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.57e-06 2024-09-17T22:41:38.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502474+0000", 2024-09-17T22:41:38.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025525808000000001 2024-09-17T22:41:38.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502510+0000", 2024-09-17T22:41:38.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6452000000000003e-05 2024-09-17T22:41:38.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502559+0000", 2024-09-17T22:41:38.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8828000000000001e-05 2024-09-17T22:41:38.470 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.470 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.470 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.470 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6551 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5525504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:38.470 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476954+0000", 2024-09-17T22:41:38.470 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 11.946763296, 2024-09-17T22:41:38.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025815945, 2024-09-17T22:41:38.470 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.470 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.470 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.471 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.471 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6551 2024-09-17T22:41:38.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.471 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476954+0000", 2024-09-17T22:41:38.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476954+0000", 2024-09-17T22:41:38.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476954+0000", 2024-09-17T22:41:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1800000000000001e-07 2024-09-17T22:41:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476956+0000", 2024-09-17T22:41:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0810000000000001e-06 2024-09-17T22:41:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476974+0000", 2024-09-17T22:41:38.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8168e-05 2024-09-17T22:41:38.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476980+0000", 2024-09-17T22:41:38.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8200000000000002e-06 2024-09-17T22:41:38.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502595+0000", 2024-09-17T22:41:38.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025614865000000001 2024-09-17T22:41:38.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502675+0000", 2024-09-17T22:41:38.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9803999999999997e-05 2024-09-17T22:41:38.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502770+0000", 2024-09-17T22:41:38.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4789000000000003e-05 2024-09-17T22:41:38.474 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.474 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.474 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.474 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6552 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5543936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:38.474 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.476988+0000", 2024-09-17T22:41:38.474 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 11.946729463, 2024-09-17T22:41:38.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02590947, 2024-09-17T22:41:38.475 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.475 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.475 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.475 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.475 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.475 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.475 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6552 2024-09-17T22:41:38.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.475 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476988+0000", 2024-09-17T22:41:38.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476988+0000", 2024-09-17T22:41:38.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476988+0000", 2024-09-17T22:41:38.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T22:41:38.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.476995+0000", 2024-09-17T22:41:38.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.533e-06 2024-09-17T22:41:38.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477001+0000", 2024-09-17T22:41:38.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6699999999999997e-06 2024-09-17T22:41:38.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477006+0000", 2024-09-17T22:41:38.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6970000000000001e-06 2024-09-17T22:41:38.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502798+0000", 2024-09-17T22:41:38.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025792234000000001 2024-09-17T22:41:38.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502835+0000", 2024-09-17T22:41:38.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7073999999999998e-05 2024-09-17T22:41:38.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502897+0000", 2024-09-17T22:41:38.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1828999999999994e-05 2024-09-17T22:41:38.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.478 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.479 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6553 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5562368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:38.479 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.477013+0000", 2024-09-17T22:41:38.479 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 11.946704487, 2024-09-17T22:41:38.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026089772000000001, 2024-09-17T22:41:38.479 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.479 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.479 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.479 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6553 2024-09-17T22:41:38.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.479 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477013+0000", 2024-09-17T22:41:38.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477013+0000", 2024-09-17T22:41:38.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477013+0000", 2024-09-17T22:41:38.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:41:38.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477016+0000", 2024-09-17T22:41:38.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.207e-06 2024-09-17T22:41:38.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477023+0000", 2024-09-17T22:41:38.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.613e-06 2024-09-17T22:41:38.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477028+0000", 2024-09-17T22:41:38.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0669999999999999e-06 2024-09-17T22:41:38.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.502933+0000", 2024-09-17T22:41:38.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025905076999999999 2024-09-17T22:41:38.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503010+0000", 2024-09-17T22:41:38.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6716000000000002e-05 2024-09-17T22:41:38.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503102+0000", 2024-09-17T22:41:38.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2651000000000005e-05 2024-09-17T22:41:38.483 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.483 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.483 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.483 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6554 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5580800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:38.483 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.477034+0000", 2024-09-17T22:41:38.483 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 11.94668326, 2024-09-17T22:41:38.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026183898000000001, 2024-09-17T22:41:38.483 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.483 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.483 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.483 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.484 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.484 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.484 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6554 2024-09-17T22:41:38.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.484 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477034+0000", 2024-09-17T22:41:38.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477034+0000", 2024-09-17T22:41:38.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477034+0000", 2024-09-17T22:41:38.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6399999999999998e-07 2024-09-17T22:41:38.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477036+0000", 2024-09-17T22:41:38.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.889e-06 2024-09-17T22:41:38.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477042+0000", 2024-09-17T22:41:38.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7910000000000001e-06 2024-09-17T22:41:38.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477046+0000", 2024-09-17T22:41:38.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.561e-06 2024-09-17T22:41:38.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503130+0000", 2024-09-17T22:41:38.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026083168 2024-09-17T22:41:38.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503166+0000", 2024-09-17T22:41:38.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6879999999999999e-05 2024-09-17T22:41:38.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503218+0000", 2024-09-17T22:41:38.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1245000000000001e-05 2024-09-17T22:41:38.487 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.487 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.487 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.487 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6555 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5599232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:38.487 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.477053+0000", 2024-09-17T22:41:38.487 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 11.946664368, 2024-09-17T22:41:38.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026368473, 2024-09-17T22:41:38.488 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.488 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.488 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.488 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6555 2024-09-17T22:41:38.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.488 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477053+0000", 2024-09-17T22:41:38.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477053+0000", 2024-09-17T22:41:38.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477053+0000", 2024-09-17T22:41:38.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6699999999999999e-07 2024-09-17T22:41:38.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477055+0000", 2024-09-17T22:41:38.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9120000000000001e-06 2024-09-17T22:41:38.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477061+0000", 2024-09-17T22:41:38.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6149999999999996e-06 2024-09-17T22:41:38.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477065+0000", 2024-09-17T22:41:38.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2080000000000002e-06 2024-09-17T22:41:38.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503252+0000", 2024-09-17T22:41:38.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026187430000000001 2024-09-17T22:41:38.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503330+0000", 2024-09-17T22:41:38.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7812000000000006e-05 2024-09-17T22:41:38.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503421+0000", 2024-09-17T22:41:38.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1129000000000001e-05 2024-09-17T22:41:38.491 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.491 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.491 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.491 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6556 3.19 3:9a3eec94:::rbd_data.115f1ca54530.0000000000000004:4 [read 5617664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:38.492 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.477071+0000", 2024-09-17T22:41:38.492 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 11.946645935999999, 2024-09-17T22:41:38.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026451061000000001, 2024-09-17T22:41:38.492 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.492 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.492 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.492 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.492 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.492 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.492 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6556 2024-09-17T22:41:38.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.492 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477071+0000", 2024-09-17T22:41:38.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477071+0000", 2024-09-17T22:41:38.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477072+0000", 2024-09-17T22:41:38.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3799999999999998e-07 2024-09-17T22:41:38.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477074+0000", 2024-09-17T22:41:38.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9870000000000002e-06 2024-09-17T22:41:38.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477080+0000", 2024-09-17T22:41:38.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3459999999999999e-06 2024-09-17T22:41:38.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.477084+0000", 2024-09-17T22:41:38.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4880000000000001e-06 2024-09-17T22:41:38.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503438+0000", 2024-09-17T22:41:38.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026353225000000001 2024-09-17T22:41:38.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503473+0000", 2024-09-17T22:41:38.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5472000000000002e-05 2024-09-17T22:41:38.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.503522+0000", 2024-09-17T22:41:38.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9104999999999998e-05 2024-09-17T22:41:38.495 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.496 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.496 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.496 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:33624 3.7 3:e206720e:::rbd_data.115fe0c30ef6.0000000000000042:5 [read 231424~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:38.496 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.501400+0000", 2024-09-17T22:41:38.496 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.9223170570000008, 2024-09-17T22:41:38.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024723893, 2024-09-17T22:41:38.496 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.496 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.496 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.496 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.496 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.496 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.496 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 33624 2024-09-17T22:41:38.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.497 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501400+0000", 2024-09-17T22:41:38.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501400+0000", 2024-09-17T22:41:38.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501400+0000", 2024-09-17T22:41:38.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8100000000000003e-07 2024-09-17T22:41:38.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501403+0000", 2024-09-17T22:41:38.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1500000000000002e-06 2024-09-17T22:41:38.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501409+0000", 2024-09-17T22:41:38.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3779999999999998e-06 2024-09-17T22:41:38.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501414+0000", 2024-09-17T22:41:38.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1139999999999997e-06 2024-09-17T22:41:38.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.525952+0000", 2024-09-17T22:41:38.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024537792999999999 2024-09-17T22:41:38.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526035+0000", 2024-09-17T22:41:38.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2948000000000006e-05 2024-09-17T22:41:38.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526124+0000", 2024-09-17T22:41:38.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9029000000000004e-05 2024-09-17T22:41:38.500 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.500 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.500 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.500 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:33625 3.7 3:e206720e:::rbd_data.115fe0c30ef6.0000000000000042:5 [read 245760~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:38.500 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.501420+0000", 2024-09-17T22:41:38.500 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.9222969990000003, 2024-09-17T22:41:38.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024907489000000001, 2024-09-17T22:41:38.500 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.501 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.501 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.501 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.501 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.501 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.501 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 33625 2024-09-17T22:41:38.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.501 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501420+0000", 2024-09-17T22:41:38.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501420+0000", 2024-09-17T22:41:38.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501420+0000", 2024-09-17T22:41:38.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:41:38.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501428+0000", 2024-09-17T22:41:38.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6520000000000006e-06 2024-09-17T22:41:38.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501435+0000", 2024-09-17T22:41:38.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9229999999999996e-06 2024-09-17T22:41:38.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501440+0000", 2024-09-17T22:41:38.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3569999999999997e-06 2024-09-17T22:41:38.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526155+0000", 2024-09-17T22:41:38.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024715114999999999 2024-09-17T22:41:38.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526233+0000", 2024-09-17T22:41:38.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7560999999999998e-05 2024-09-17T22:41:38.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526328+0000", 2024-09-17T22:41:38.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4526999999999998e-05 2024-09-17T22:41:38.504 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.504 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.504 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.505 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:33626 3.7 3:e206720e:::rbd_data.115fe0c30ef6.0000000000000042:5 [read 253952~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:38.505 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.501477+0000", 2024-09-17T22:41:38.505 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.9222403089999993, 2024-09-17T22:41:38.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025046036000000001, 2024-09-17T22:41:38.505 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.505 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.505 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.505 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.505 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.505 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.505 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 33626 2024-09-17T22:41:38.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.505 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501477+0000", 2024-09-17T22:41:38.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501477+0000", 2024-09-17T22:41:38.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501478+0000", 2024-09-17T22:41:38.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0299999999999998e-07 2024-09-17T22:41:38.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501483+0000", 2024-09-17T22:41:38.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.519e-06 2024-09-17T22:41:38.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501492+0000", 2024-09-17T22:41:38.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4819999999999994e-06 2024-09-17T22:41:38.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501498+0000", 2024-09-17T22:41:38.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6780000000000003e-06 2024-09-17T22:41:38.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526356+0000", 2024-09-17T22:41:38.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024858203999999998 2024-09-17T22:41:38.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526427+0000", 2024-09-17T22:41:38.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1021999999999999e-05 2024-09-17T22:41:38.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526523+0000", 2024-09-17T22:41:38.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5328000000000006e-05 2024-09-17T22:41:38.508 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.508 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.508 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.509 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:33627 3.7 3:e206720e:::rbd_data.115fe0c30ef6.0000000000000042:5 [read 260096~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:38.509 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.501510+0000", 2024-09-17T22:41:38.509 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.9222067500000009, 2024-09-17T22:41:38.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025207994000000001, 2024-09-17T22:41:38.509 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.509 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.509 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.509 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 33627 2024-09-17T22:41:38.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.509 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501510+0000", 2024-09-17T22:41:38.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501510+0000", 2024-09-17T22:41:38.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501511+0000", 2024-09-17T22:41:38.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3399999999999999e-07 2024-09-17T22:41:38.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501513+0000", 2024-09-17T22:41:38.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5009999999999999e-06 2024-09-17T22:41:38.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501520+0000", 2024-09-17T22:41:38.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7800000000000003e-06 2024-09-17T22:41:38.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.501525+0000", 2024-09-17T22:41:38.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3399999999999997e-06 2024-09-17T22:41:38.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526550+0000", 2024-09-17T22:41:38.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025024066000000001 2024-09-17T22:41:38.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526623+0000", 2024-09-17T22:41:38.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3240999999999999e-05 2024-09-17T22:41:38.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.526718+0000", 2024-09-17T22:41:38.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5532000000000006e-05 2024-09-17T22:41:38.513 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.513 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.513 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.513 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:51796 3.1b 3:dc9348e8:::rbd_data.115fe9ac8535.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T22:41:38.513 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:33.665030+0000", 2024-09-17T22:41:38.513 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.758687471, 2024-09-17T22:41:38.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027937322000000001, 2024-09-17T22:41:38.513 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.513 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.513 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:38.513 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.514 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.514 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.514 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 51796 2024-09-17T22:41:38.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.514 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.665030+0000", 2024-09-17T22:41:38.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.665030+0000", 2024-09-17T22:41:38.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.665030+0000", 2024-09-17T22:41:38.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.37e-07 2024-09-17T22:41:38.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.665034+0000", 2024-09-17T22:41:38.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8539999999999999e-06 2024-09-17T22:41:38.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.665043+0000", 2024-09-17T22:41:38.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6000000000000007e-06 2024-09-17T22:41:38.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.665047+0000", 2024-09-17T22:41:38.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4490000000000001e-06 2024-09-17T22:41:38.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.665095+0000", 2024-09-17T22:41:38.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7703000000000001e-05 2024-09-17T22:41:38.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.665180+0000", 2024-09-17T22:41:38.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4950000000000005e-05 2024-09-17T22:41:38.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 0", 2024-09-17T22:41:38.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.669629+0000", 2024-09-17T22:41:38.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0044492919999999997 2024-09-17T22:41:38.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:38.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.692861+0000", 2024-09-17T22:41:38.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023232222 2024-09-17T22:41:38.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:41:38.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.692874+0000", 2024-09-17T22:41:38.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2813000000000001e-05 2024-09-17T22:41:38.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:38.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.692911+0000", 2024-09-17T22:41:38.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7064999999999998e-05 2024-09-17T22:41:38.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.692967+0000", 2024-09-17T22:41:38.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5637000000000003e-05 2024-09-17T22:41:38.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.518 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.519 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.519 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:38.520 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:41:38.523 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:38.520+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:38.623 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:38.621+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:38.626 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:38.626 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:38.626 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:41:38.626 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:38.634 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:41:38.724 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:38.721+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:38.763 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:38.763 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:38.763 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:38.763 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:38.763 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:38.772 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:41:38.825 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:38.823+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:38.847 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:38.847 INFO:teuthology.orchestra.run.smithi115.stdout:693 writesame 0x682d8e6 thru 0x683463d data_size 0x4(0x6d58 bytes) 2024-09-17T22:41:38.858 INFO:teuthology.orchestra.run.smithi115.stdout:694 writesame 0x3f6ca56 thru 0x3f6fe7b data_size 0x4b(0x3426 bytes) 2024-09-17T22:41:38.864 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:38.873 INFO:teuthology.orchestra.run.smithi115.stdout:696 trunc from 0x936b47e to 0x1834074 2024-09-17T22:41:38.919 INFO:teuthology.orchestra.run.smithi115.stdout:700 write 0xa3f148e thru 0xa3f7675 (0x61e8 bytes) 2024-09-17T22:41:38.922 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:38.922 INFO:teuthology.orchestra.run.smithi115.stdout:702 read 0x537a755 thru 0x53881d6 (0xda82 bytes) 2024-09-17T22:41:38.922 INFO:teuthology.orchestra.run.smithi115.stdout:703 read 0x4b34527 thru 0x4b39dc0 (0x589a bytes) 2024-09-17T22:41:38.924 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:1016 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-17T22:41:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:24.899766+0000", 2024-09-17T22:41:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.020960577, 2024-09-17T22:41:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.020466634000000001, 2024-09-17T22:41:38.924 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 1016 2024-09-17T22:41:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899766+0000", 2024-09-17T22:41:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899766+0000", 2024-09-17T22:41:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899766+0000", 2024-09-17T22:41:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4799999999999997e-07 2024-09-17T22:41:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899771+0000", 2024-09-17T22:41:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8489999999999997e-06 2024-09-17T22:41:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899782+0000", 2024-09-17T22:41:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0702e-05 2024-09-17T22:41:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899787+0000", 2024-09-17T22:41:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1460000000000004e-06 2024-09-17T22:41:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899823+0000", 2024-09-17T22:41:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5478000000000002e-05 2024-09-17T22:41:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.899878+0000", 2024-09-17T22:41:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5034999999999997e-05 2024-09-17T22:41:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T22:41:38.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.903518+0000", 2024-09-17T22:41:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036402240000000001 2024-09-17T22:41:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.920108+0000", 2024-09-17T22:41:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016589650000000001 2024-09-17T22:41:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:41:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.920127+0000", 2024-09-17T22:41:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9504000000000002e-05 2024-09-17T22:41:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.920180+0000", 2024-09-17T22:41:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3276999999999998e-05 2024-09-17T22:41:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:24.920232+0000", 2024-09-17T22:41:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2021000000000002e-05 2024-09-17T22:41:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.930 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:6105 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 5564416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T22:41:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:26.471471+0000", 2024-09-17T22:41:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 12.449255686000001, 2024-09-17T22:41:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017349954000000001, 2024-09-17T22:41:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 6105 2024-09-17T22:41:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471471+0000", 2024-09-17T22:41:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471471+0000", 2024-09-17T22:41:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471471+0000", 2024-09-17T22:41:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7599999999999998e-07 2024-09-17T22:41:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471481+0000", 2024-09-17T22:41:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0207000000000001e-05 2024-09-17T22:41:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471488+0000", 2024-09-17T22:41:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0700000000000001e-06 2024-09-17T22:41:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.471493+0000", 2024-09-17T22:41:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1900000000000003e-06 2024-09-17T22:41:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.481097+0000", 2024-09-17T22:41:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0096031459999999999 2024-09-17T22:41:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.481170+0000", 2024-09-17T22:41:38.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3187000000000002e-05 2024-09-17T22:41:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:26.488821+0000", 2024-09-17T22:41:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0076507779999999996 2024-09-17T22:41:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35632 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2605056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.533932+0000", 2024-09-17T22:41:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.3867945109999997, 2024-09-17T22:41:38.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016583209000000002, 2024-09-17T22:41:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35632 2024-09-17T22:41:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.533932+0000", 2024-09-17T22:41:38.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.533932+0000", 2024-09-17T22:41:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.533932+0000", 2024-09-17T22:41:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.05e-07 2024-09-17T22:41:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.533933+0000", 2024-09-17T22:41:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6200000000000006e-07 2024-09-17T22:41:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.533936+0000", 2024-09-17T22:41:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9000000000000002e-06 2024-09-17T22:41:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.533939+0000", 2024-09-17T22:41:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6639999999999998e-06 2024-09-17T22:41:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.550351+0000", 2024-09-17T22:41:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016412663000000001 2024-09-17T22:41:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.550423+0000", 2024-09-17T22:41:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2175999999999995e-05 2024-09-17T22:41:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.550515+0000", 2024-09-17T22:41:38.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1638999999999995e-05 2024-09-17T22:41:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35695 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2553856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.536131+0000", 2024-09-17T22:41:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.3845957609999999, 2024-09-17T22:41:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.01647641, 2024-09-17T22:41:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35695 2024-09-17T22:41:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536131+0000", 2024-09-17T22:41:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536131+0000", 2024-09-17T22:41:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536131+0000", 2024-09-17T22:41:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.39e-07 2024-09-17T22:41:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536133+0000", 2024-09-17T22:41:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.95e-06 2024-09-17T22:41:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536138+0000", 2024-09-17T22:41:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1950000000000002e-06 2024-09-17T22:41:38.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536143+0000", 2024-09-17T22:41:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7779999999999998e-06 2024-09-17T22:41:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.552473+0000", 2024-09-17T22:41:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016330458999999999 2024-09-17T22:41:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.552517+0000", 2024-09-17T22:41:38.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3680999999999997e-05 2024-09-17T22:41:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.552607+0000", 2024-09-17T22:41:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0007999999999996e-05 2024-09-17T22:41:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35696 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2609152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.536148+0000", 2024-09-17T22:41:38.944 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.3845778509999995, 2024-09-17T22:41:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016687067999999999, 2024-09-17T22:41:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35696 2024-09-17T22:41:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536148+0000", 2024-09-17T22:41:38.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536148+0000", 2024-09-17T22:41:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536149+0000", 2024-09-17T22:41:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:41:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536155+0000", 2024-09-17T22:41:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8690000000000001e-06 2024-09-17T22:41:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536161+0000", 2024-09-17T22:41:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7050000000000002e-06 2024-09-17T22:41:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536166+0000", 2024-09-17T22:41:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5909999999999998e-06 2024-09-17T22:41:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.552657+0000", 2024-09-17T22:41:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016490916000000001 2024-09-17T22:41:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.552741+0000", 2024-09-17T22:41:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3852999999999998e-05 2024-09-17T22:41:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.552836+0000", 2024-09-17T22:41:38.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4748999999999997e-05 2024-09-17T22:41:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35742 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2670592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.536998+0000", 2024-09-17T22:41:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.3837283849999995, 2024-09-17T22:41:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016453115000000001, 2024-09-17T22:41:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.949 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35742 2024-09-17T22:41:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536998+0000", 2024-09-17T22:41:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536998+0000", 2024-09-17T22:41:38.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.536998+0000", 2024-09-17T22:41:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:41:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537000+0000", 2024-09-17T22:41:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1299999999999999e-06 2024-09-17T22:41:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537007+0000", 2024-09-17T22:41:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5289999999999997e-06 2024-09-17T22:41:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537012+0000", 2024-09-17T22:41:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2970000000000003e-06 2024-09-17T22:41:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553368+0000", 2024-09-17T22:41:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016356103 2024-09-17T22:41:38.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553404+0000", 2024-09-17T22:41:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5729000000000003e-05 2024-09-17T22:41:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553451+0000", 2024-09-17T22:41:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6935999999999999e-05 2024-09-17T22:41:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.953 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35743 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2744320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.537019+0000", 2024-09-17T22:41:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.3837075500000005, 2024-09-17T22:41:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016542099000000001, 2024-09-17T22:41:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35743 2024-09-17T22:41:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.954 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537019+0000", 2024-09-17T22:41:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537019+0000", 2024-09-17T22:41:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537019+0000", 2024-09-17T22:41:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6300000000000001e-07 2024-09-17T22:41:38.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537021+0000", 2024-09-17T22:41:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9080000000000002e-06 2024-09-17T22:41:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537027+0000", 2024-09-17T22:41:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3090000000000001e-06 2024-09-17T22:41:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537033+0000", 2024-09-17T22:41:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4539999999999998e-06 2024-09-17T22:41:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553475+0000", 2024-09-17T22:41:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016442142999999999 2024-09-17T22:41:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553510+0000", 2024-09-17T22:41:38.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5030999999999997e-05 2024-09-17T22:41:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553561+0000", 2024-09-17T22:41:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0890999999999998e-05 2024-09-17T22:41:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35745 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2480128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.537054+0000", 2024-09-17T22:41:38.958 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.3836725780000005, 2024-09-17T22:41:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016621817000000001, 2024-09-17T22:41:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35745 2024-09-17T22:41:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537054+0000", 2024-09-17T22:41:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537054+0000", 2024-09-17T22:41:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537055+0000", 2024-09-17T22:41:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0400000000000005e-07 2024-09-17T22:41:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537059+0000", 2024-09-17T22:41:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9910000000000004e-06 2024-09-17T22:41:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537070+0000", 2024-09-17T22:41:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1398e-05 2024-09-17T22:41:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.537077+0000", 2024-09-17T22:41:38.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4959999999999999e-06 2024-09-17T22:41:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553585+0000", 2024-09-17T22:41:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016508020000000002 2024-09-17T22:41:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553621+0000", 2024-09-17T22:41:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6013999999999998e-05 2024-09-17T22:41:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.553676+0000", 2024-09-17T22:41:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4094000000000001e-05 2024-09-17T22:41:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35791 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2615296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.538661+0000", 2024-09-17T22:41:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.3820654030000004, 2024-09-17T22:41:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016411623, 2024-09-17T22:41:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35791 2024-09-17T22:41:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538661+0000", 2024-09-17T22:41:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538661+0000", 2024-09-17T22:41:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538661+0000", 2024-09-17T22:41:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3399999999999999e-07 2024-09-17T22:41:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538664+0000", 2024-09-17T22:41:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1289999999999999e-06 2024-09-17T22:41:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538685+0000", 2024-09-17T22:41:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1608999999999999e-05 2024-09-17T22:41:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538691+0000", 2024-09-17T22:41:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7629999999999997e-06 2024-09-17T22:41:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.554892+0000", 2024-09-17T22:41:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016200951000000002 2024-09-17T22:41:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.554962+0000", 2024-09-17T22:41:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9962999999999994e-05 2024-09-17T22:41:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555073+0000", 2024-09-17T22:41:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000110674 2024-09-17T22:41:38.967 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35805 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2674688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.538700+0000", 2024-09-17T22:41:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.3820267739999998, 2024-09-17T22:41:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016568598, 2024-09-17T22:41:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35805 2024-09-17T22:41:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538700+0000", 2024-09-17T22:41:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538700+0000", 2024-09-17T22:41:38.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538700+0000", 2024-09-17T22:41:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8500000000000002e-07 2024-09-17T22:41:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538702+0000", 2024-09-17T22:41:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.447e-06 2024-09-17T22:41:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538709+0000", 2024-09-17T22:41:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2530000000000001e-06 2024-09-17T22:41:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538714+0000", 2024-09-17T22:41:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8770000000000001e-06 2024-09-17T22:41:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555104+0000", 2024-09-17T22:41:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016390372 2024-09-17T22:41:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555168+0000", 2024-09-17T22:41:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3559000000000006e-05 2024-09-17T22:41:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555268+0000", 2024-09-17T22:41:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100605 2024-09-17T22:41:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.972 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35806 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2748416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.538719+0000", 2024-09-17T22:41:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.3820068180000007, 2024-09-17T22:41:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016746647999999999, 2024-09-17T22:41:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35806 2024-09-17T22:41:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538719+0000", 2024-09-17T22:41:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538719+0000", 2024-09-17T22:41:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538720+0000", 2024-09-17T22:41:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:41:38.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538722+0000", 2024-09-17T22:41:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9429999999999999e-06 2024-09-17T22:41:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538727+0000", 2024-09-17T22:41:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6559999999999997e-06 2024-09-17T22:41:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538732+0000", 2024-09-17T22:41:38.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7949999999999998e-06 2024-09-17T22:41:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555297+0000", 2024-09-17T22:41:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016564861 2024-09-17T22:41:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555358+0000", 2024-09-17T22:41:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1177000000000001e-05 2024-09-17T22:41:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555466+0000", 2024-09-17T22:41:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010785000000000001 2024-09-17T22:41:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35809 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2410496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.538738+0000", 2024-09-17T22:41:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.3819884239999993, 2024-09-17T22:41:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016923453000000001, 2024-09-17T22:41:38.977 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35809 2024-09-17T22:41:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538738+0000", 2024-09-17T22:41:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538738+0000", 2024-09-17T22:41:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538738+0000", 2024-09-17T22:41:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:41:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538740+0000", 2024-09-17T22:41:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9080000000000002e-06 2024-09-17T22:41:38.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538746+0000", 2024-09-17T22:41:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6330000000000001e-06 2024-09-17T22:41:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538750+0000", 2024-09-17T22:41:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5040000000000004e-06 2024-09-17T22:41:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555497+0000", 2024-09-17T22:41:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016747171000000002 2024-09-17T22:41:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555559+0000", 2024-09-17T22:41:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1822999999999994e-05 2024-09-17T22:41:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555661+0000", 2024-09-17T22:41:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010206 2024-09-17T22:41:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.981 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35810 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2484224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.538756+0000", 2024-09-17T22:41:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.3819699389999993, 2024-09-17T22:41:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017104030999999999, 2024-09-17T22:41:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35810 2024-09-17T22:41:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538756+0000", 2024-09-17T22:41:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538756+0000", 2024-09-17T22:41:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538757+0000", 2024-09-17T22:41:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:41:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538759+0000", 2024-09-17T22:41:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.778e-06 2024-09-17T22:41:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538764+0000", 2024-09-17T22:41:38.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6840000000000001e-06 2024-09-17T22:41:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.538771+0000", 2024-09-17T22:41:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5590000000000002e-06 2024-09-17T22:41:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555692+0000", 2024-09-17T22:41:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016921281 2024-09-17T22:41:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555752+0000", 2024-09-17T22:41:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0402000000000002e-05 2024-09-17T22:41:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555860+0000", 2024-09-17T22:41:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010796100000000001 2024-09-17T22:41:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35823 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2543616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.539339+0000", 2024-09-17T22:41:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.3813869220000008, 2024-09-17T22:41:38.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016657568000000001, 2024-09-17T22:41:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35823 2024-09-17T22:41:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539339+0000", 2024-09-17T22:41:38.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539339+0000", 2024-09-17T22:41:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539340+0000", 2024-09-17T22:41:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2900000000000002e-07 2024-09-17T22:41:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539344+0000", 2024-09-17T22:41:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0729999999999998e-06 2024-09-17T22:41:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539356+0000", 2024-09-17T22:41:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1484000000000001e-05 2024-09-17T22:41:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539360+0000", 2024-09-17T22:41:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7260000000000002e-06 2024-09-17T22:41:38.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555893+0000", 2024-09-17T22:41:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016532958 2024-09-17T22:41:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555941+0000", 2024-09-17T22:41:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7729999999999999e-05 2024-09-17T22:41:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.555997+0000", 2024-09-17T22:41:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5767999999999999e-05 2024-09-17T22:41:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35824 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2617344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.539370+0000", 2024-09-17T22:41:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.3813565249999993, 2024-09-17T22:41:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016758449000000002, 2024-09-17T22:41:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35824 2024-09-17T22:41:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539370+0000", 2024-09-17T22:41:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539370+0000", 2024-09-17T22:41:38.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539370+0000", 2024-09-17T22:41:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3799999999999998e-07 2024-09-17T22:41:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539372+0000", 2024-09-17T22:41:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8160000000000001e-06 2024-09-17T22:41:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539376+0000", 2024-09-17T22:41:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.844e-06 2024-09-17T22:41:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539379+0000", 2024-09-17T22:41:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0910000000000001e-06 2024-09-17T22:41:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556031+0000", 2024-09-17T22:41:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016651750999999999 2024-09-17T22:41:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556072+0000", 2024-09-17T22:41:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1690999999999998e-05 2024-09-17T22:41:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556128+0000", 2024-09-17T22:41:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5918000000000002e-05 2024-09-17T22:41:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35837 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2676736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.539383+0000", 2024-09-17T22:41:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.3813435559999991, 2024-09-17T22:41:38.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0168665, 2024-09-17T22:41:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35837 2024-09-17T22:41:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539383+0000", 2024-09-17T22:41:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539383+0000", 2024-09-17T22:41:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539383+0000", 2024-09-17T22:41:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2399999999999999e-07 2024-09-17T22:41:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539384+0000", 2024-09-17T22:41:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.305e-06 2024-09-17T22:41:38.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539387+0000", 2024-09-17T22:41:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0939999999999999e-06 2024-09-17T22:41:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539390+0000", 2024-09-17T22:41:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.678e-06 2024-09-17T22:41:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556152+0000", 2024-09-17T22:41:38.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016762353000000001 2024-09-17T22:41:38.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:38.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556193+0000", 2024-09-17T22:41:38.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0399999999999999e-05 2024-09-17T22:41:38.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:38.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:38.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:38.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556249+0000", 2024-09-17T22:41:38.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6446000000000001e-05 2024-09-17T22:41:38.999 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.999 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:38.999 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:38.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.000 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35838 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2750464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.000 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.539393+0000", 2024-09-17T22:41:39.000 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.3813330369999992, 2024-09-17T22:41:39.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017005407, 2024-09-17T22:41:39.000 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.000 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.000 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.000 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.000 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.000 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.000 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35838 2024-09-17T22:41:39.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.001 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539393+0000", 2024-09-17T22:41:39.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539393+0000", 2024-09-17T22:41:39.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539393+0000", 2024-09-17T22:41:39.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.05e-07 2024-09-17T22:41:39.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539395+0000", 2024-09-17T22:41:39.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.11e-06 2024-09-17T22:41:39.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539398+0000", 2024-09-17T22:41:39.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0240000000000002e-06 2024-09-17T22:41:39.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539401+0000", 2024-09-17T22:41:39.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1729999999999999e-06 2024-09-17T22:41:39.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556291+0000", 2024-09-17T22:41:39.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016890179000000002 2024-09-17T22:41:39.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556343+0000", 2024-09-17T22:41:39.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2380999999999999e-05 2024-09-17T22:41:39.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556399+0000", 2024-09-17T22:41:39.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5334999999999998e-05 2024-09-17T22:41:39.004 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.004 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.004 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.004 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35841 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2412544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.004 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.539404+0000", 2024-09-17T22:41:39.004 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.3813224880000003, 2024-09-17T22:41:39.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017168085999999999, 2024-09-17T22:41:39.004 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.004 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.004 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.005 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.005 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.005 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.005 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35841 2024-09-17T22:41:39.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.005 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539404+0000", 2024-09-17T22:41:39.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539404+0000", 2024-09-17T22:41:39.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539404+0000", 2024-09-17T22:41:39.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0599999999999999e-07 2024-09-17T22:41:39.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539407+0000", 2024-09-17T22:41:39.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3890000000000001e-06 2024-09-17T22:41:39.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539411+0000", 2024-09-17T22:41:39.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2009999999999999e-06 2024-09-17T22:41:39.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539413+0000", 2024-09-17T22:41:39.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6110000000000001e-06 2024-09-17T22:41:39.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556421+0000", 2024-09-17T22:41:39.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017008079999999998 2024-09-17T22:41:39.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556470+0000", 2024-09-17T22:41:39.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8987999999999999e-05 2024-09-17T22:41:39.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556572+0000", 2024-09-17T22:41:39.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101611 2024-09-17T22:41:39.008 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.008 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.008 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.008 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35845 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2486272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.008 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.539416+0000", 2024-09-17T22:41:39.009 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.3813098339999996, 2024-09-17T22:41:39.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017358905000000001, 2024-09-17T22:41:39.009 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.009 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.009 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.009 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.009 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.009 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.009 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35845 2024-09-17T22:41:39.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.009 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539416+0000", 2024-09-17T22:41:39.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539416+0000", 2024-09-17T22:41:39.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539417+0000", 2024-09-17T22:41:39.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3200000000000001e-07 2024-09-17T22:41:39.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539418+0000", 2024-09-17T22:41:39.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.094e-06 2024-09-17T22:41:39.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539421+0000", 2024-09-17T22:41:39.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0010000000000002e-06 2024-09-17T22:41:39.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.539423+0000", 2024-09-17T22:41:39.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5919999999999999e-06 2024-09-17T22:41:39.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556602+0000", 2024-09-17T22:41:39.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017178729 2024-09-17T22:41:39.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556677+0000", 2024-09-17T22:41:39.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4825000000000003e-05 2024-09-17T22:41:39.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.556775+0000", 2024-09-17T22:41:39.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8431999999999995e-05 2024-09-17T22:41:39.013 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.013 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.013 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.013 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:35870 3.1e 3:7bddf562:::rbd_data.115f1ca54530.0000000000000000:4 [read 2678784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.013 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.540812+0000", 2024-09-17T22:41:39.013 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.379914436, 2024-09-17T22:41:39.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016561679999999999, 2024-09-17T22:41:39.013 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.013 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.013 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.013 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.014 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 35870 2024-09-17T22:41:39.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.014 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.540812+0000", 2024-09-17T22:41:39.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.540812+0000", 2024-09-17T22:41:39.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.540812+0000", 2024-09-17T22:41:39.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7300000000000001e-07 2024-09-17T22:41:39.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.540814+0000", 2024-09-17T22:41:39.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1610000000000001e-06 2024-09-17T22:41:39.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.540821+0000", 2024-09-17T22:41:39.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2249999999999997e-06 2024-09-17T22:41:39.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.540827+0000", 2024-09-17T22:41:39.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4540000000000004e-06 2024-09-17T22:41:39.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.557219+0000", 2024-09-17T22:41:39.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.016392317 2024-09-17T22:41:39.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.557290+0000", 2024-09-17T22:41:39.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0239000000000003e-05 2024-09-17T22:41:39.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.557374+0000", 2024-09-17T22:41:39.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3811000000000001e-05 2024-09-17T22:41:39.017 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.017 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.017 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.017 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.017 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.017 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:39.020 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:41:39.022 INFO:teuthology.orchestra.run.smithi115.stdout:704 write 0xc0b42ae thru 0xc0ba8ed (0x6640 bytes) 2024-09-17T22:41:39.022 INFO:teuthology.orchestra.run.smithi115.stdout:705 read 0x1e2219 thru 0x1f17c6 (0xf5ae bytes) 2024-09-17T22:41:39.022 INFO:teuthology.orchestra.run.smithi115.stdout:706 writesame 0xa58fa0e thru 0xa591e22 data_size 0x1(0x2415 bytes) 2024-09-17T22:41:39.022 INFO:teuthology.orchestra.run.smithi115.stdout:708 read 0xbf1bd8a thru 0xbf1d648 (0x18bf bytes) 2024-09-17T22:41:39.022 INFO:teuthology.orchestra.run.smithi115.stdout:709 read 0x9da6de3 thru 0x9db587a (0xea98 bytes) 2024-09-17T22:41:39.022 INFO:teuthology.orchestra.run.smithi115.stdout:710 read 0x3366ed7 thru 0x336fb01 (0x8c2b bytes) 2024-09-17T22:41:39.022 INFO:teuthology.orchestra.run.smithi115.stdout:713 write 0x75c892e thru 0x75cdb53 (0x5226 bytes) 2024-09-17T22:41:39.022 INFO:teuthology.orchestra.run.smithi115.stdout:714 writesame 0x36256c7 thru 0x362bdd7 data_size 0x5(0x6711 bytes) 2024-09-17T22:41:39.022 INFO:teuthology.orchestra.run.smithi115.stdout:717 read 0x41c3596 thru 0x41d2926 (0xf391 bytes) 2024-09-17T22:41:39.022 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:39.022 INFO:teuthology.orchestra.run.smithi115.stdout:719 write 0x6544038 thru 0x65495dd (0x55a6 bytes) 2024-09-17T22:41:39.023 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:39.023 INFO:teuthology.orchestra.run.smithi115.stdout:721 writesame 0xee0e2d9 thru 0xee150f0 data_size 0xd(0x6e18 bytes) 2024-09-17T22:41:39.023 INFO:teuthology.orchestra.run.smithi115.stdout:722 read 0x2c2ea30 thru 0x2c2f63b (0xc0c bytes) 2024-09-17T22:41:39.023 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:39.023 INFO:teuthology.orchestra.run.smithi115.stdout:725 read 0xafa6510 thru 0xafb02db (0x9dcc bytes) 2024-09-17T22:41:39.023 INFO:teuthology.orchestra.run.smithi115.stdout:726 read 0x9ca4107 thru 0x9cb35c4 (0xf4be bytes) 2024-09-17T22:41:39.023 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:39.023 INFO:teuthology.orchestra.run.smithi115.stdout:728 read 0x103ad5c thru 0x103bd0e (0xfb3 bytes) 2024-09-17T22:41:39.023 INFO:teuthology.orchestra.run.smithi115.stdout:729 writesame 0x82028c3 thru 0x82105ca data_size 0x2c(0xdd08 bytes) 2024-09-17T22:41:39.023 INFO:teuthology.orchestra.run.smithi115.stdout:730 read 0xc2031a5 thru 0xc20e7da (0xb636 bytes) 2024-09-17T22:41:39.023 INFO:teuthology.orchestra.run.smithi115.stdout:732 write 0x71c6f9a thru 0x71ccccd (0x5d34 bytes) 2024-09-17T22:41:39.023 INFO:teuthology.orchestra.run.smithi115.stdout:733 trunc from 0xee150f1 to 0x7d0305d 2024-09-17T22:41:39.023 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:39.024 INFO:teuthology.orchestra.run.smithi115.stdout:735 punch from 0x5807095 to 0x5808a95, (0x1a00 bytes) 2024-09-17T22:41:39.024 INFO:teuthology.orchestra.run.smithi115.stdout:737 writesame 0xbb2fafd thru 0xbb3615b data_size 0x1(0x665f bytes) 2024-09-17T22:41:39.024 INFO:teuthology.orchestra.run.smithi115.stdout:738 trunc from 0xbb3615c to 0x285fe35 2024-09-17T22:41:39.024 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:39.016+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:39.024 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T22:41:39.029 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:41:39.035 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-17T22:41:39.036 INFO:teuthology.orchestra.run.smithi115.stdout:739 trunc from 0x285fe35 to 0x79d52e6 2024-09-17T22:41:39.049 INFO:teuthology.orchestra.run.smithi115.stdout:740 write 0xa78cb02 thru 0xa79bda1 (0xf2a0 bytes) 2024-09-17T22:41:39.053 INFO:teuthology.orchestra.run.smithi115.stdout:741 write 0x7c2bc58 thru 0x7c331bb (0x7564 bytes) 2024-09-17T22:41:39.056 INFO:teuthology.orchestra.run.smithi115.stdout:743 punch from 0x5c56282 to 0x5c5d4fe, (0x727c bytes) 2024-09-17T22:41:39.056 INFO:teuthology.orchestra.run.smithi115.stdout:744 punch from 0x842e224 to 0x8438a26, (0xa802 bytes) 2024-09-17T22:41:39.062 INFO:teuthology.orchestra.run.smithi115.stdout:745 write 0xad71347 thru 0xad73f1e (0x2bd8 bytes) 2024-09-17T22:41:39.067 INFO:teuthology.orchestra.run.smithi115.stdout:746 trunc from 0xad73f1f to 0xab4a233 2024-09-17T22:41:39.078 INFO:teuthology.orchestra.run.smithi115.stdout:747 write 0x8f3af48 thru 0x8f4abba (0xfc73 bytes) 2024-09-17T22:41:39.083 INFO:teuthology.orchestra.run.smithi115.stdout:751 read 0x3e864bc thru 0x3e8c0c0 (0x5c05 bytes) 2024-09-17T22:41:39.083 INFO:teuthology.orchestra.run.smithi115.stdout:752 write 0x5dbcb9b thru 0x5dc1704 (0x4b6a bytes) 2024-09-17T22:41:39.087 INFO:teuthology.orchestra.run.smithi115.stdout:753 clone 11 order 24 su 4194304 sc 13 2024-09-17T22:41:39.119 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:39.116+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:39.130 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:39.130 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:39.130 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:41:39.130 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:39.139 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:41:39.220 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:39.218+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:39.282 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:39.282 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:39.282 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:39.282 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:39.282 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:39.291 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:41:39.335 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:39.333+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:39.397 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38083 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.763873+0000", 2024-09-17T22:41:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.6296507760000001, 2024-09-17T22:41:39.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023344747999999999, 2024-09-17T22:41:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38083 2024-09-17T22:41:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763873+0000", 2024-09-17T22:41:39.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763873+0000", 2024-09-17T22:41:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763874+0000", 2024-09-17T22:41:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0899999999999997e-07 2024-09-17T22:41:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763875+0000", 2024-09-17T22:41:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8530000000000001e-06 2024-09-17T22:41:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763881+0000", 2024-09-17T22:41:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4879999999999998e-06 2024-09-17T22:41:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763886+0000", 2024-09-17T22:41:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.587e-06 2024-09-17T22:41:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787057+0000", 2024-09-17T22:41:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023170611000000001 2024-09-17T22:41:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787127+0000", 2024-09-17T22:41:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0259000000000006e-05 2024-09-17T22:41:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787218+0000", 2024-09-17T22:41:39.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0641000000000002e-05 2024-09-17T22:41:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38105 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.763938+0000", 2024-09-17T22:41:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.6295859690000007, 2024-09-17T22:41:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023471315999999999, 2024-09-17T22:41:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.402 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38105 2024-09-17T22:41:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763938+0000", 2024-09-17T22:41:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763938+0000", 2024-09-17T22:41:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763939+0000", 2024-09-17T22:41:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:41:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763941+0000", 2024-09-17T22:41:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9489999999999999e-06 2024-09-17T22:41:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763946+0000", 2024-09-17T22:41:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9139999999999996e-06 2024-09-17T22:41:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763951+0000", 2024-09-17T22:41:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6589999999999998e-06 2024-09-17T22:41:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787249+0000", 2024-09-17T22:41:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023297989000000002 2024-09-17T22:41:39.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787319+0000", 2024-09-17T22:41:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9836e-05 2024-09-17T22:41:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787409+0000", 2024-09-17T22:41:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0457000000000005e-05 2024-09-17T22:41:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38107 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.6295671800000004, 2024-09-17T22:41:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023669550000000001, 2024-09-17T22:41:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38107 2024-09-17T22:41:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.407 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:41:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763959+0000", 2024-09-17T22:41:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.849e-06 2024-09-17T22:41:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763965+0000", 2024-09-17T22:41:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6110000000000001e-06 2024-09-17T22:41:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763969+0000", 2024-09-17T22:41:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-06 2024-09-17T22:41:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787451+0000", 2024-09-17T22:41:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023482281000000001 2024-09-17T22:41:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787516+0000", 2024-09-17T22:41:39.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4930000000000003e-05 2024-09-17T22:41:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787626+0000", 2024-09-17T22:41:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011000700000000001 2024-09-17T22:41:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38141 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764408+0000", 2024-09-17T22:41:39.413 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.6291159480000008, 2024-09-17T22:41:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023422268, 2024-09-17T22:41:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38141 2024-09-17T22:41:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764408+0000", 2024-09-17T22:41:39.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764408+0000", 2024-09-17T22:41:39.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764409+0000", 2024-09-17T22:41:39.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:41:39.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764412+0000", 2024-09-17T22:41:39.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4470000000000002e-06 2024-09-17T22:41:39.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764421+0000", 2024-09-17T22:41:39.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7299999999999994e-06 2024-09-17T22:41:39.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764427+0000", 2024-09-17T22:41:39.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.392e-06 2024-09-17T22:41:39.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787669+0000", 2024-09-17T22:41:39.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023241385 2024-09-17T22:41:39.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787743+0000", 2024-09-17T22:41:39.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4260999999999999e-05 2024-09-17T22:41:39.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787830+0000", 2024-09-17T22:41:39.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7398999999999994e-05 2024-09-17T22:41:39.418 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.418 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.418 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.418 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38143 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.418 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:39.418 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.6290860200000008, 2024-09-17T22:41:39.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023565691, 2024-09-17T22:41:39.418 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.418 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.418 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.419 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.419 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.419 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.419 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38143 2024-09-17T22:41:39.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.419 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:39.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:39.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:39.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6199999999999998e-07 2024-09-17T22:41:39.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764441+0000", 2024-09-17T22:41:39.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1179999999999999e-06 2024-09-17T22:41:39.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764447+0000", 2024-09-17T22:41:39.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1940000000000003e-06 2024-09-17T22:41:39.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764452+0000", 2024-09-17T22:41:39.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9150000000000004e-06 2024-09-17T22:41:39.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787881+0000", 2024-09-17T22:41:39.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023429141000000001 2024-09-17T22:41:39.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787947+0000", 2024-09-17T22:41:39.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5902999999999996e-05 2024-09-17T22:41:39.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788004+0000", 2024-09-17T22:41:39.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6957999999999999e-05 2024-09-17T22:41:39.422 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.422 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.422 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.422 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38150 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.423 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:39.423 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.6290660179999996, 2024-09-17T22:41:39.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023741399, 2024-09-17T22:41:39.423 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.423 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.423 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.423 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.423 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.423 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.423 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38150 2024-09-17T22:41:39.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.423 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:39.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:39.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:39.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8299999999999998e-07 2024-09-17T22:41:39.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764460+0000", 2024-09-17T22:41:39.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.855e-06 2024-09-17T22:41:39.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764466+0000", 2024-09-17T22:41:39.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8200000000000002e-06 2024-09-17T22:41:39.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764471+0000", 2024-09-17T22:41:39.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8319999999999996e-06 2024-09-17T22:41:39.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788032+0000", 2024-09-17T22:41:39.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023561480999999999 2024-09-17T22:41:39.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788106+0000", 2024-09-17T22:41:39.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3406999999999997e-05 2024-09-17T22:41:39.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788199+0000", 2024-09-17T22:41:39.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3621000000000004e-05 2024-09-17T22:41:39.427 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.427 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.427 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.428 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38152 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 83968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.428 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:39.428 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.6290473009999999, 2024-09-17T22:41:39.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023859726000000001, 2024-09-17T22:41:39.428 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.428 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.428 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.428 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38152 2024-09-17T22:41:39.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.429 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:39.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:39.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:39.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3000000000000001e-07 2024-09-17T22:41:39.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764479+0000", 2024-09-17T22:41:39.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8920000000000001e-06 2024-09-17T22:41:39.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764485+0000", 2024-09-17T22:41:39.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:41:39.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764490+0000", 2024-09-17T22:41:39.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6890000000000003e-06 2024-09-17T22:41:39.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788232+0000", 2024-09-17T22:41:39.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023742452000000001 2024-09-17T22:41:39.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788276+0000", 2024-09-17T22:41:39.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3689000000000001e-05 2024-09-17T22:41:39.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788336+0000", 2024-09-17T22:41:39.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0581999999999997e-05 2024-09-17T22:41:39.432 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.432 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.433 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.433 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38184 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 161792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.433 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764589+0000", 2024-09-17T22:41:39.433 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.6289349079999997, 2024-09-17T22:41:39.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023956801, 2024-09-17T22:41:39.433 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.433 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.433 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.433 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.433 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.433 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.433 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38184 2024-09-17T22:41:39.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.434 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764589+0000", 2024-09-17T22:41:39.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764589+0000", 2024-09-17T22:41:39.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764590+0000", 2024-09-17T22:41:39.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7800000000000001e-07 2024-09-17T22:41:39.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764592+0000", 2024-09-17T22:41:39.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6929999999999999e-06 2024-09-17T22:41:39.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764600+0000", 2024-09-17T22:41:39.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1140000000000006e-06 2024-09-17T22:41:39.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764606+0000", 2024-09-17T22:41:39.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6200000000000004e-06 2024-09-17T22:41:39.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788374+0000", 2024-09-17T22:41:39.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023768083999999998 2024-09-17T22:41:39.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788447+0000", 2024-09-17T22:41:39.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2922000000000004e-05 2024-09-17T22:41:39.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788546+0000", 2024-09-17T22:41:39.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8789999999999994e-05 2024-09-17T22:41:39.437 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.437 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.437 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.438 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38186 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.438 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:39.438 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.6289099579999995, 2024-09-17T22:41:39.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024156486000000001, 2024-09-17T22:41:39.438 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.438 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.438 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.438 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.438 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.438 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.438 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38186 2024-09-17T22:41:39.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.439 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:39.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:39.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:39.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:41:39.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764621+0000", 2024-09-17T22:41:39.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6409999999999996e-06 2024-09-17T22:41:39.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764628+0000", 2024-09-17T22:41:39.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0360000000000001e-06 2024-09-17T22:41:39.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764633+0000", 2024-09-17T22:41:39.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0300000000000001e-06 2024-09-17T22:41:39.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788594+0000", 2024-09-17T22:41:39.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023960921999999999 2024-09-17T22:41:39.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788676+0000", 2024-09-17T22:41:39.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2123e-05 2024-09-17T22:41:39.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788771+0000", 2024-09-17T22:41:39.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4284999999999996e-05 2024-09-17T22:41:39.442 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.442 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.442 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.442 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38218 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 350208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.442 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764706+0000", 2024-09-17T22:41:39.443 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.6288177049999994, 2024-09-17T22:41:39.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024324732000000002, 2024-09-17T22:41:39.443 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.443 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.443 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.443 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38218 2024-09-17T22:41:39.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.443 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764706+0000", 2024-09-17T22:41:39.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764706+0000", 2024-09-17T22:41:39.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764707+0000", 2024-09-17T22:41:39.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7199999999999999e-07 2024-09-17T22:41:39.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764710+0000", 2024-09-17T22:41:39.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6819999999999999e-06 2024-09-17T22:41:39.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764716+0000", 2024-09-17T22:41:39.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7900000000000002e-06 2024-09-17T22:41:39.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764722+0000", 2024-09-17T22:41:39.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2560000000000002e-06 2024-09-17T22:41:39.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788867+0000", 2024-09-17T22:41:39.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024144982999999998 2024-09-17T22:41:39.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788941+0000", 2024-09-17T22:41:39.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4529000000000004e-05 2024-09-17T22:41:39.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789031+0000", 2024-09-17T22:41:39.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9920000000000006e-05 2024-09-17T22:41:39.447 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.447 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.447 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.447 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38222 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.447 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764765+0000", 2024-09-17T22:41:39.447 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.6287588510000006, 2024-09-17T22:41:39.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024443910999999999, 2024-09-17T22:41:39.447 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.447 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.447 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.447 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.448 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.448 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.448 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38222 2024-09-17T22:41:39.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.448 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764765+0000", 2024-09-17T22:41:39.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764765+0000", 2024-09-17T22:41:39.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764766+0000", 2024-09-17T22:41:39.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8999999999999996e-07 2024-09-17T22:41:39.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764769+0000", 2024-09-17T22:41:39.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7659999999999999e-06 2024-09-17T22:41:39.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764776+0000", 2024-09-17T22:41:39.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4229999999999999e-06 2024-09-17T22:41:39.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764783+0000", 2024-09-17T22:41:39.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9600000000000003e-06 2024-09-17T22:41:39.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789060+0000", 2024-09-17T22:41:39.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024277343 2024-09-17T22:41:39.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789129+0000", 2024-09-17T22:41:39.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8367999999999993e-05 2024-09-17T22:41:39.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789209+0000", 2024-09-17T22:41:39.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0461000000000001e-05 2024-09-17T22:41:39.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.451 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.452 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38671 3.12 3:4a5b92ac:::rbd_data.115fe0c30ef6.0000000000000011:5 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.452 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778865+0000", 2024-09-17T22:41:39.452 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.6146587910000001, 2024-09-17T22:41:39.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023227351, 2024-09-17T22:41:39.452 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.452 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.452 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.452 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38671 2024-09-17T22:41:39.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.452 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778865+0000", 2024-09-17T22:41:39.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778865+0000", 2024-09-17T22:41:39.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778866+0000", 2024-09-17T22:41:39.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0900000000000002e-07 2024-09-17T22:41:39.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778868+0000", 2024-09-17T22:41:39.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4339999999999999e-06 2024-09-17T22:41:39.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778875+0000", 2024-09-17T22:41:39.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4559999999999997e-06 2024-09-17T22:41:39.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778880+0000", 2024-09-17T22:41:39.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2689999999999999e-06 2024-09-17T22:41:39.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.801978+0000", 2024-09-17T22:41:39.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023098225999999999 2024-09-17T22:41:39.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802035+0000", 2024-09-17T22:41:39.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7123999999999997e-05 2024-09-17T22:41:39.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802093+0000", 2024-09-17T22:41:39.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7432999999999997e-05 2024-09-17T22:41:39.456 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.456 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.456 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.456 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38673 3.12 3:4a5b92ac:::rbd_data.115fe0c30ef6.0000000000000011:5 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.456 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778886+0000", 2024-09-17T22:41:39.456 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.614637643, 2024-09-17T22:41:39.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023389259999999999, 2024-09-17T22:41:39.456 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.457 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.457 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.457 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38673 2024-09-17T22:41:39.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.457 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778886+0000", 2024-09-17T22:41:39.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778886+0000", 2024-09-17T22:41:39.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778887+0000", 2024-09-17T22:41:39.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:41:39.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778889+0000", 2024-09-17T22:41:39.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9379999999999999e-06 2024-09-17T22:41:39.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778895+0000", 2024-09-17T22:41:39.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0360000000000003e-06 2024-09-17T22:41:39.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778900+0000", 2024-09-17T22:41:39.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1220000000000003e-06 2024-09-17T22:41:39.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802126+0000", 2024-09-17T22:41:39.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023226607 2024-09-17T22:41:39.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802208+0000", 2024-09-17T22:41:39.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1157999999999998e-05 2024-09-17T22:41:39.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802276+0000", 2024-09-17T22:41:39.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8122999999999998e-05 2024-09-17T22:41:39.461 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.461 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.461 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.461 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38675 3.12 3:4a5b92ac:::rbd_data.115fe0c30ef6.0000000000000011:5 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.461 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:39.461 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.6146024380000004, 2024-09-17T22:41:39.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023484221999999999, 2024-09-17T22:41:39.461 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.461 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.461 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38675 2024-09-17T22:41:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:39.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9999999999999997e-07 2024-09-17T22:41:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778924+0000", 2024-09-17T22:41:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3039999999999999e-06 2024-09-17T22:41:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778931+0000", 2024-09-17T22:41:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9519999999999997e-06 2024-09-17T22:41:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778937+0000", 2024-09-17T22:41:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3639999999999998e-06 2024-09-17T22:41:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802300+0000", 2024-09-17T22:41:39.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023363544 2024-09-17T22:41:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802347+0000", 2024-09-17T22:41:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6897999999999997e-05 2024-09-17T22:41:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802406+0000", 2024-09-17T22:41:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8560000000000002e-05 2024-09-17T22:41:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38679 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778945+0000", 2024-09-17T22:41:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.6145788220000004, 2024-09-17T22:41:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023578015000000001, 2024-09-17T22:41:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38679 2024-09-17T22:41:39.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778945+0000", 2024-09-17T22:41:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778945+0000", 2024-09-17T22:41:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778946+0000", 2024-09-17T22:41:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.44e-07 2024-09-17T22:41:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778948+0000", 2024-09-17T22:41:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-17T22:41:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778955+0000", 2024-09-17T22:41:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.528e-06 2024-09-17T22:41:39.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778960+0000", 2024-09-17T22:41:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1919999999999996e-06 2024-09-17T22:41:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802429+0000", 2024-09-17T22:41:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023469561999999999 2024-09-17T22:41:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802470+0000", 2024-09-17T22:41:39.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1068000000000001e-05 2024-09-17T22:41:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802523+0000", 2024-09-17T22:41:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2852999999999997e-05 2024-09-17T22:41:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38680 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778966+0000", 2024-09-17T22:41:39.470 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.614557864, 2024-09-17T22:41:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023670668999999998, 2024-09-17T22:41:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38680 2024-09-17T22:41:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778966+0000", 2024-09-17T22:41:39.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778966+0000", 2024-09-17T22:41:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778967+0000", 2024-09-17T22:41:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.89e-07 2024-09-17T22:41:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778969+0000", 2024-09-17T22:41:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0219999999999999e-06 2024-09-17T22:41:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778975+0000", 2024-09-17T22:41:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3779999999999998e-06 2024-09-17T22:41:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778980+0000", 2024-09-17T22:41:39.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0699999999999997e-06 2024-09-17T22:41:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802545+0000", 2024-09-17T22:41:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023565358000000002 2024-09-17T22:41:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802585+0000", 2024-09-17T22:41:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0136999999999999e-05 2024-09-17T22:41:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802637+0000", 2024-09-17T22:41:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1314999999999999e-05 2024-09-17T22:41:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38697 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 675840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.475 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.779074+0000", 2024-09-17T22:41:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.6144496109999995, 2024-09-17T22:41:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023676402999999999, 2024-09-17T22:41:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38697 2024-09-17T22:41:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779074+0000", 2024-09-17T22:41:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779074+0000", 2024-09-17T22:41:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779075+0000", 2024-09-17T22:41:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:41:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779077+0000", 2024-09-17T22:41:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8810000000000001e-06 2024-09-17T22:41:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779084+0000", 2024-09-17T22:41:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.691e-06 2024-09-17T22:41:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779088+0000", 2024-09-17T22:41:39.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6970000000000001e-06 2024-09-17T22:41:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802659+0000", 2024-09-17T22:41:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023571214 2024-09-17T22:41:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802699+0000", 2024-09-17T22:41:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9762000000000003e-05 2024-09-17T22:41:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802751+0000", 2024-09-17T22:41:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1572e-05 2024-09-17T22:41:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38698 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 749568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.779094+0000", 2024-09-17T22:41:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.6144296740000001, 2024-09-17T22:41:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023778845999999999, 2024-09-17T22:41:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.480 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38698 2024-09-17T22:41:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779094+0000", 2024-09-17T22:41:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779094+0000", 2024-09-17T22:41:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779095+0000", 2024-09-17T22:41:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8799999999999998e-07 2024-09-17T22:41:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779097+0000", 2024-09-17T22:41:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8530000000000001e-06 2024-09-17T22:41:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779103+0000", 2024-09-17T22:41:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0410000000000003e-06 2024-09-17T22:41:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779107+0000", 2024-09-17T22:41:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T22:41:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802774+0000", 2024-09-17T22:41:39.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023666348 2024-09-17T22:41:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802814+0000", 2024-09-17T22:41:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0173000000000003e-05 2024-09-17T22:41:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802873+0000", 2024-09-17T22:41:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9484999999999997e-05 2024-09-17T22:41:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38825 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.781521+0000", 2024-09-17T22:41:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.6120026070000009, 2024-09-17T22:41:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023214104999999999, 2024-09-17T22:41:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38825 2024-09-17T22:41:39.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781521+0000", 2024-09-17T22:41:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781521+0000", 2024-09-17T22:41:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781522+0000", 2024-09-17T22:41:39.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.53e-07 2024-09-17T22:41:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781524+0000", 2024-09-17T22:41:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0389999999999999e-06 2024-09-17T22:41:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781530+0000", 2024-09-17T22:41:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8209999999999998e-06 2024-09-17T22:41:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781534+0000", 2024-09-17T22:41:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6419999999999998e-06 2024-09-17T22:41:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804570+0000", 2024-09-17T22:41:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023035951999999998 2024-09-17T22:41:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804645+0000", 2024-09-17T22:41:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4696000000000004e-05 2024-09-17T22:41:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804736+0000", 2024-09-17T22:41:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0601999999999998e-05 2024-09-17T22:41:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38843 3.12 3.c4845672 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.6118803970000002, 2024-09-17T22:41:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023236265999999998, 2024-09-17T22:41:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.489 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38843 2024-09-17T22:41:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:41:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781646+0000", 2024-09-17T22:41:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9290000000000001e-06 2024-09-17T22:41:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781652+0000", 2024-09-17T22:41:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7389999999999996e-06 2024-09-17T22:41:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781656+0000", 2024-09-17T22:41:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3830000000000002e-06 2024-09-17T22:41:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804871+0000", 2024-09-17T22:41:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023215257999999999 2024-09-17T22:41:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804880+0000", 2024-09-17T22:41:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5769999999999994e-06 2024-09-17T22:41:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.493 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.493 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:39.495 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:41:39.495 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:39.492+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:39.595 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:39.593+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:39.639 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:39.639 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:39.639 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:41:39.639 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:39.648 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:41:39.696 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:39.694+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:39.751 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:39.751 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:39.751 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:39.751 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:39.751 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:39.760 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:41:39.803 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:39.801+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:39.865 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:39.865 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:39.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:39.865 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:39.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.865 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36437 3.b 3:d64eaf12:::rbd_data.115f86d8437d.000000000000001c:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.865 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:41:39.865 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.146066749999999, 2024-09-17T22:41:39.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068296390000000002, 2024-09-17T22:41:39.866 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.866 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.866 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:39.866 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.866 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.866 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.866 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36437 2024-09-17T22:41:39.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.866 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:41:39.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:41:39.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715035+0000", 2024-09-17T22:41:39.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.288e-06 2024-09-17T22:41:39.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715041+0000", 2024-09-17T22:41:39.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.643e-06 2024-09-17T22:41:39.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715051+0000", 2024-09-17T22:41:39.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0597e-05 2024-09-17T22:41:39.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715056+0000", 2024-09-17T22:41:39.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1379999999999998e-06 2024-09-17T22:41:39.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715101+0000", 2024-09-17T22:41:39.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4433e-05 2024-09-17T22:41:39.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715187+0000", 2024-09-17T22:41:39.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6148000000000003e-05 2024-09-17T22:41:39.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:39.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.716264+0000", 2024-09-17T22:41:39.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010770179999999999 2024-09-17T22:41:39.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:39.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721735+0000", 2024-09-17T22:41:39.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0054709019999999997 2024-09-17T22:41:39.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:41:39.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721754+0000", 2024-09-17T22:41:39.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9133000000000001e-05 2024-09-17T22:41:39.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:39.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721807+0000", 2024-09-17T22:41:39.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2831999999999998e-05 2024-09-17T22:41:39.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721863+0000", 2024-09-17T22:41:39.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6507e-05 2024-09-17T22:41:39.871 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.871 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.871 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.872 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36743 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 188416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.872 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735018+0000", 2024-09-17T22:41:39.872 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.126082634999999, 2024-09-17T22:41:39.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0060955260000000004, 2024-09-17T22:41:39.872 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.872 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.872 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.872 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.872 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.872 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.872 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36743 2024-09-17T22:41:39.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.873 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735018+0000", 2024-09-17T22:41:39.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735018+0000", 2024-09-17T22:41:39.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735018+0000", 2024-09-17T22:41:39.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7199999999999999e-07 2024-09-17T22:41:39.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735020+0000", 2024-09-17T22:41:39.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9889999999999999e-06 2024-09-17T22:41:39.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735027+0000", 2024-09-17T22:41:39.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3409999999999999e-06 2024-09-17T22:41:39.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735032+0000", 2024-09-17T22:41:39.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.904e-06 2024-09-17T22:41:39.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.740910+0000", 2024-09-17T22:41:39.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.005878266 2024-09-17T22:41:39.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741009+0000", 2024-09-17T22:41:39.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8754999999999999e-05 2024-09-17T22:41:39.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741113+0000", 2024-09-17T22:41:39.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000104799 2024-09-17T22:41:39.876 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.876 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.876 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.877 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36744 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.877 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:41:39.877 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.126062608, 2024-09-17T22:41:39.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063082559999999999, 2024-09-17T22:41:39.877 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.877 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.877 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.877 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.877 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.877 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.877 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36744 2024-09-17T22:41:39.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.878 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:41:39.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:41:39.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:41:39.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:41:39.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735040+0000", 2024-09-17T22:41:39.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9630000000000001e-06 2024-09-17T22:41:39.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735047+0000", 2024-09-17T22:41:39.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3439999999999997e-06 2024-09-17T22:41:39.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735052+0000", 2024-09-17T22:41:39.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9250000000000003e-06 2024-09-17T22:41:39.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741147+0000", 2024-09-17T22:41:39.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0060954210000000002 2024-09-17T22:41:39.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741243+0000", 2024-09-17T22:41:39.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5674999999999995e-05 2024-09-17T22:41:39.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741346+0000", 2024-09-17T22:41:39.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103522 2024-09-17T22:41:39.881 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.881 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.881 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.882 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36745 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.882 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:41:39.882 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.126042624, 2024-09-17T22:41:39.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065186740000000003, 2024-09-17T22:41:39.882 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.882 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.882 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.882 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.882 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.882 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.882 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36745 2024-09-17T22:41:39.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.882 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:41:39.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:41:39.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:41:39.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2500000000000001e-07 2024-09-17T22:41:39.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735061+0000", 2024-09-17T22:41:39.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.824e-06 2024-09-17T22:41:39.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735068+0000", 2024-09-17T22:41:39.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8580000000000002e-06 2024-09-17T22:41:39.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735074+0000", 2024-09-17T22:41:39.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5829999999999997e-06 2024-09-17T22:41:39.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741379+0000", 2024-09-17T22:41:39.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063051909999999999 2024-09-17T22:41:39.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741472+0000", 2024-09-17T22:41:39.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3238000000000004e-05 2024-09-17T22:41:39.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741577+0000", 2024-09-17T22:41:39.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000104555 2024-09-17T22:41:39.886 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.886 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.887 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.887 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36746 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.887 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:41:39.887 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.126019658000001, 2024-09-17T22:41:39.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067275240000000004, 2024-09-17T22:41:39.887 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.887 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.887 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.887 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.887 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.887 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.888 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36746 2024-09-17T22:41:39.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.888 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:41:39.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:41:39.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:41:39.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.03e-07 2024-09-17T22:41:39.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735083+0000", 2024-09-17T22:41:39.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T22:41:39.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735090+0000", 2024-09-17T22:41:39.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7800000000000003e-06 2024-09-17T22:41:39.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735095+0000", 2024-09-17T22:41:39.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8500000000000002e-06 2024-09-17T22:41:39.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741608+0000", 2024-09-17T22:41:39.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065131809999999998 2024-09-17T22:41:39.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741703+0000", 2024-09-17T22:41:39.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4914999999999995e-05 2024-09-17T22:41:39.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741808+0000", 2024-09-17T22:41:39.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000105448 2024-09-17T22:41:39.891 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.891 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.891 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.891 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36747 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.891 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:41:39.891 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.125999105, 2024-09-17T22:41:39.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069468790000000004, 2024-09-17T22:41:39.891 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.891 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.892 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.892 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36747 2024-09-17T22:41:39.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.892 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:41:39.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:41:39.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735102+0000", 2024-09-17T22:41:39.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:41:39.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735104+0000", 2024-09-17T22:41:39.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7400000000000001e-06 2024-09-17T22:41:39.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735111+0000", 2024-09-17T22:41:39.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4189999999999996e-06 2024-09-17T22:41:39.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735116+0000", 2024-09-17T22:41:39.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0860000000000001e-06 2024-09-17T22:41:39.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741847+0000", 2024-09-17T22:41:39.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067306859999999996 2024-09-17T22:41:39.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741942+0000", 2024-09-17T22:41:39.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5180999999999996e-05 2024-09-17T22:41:39.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742048+0000", 2024-09-17T22:41:39.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106387 2024-09-17T22:41:39.895 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.895 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.895 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.896 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36748 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.896 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:41:39.896 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.125977784, 2024-09-17T22:41:39.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071864119999999997, 2024-09-17T22:41:39.896 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.896 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.896 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.896 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36748 2024-09-17T22:41:39.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.896 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:41:39.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:41:39.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:41:39.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.34e-07 2024-09-17T22:41:39.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735125+0000", 2024-09-17T22:41:39.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0020000000000001e-06 2024-09-17T22:41:39.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735132+0000", 2024-09-17T22:41:39.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3489999999999997e-06 2024-09-17T22:41:39.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735138+0000", 2024-09-17T22:41:39.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9709999999999998e-06 2024-09-17T22:41:39.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742087+0000", 2024-09-17T22:41:39.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069480669999999996 2024-09-17T22:41:39.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742188+0000", 2024-09-17T22:41:39.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101279 2024-09-17T22:41:39.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742309+0000", 2024-09-17T22:41:39.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012131 2024-09-17T22:41:39.900 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.900 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.900 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.900 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36749 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.900 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:41:39.900 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.125955097, 2024-09-17T22:41:39.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0074112570000000001, 2024-09-17T22:41:39.900 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.900 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.900 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.900 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.901 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36749 2024-09-17T22:41:39.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.901 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:41:39.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:41:39.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735146+0000", 2024-09-17T22:41:39.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9200000000000002e-07 2024-09-17T22:41:39.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735158+0000", 2024-09-17T22:41:39.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2508e-05 2024-09-17T22:41:39.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735166+0000", 2024-09-17T22:41:39.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6290000000000001e-06 2024-09-17T22:41:39.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735174+0000", 2024-09-17T22:41:39.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1000000000000004e-06 2024-09-17T22:41:39.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742354+0000", 2024-09-17T22:41:39.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071795660000000001 2024-09-17T22:41:39.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742454+0000", 2024-09-17T22:41:39.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100408 2024-09-17T22:41:39.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742557+0000", 2024-09-17T22:41:39.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102654 2024-09-17T22:41:39.904 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.904 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.904 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.905 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36750 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.905 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:41:39.905 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.125917971, 2024-09-17T22:41:39.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0077413890000000004, 2024-09-17T22:41:39.905 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.905 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.905 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.905 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36750 2024-09-17T22:41:39.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.905 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:41:39.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:41:39.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:41:39.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2399999999999998e-07 2024-09-17T22:41:39.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735185+0000", 2024-09-17T22:41:39.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1710000000000001e-06 2024-09-17T22:41:39.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735192+0000", 2024-09-17T22:41:39.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.81e-06 2024-09-17T22:41:39.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735197+0000", 2024-09-17T22:41:39.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9080000000000003e-06 2024-09-17T22:41:39.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742587+0000", 2024-09-17T22:41:39.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.007390359 2024-09-17T22:41:39.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742694+0000", 2024-09-17T22:41:39.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106901 2024-09-17T22:41:39.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742924+0000", 2024-09-17T22:41:39.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022961600000000001 2024-09-17T22:41:39.908 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.909 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.909 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.909 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36751 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.909 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:41:39.909 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.125895997000001, 2024-09-17T22:41:39.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079273859999999998, 2024-09-17T22:41:39.909 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.909 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.909 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.910 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36751 2024-09-17T22:41:39.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.910 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:41:39.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:41:39.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:41:39.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1600000000000001e-07 2024-09-17T22:41:39.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735207+0000", 2024-09-17T22:41:39.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1909999999999999e-06 2024-09-17T22:41:39.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735214+0000", 2024-09-17T22:41:39.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9269999999999999e-06 2024-09-17T22:41:39.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735220+0000", 2024-09-17T22:41:39.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3739999999999997e-06 2024-09-17T22:41:39.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742957+0000", 2024-09-17T22:41:39.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0077371699999999998 2024-09-17T22:41:39.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.743037+0000", 2024-09-17T22:41:39.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0370000000000005e-05 2024-09-17T22:41:39.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.743132+0000", 2024-09-17T22:41:39.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4838000000000002e-05 2024-09-17T22:41:39.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.913 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.913 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37658 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.913 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749751+0000", 2024-09-17T22:41:39.913 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.111349917, 2024-09-17T22:41:39.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061048459999999997, 2024-09-17T22:41:39.913 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.913 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.914 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.914 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37658 2024-09-17T22:41:39.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.914 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749751+0000", 2024-09-17T22:41:39.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749751+0000", 2024-09-17T22:41:39.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749751+0000", 2024-09-17T22:41:39.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8500000000000002e-07 2024-09-17T22:41:39.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749753+0000", 2024-09-17T22:41:39.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2579999999999998e-06 2024-09-17T22:41:39.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749760+0000", 2024-09-17T22:41:39.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9099999999999999e-06 2024-09-17T22:41:39.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749765+0000", 2024-09-17T22:41:39.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1170000000000003e-06 2024-09-17T22:41:39.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755677+0000", 2024-09-17T22:41:39.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059120140000000002 2024-09-17T22:41:39.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755756+0000", 2024-09-17T22:41:39.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9013999999999999e-05 2024-09-17T22:41:39.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755855+0000", 2024-09-17T22:41:39.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9048000000000004e-05 2024-09-17T22:41:39.917 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.917 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.917 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.918 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37659 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.918 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:41:39.918 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.111328353999999, 2024-09-17T22:41:39.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063144200000000003, 2024-09-17T22:41:39.918 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.918 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.918 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.918 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37659 2024-09-17T22:41:39.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.918 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:41:39.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:41:39.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749773+0000", 2024-09-17T22:41:39.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:41:39.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749774+0000", 2024-09-17T22:41:39.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:41:39.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749781+0000", 2024-09-17T22:41:39.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5740000000000001e-06 2024-09-17T22:41:39.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749786+0000", 2024-09-17T22:41:39.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9099999999999996e-06 2024-09-17T22:41:39.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755889+0000", 2024-09-17T22:41:39.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061034260000000003 2024-09-17T22:41:39.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755972+0000", 2024-09-17T22:41:39.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2795999999999998e-05 2024-09-17T22:41:39.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756087+0000", 2024-09-17T22:41:39.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000114418 2024-09-17T22:41:39.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.922 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.922 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37660 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.922 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:41:39.922 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.111298516, 2024-09-17T22:41:39.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065587800000000002, 2024-09-17T22:41:39.922 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.922 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.922 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.922 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.923 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37660 2024-09-17T22:41:39.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.923 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:41:39.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:41:39.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:41:39.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:41:39.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749805+0000", 2024-09-17T22:41:39.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-17T22:41:39.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749812+0000", 2024-09-17T22:41:39.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270000000000002e-06 2024-09-17T22:41:39.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749819+0000", 2024-09-17T22:41:39.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.759e-06 2024-09-17T22:41:39.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756153+0000", 2024-09-17T22:41:39.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063345809999999997 2024-09-17T22:41:39.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756257+0000", 2024-09-17T22:41:39.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103583 2024-09-17T22:41:39.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756361+0000", 2024-09-17T22:41:39.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103853 2024-09-17T22:41:39.926 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.926 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.926 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.926 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37661 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.926 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:41:39.926 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.111269922, 2024-09-17T22:41:39.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067442300000000004, 2024-09-17T22:41:39.927 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.927 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.927 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.927 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.927 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.927 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.927 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37661 2024-09-17T22:41:39.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.927 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:41:39.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:41:39.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:41:39.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9399999999999995e-07 2024-09-17T22:41:39.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749835+0000", 2024-09-17T22:41:39.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1049999999999997e-06 2024-09-17T22:41:39.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749855+0000", 2024-09-17T22:41:39.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9343000000000001e-05 2024-09-17T22:41:39.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749860+0000", 2024-09-17T22:41:39.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:41:39.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756395+0000", 2024-09-17T22:41:39.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065349900000000001 2024-09-17T22:41:39.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756479+0000", 2024-09-17T22:41:39.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4030999999999996e-05 2024-09-17T22:41:39.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756575+0000", 2024-09-17T22:41:39.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5835999999999995e-05 2024-09-17T22:41:39.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.930 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.931 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37662 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.931 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:41:39.931 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.111216116, 2024-09-17T22:41:39.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006927295, 2024-09-17T22:41:39.931 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.931 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.931 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.932 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37662 2024-09-17T22:41:39.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.932 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:41:39.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:41:39.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749885+0000", 2024-09-17T22:41:39.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2699999999999999e-07 2024-09-17T22:41:39.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749888+0000", 2024-09-17T22:41:39.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6599999999999999e-06 2024-09-17T22:41:39.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749895+0000", 2024-09-17T22:41:39.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4200000000000001e-06 2024-09-17T22:41:39.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749901+0000", 2024-09-17T22:41:39.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5090000000000001e-06 2024-09-17T22:41:39.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756605+0000", 2024-09-17T22:41:39.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067040629999999997 2024-09-17T22:41:39.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756691+0000", 2024-09-17T22:41:39.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5846000000000004e-05 2024-09-17T22:41:39.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756812+0000", 2024-09-17T22:41:39.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012116999999999999 2024-09-17T22:41:39.935 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.935 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.935 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.936 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37663 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.936 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:41:39.936 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.111191906, 2024-09-17T22:41:39.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071811440000000004, 2024-09-17T22:41:39.936 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.936 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.936 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.936 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37663 2024-09-17T22:41:39.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.936 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:41:39.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:41:39.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:41:39.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:41:39.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749911+0000", 2024-09-17T22:41:39.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.035e-06 2024-09-17T22:41:39.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749917+0000", 2024-09-17T22:41:39.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4370000000000004e-06 2024-09-17T22:41:39.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749922+0000", 2024-09-17T22:41:39.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6249999999999998e-06 2024-09-17T22:41:39.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756845+0000", 2024-09-17T22:41:39.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069227280000000004 2024-09-17T22:41:39.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756994+0000", 2024-09-17T22:41:39.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000149068 2024-09-17T22:41:39.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757090+0000", 2024-09-17T22:41:39.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5851000000000001e-05 2024-09-17T22:41:39.940 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.940 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.940 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.940 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37664 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.940 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:41:39.940 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.111171061, 2024-09-17T22:41:39.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.007384743, 2024-09-17T22:41:39.940 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.940 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.940 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.941 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.941 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.941 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.941 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37664 2024-09-17T22:41:39.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.941 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:41:39.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:41:39.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749930+0000", 2024-09-17T22:41:39.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:41:39.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749933+0000", 2024-09-17T22:41:39.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8430000000000001e-06 2024-09-17T22:41:39.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749939+0000", 2024-09-17T22:41:39.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:41:39.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749944+0000", 2024-09-17T22:41:39.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9230000000000001e-06 2024-09-17T22:41:39.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757124+0000", 2024-09-17T22:41:39.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071800020000000004 2024-09-17T22:41:39.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757212+0000", 2024-09-17T22:41:39.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7383e-05 2024-09-17T22:41:39.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757314+0000", 2024-09-17T22:41:39.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010247399999999999 2024-09-17T22:41:39.944 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.944 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.944 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.945 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:39363 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:39.945 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:41:39.945 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.069182722000001, 2024-09-17T22:41:39.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006226245, 2024-09-17T22:41:39.945 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.945 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.945 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:39.945 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:39.945 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:39.945 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:39.945 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 39363 2024-09-17T22:41:39.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.945 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:41:39.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:41:39.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:41:39.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3500000000000002e-07 2024-09-17T22:41:39.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791920+0000", 2024-09-17T22:41:39.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.869e-06 2024-09-17T22:41:39.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791926+0000", 2024-09-17T22:41:39.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8490000000000002e-06 2024-09-17T22:41:39.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791931+0000", 2024-09-17T22:41:39.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4269999999999999e-06 2024-09-17T22:41:39.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.797912+0000", 2024-09-17T22:41:39.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059808819999999999 2024-09-17T22:41:39.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.798020+0000", 2024-09-17T22:41:39.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010783800000000001 2024-09-17T22:41:39.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.798144+0000", 2024-09-17T22:41:39.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012404500000000001 2024-09-17T22:41:39.949 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.949 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.949 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.949 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4447.0:51869 3.6 e35/23 3:6037e35f:::rbd_object_map.115fe9ac8535:head v 35'71, mlcod=35'66)", 2024-09-17T22:41:39.949 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:41:39.949 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.0838604219999999, 2024-09-17T22:41:39.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0066208669999999999, 2024-09-17T22:41:39.949 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.949 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.949 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:39.949 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:41:39.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:41:39.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777241+0000", 2024-09-17T22:41:39.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4000000000000001e-07 2024-09-17T22:41:39.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777246+0000", 2024-09-17T22:41:39.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2789999999999998e-06 2024-09-17T22:41:39.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777255+0000", 2024-09-17T22:41:39.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6209999999999993e-06 2024-09-17T22:41:39.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777259+0000", 2024-09-17T22:41:39.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7650000000000001e-06 2024-09-17T22:41:39.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777298+0000", 2024-09-17T22:41:39.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8797999999999997e-05 2024-09-17T22:41:39.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777311+0000", 2024-09-17T22:41:39.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2364999999999999e-05 2024-09-17T22:41:39.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:39.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.783814+0000", 2024-09-17T22:41:39.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065039169999999997 2024-09-17T22:41:39.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.783861+0000", 2024-09-17T22:41:39.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6482e-05 2024-09-17T22:41:39.953 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.953 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.953 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.953 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4447.0:51870 3.6 e35/23 3:6037e35f:::rbd_object_map.115fe9ac8535:head v 35'72, mlcod=35'66)", 2024-09-17T22:41:39.953 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:33.777849+0000", 2024-09-17T22:41:39.953 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 6.083251443, 2024-09-17T22:41:39.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0060422890000000002, 2024-09-17T22:41:39.954 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:39.954 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:39.954 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:39.954 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:39.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:39.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777849+0000", 2024-09-17T22:41:39.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:39.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777849+0000", 2024-09-17T22:41:39.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:39.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:39.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777850+0000", 2024-09-17T22:41:39.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2e-07 2024-09-17T22:41:39.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:39.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777854+0000", 2024-09-17T22:41:39.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0899999999999998e-06 2024-09-17T22:41:39.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:39.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777862+0000", 2024-09-17T22:41:39.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1310000000000006e-06 2024-09-17T22:41:39.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:39.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777866+0000", 2024-09-17T22:41:39.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1019999999999999e-06 2024-09-17T22:41:39.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:39.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777904+0000", 2024-09-17T22:41:39.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8124999999999998e-05 2024-09-17T22:41:39.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:39.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777917+0000", 2024-09-17T22:41:39.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2561000000000001e-05 2024-09-17T22:41:39.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:39.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.783864+0000", 2024-09-17T22:41:39.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059470080000000002 2024-09-17T22:41:39.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:39.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:39.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:39.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.783891+0000", 2024-09-17T22:41:39.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7752000000000001e-05 2024-09-17T22:41:39.957 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.957 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:39.958 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:39.958 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:39.959 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:41:39.960 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:39.956+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:40.029 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is unset 2024-09-17T22:41:40.029 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T22:41:40.049 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-17T22:41:40.060 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:40.057+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:40.091 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:40.091 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:40.091 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:41:40.091 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:40.100 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:41:40.122 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone10 from 0xab4a233 (overlap 0x1834074) to 0xc1a03a 2024-09-17T22:41:40.165 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:40.163+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:40.258 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:41:40.262 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:40.259+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:40.297 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:41:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:41:40.298 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:40.309 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:41:40.362 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:40.360+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:40.433 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0: {} 2024-09-17T22:41:40.434 INFO:teuthology.orchestra.run.smithi115.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:41:40.455 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:41:40.456 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:41:40.459 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-17T22:41:40.463 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:40.460+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:40.473 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:41:40.474 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:41:40.481 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:40.481 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:40.492 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:41:40.493 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:41:40.493 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:41:40.532 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:41:40.532 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:41:40.563 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:40.561+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:40.568 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:41:40.568 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:41:40.604 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:41:40.605 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:41:40.641 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:41:40.641 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:41:40.664 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:40.662+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:40.692 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:40.692 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:40.704 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:41:40.788 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:40.785+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:40.866 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:40.863+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:40.907 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:40.908 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:40.921 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:41:40.967 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:40.965+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:41.034 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T22:41:41.057 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-17T22:41:41.101 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:41.099+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:41.133 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:41.133 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:41.147 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:41:41.169 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:41.167+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:41.271 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:41.268+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:41.357 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:41.357 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:41.368 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:41:41.409 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:41.406+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:41.472 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:41.470+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:41.570 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:41.571 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:41.573 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:41.570+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:41.585 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:41:41.674 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:41.671+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:41.706 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:41.707 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:41.716 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:41:41.775 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:41.772+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:41.792 INFO:teuthology.orchestra.run.smithi115.stdout:754 trunc from 0xab4a233 to 0x8a61c48 2024-09-17T22:41:41.846 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:41.846 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:41.859 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:41:41.900 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:41.897+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:41.977 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:41.975+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:42.045 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T22:41:42.053 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:41:42.068 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:42.069 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:42.073 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T22:41:42.080 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:41:42.080 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:42.076+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:42.080 INFO:teuthology.orchestra.run.smithi115.stdout:755 write 0x43329e2 thru 0x433750c (0x4b2b bytes) 2024-09-17T22:41:42.090 INFO:teuthology.orchestra.run.smithi115.stdout:756 write 0xc0dcdad thru 0xc0ddc71 (0xec5 bytes) 2024-09-17T22:41:42.092 INFO:teuthology.orchestra.run.smithi115.stdout:757 write 0x36ec7b4 thru 0x36f07be (0x400b bytes) 2024-09-17T22:41:42.094 INFO:teuthology.orchestra.run.smithi115.stdout:758 punch from 0x1dfec6a to 0x1dff018, (0x3ae bytes) 2024-09-17T22:41:42.094 INFO:teuthology.orchestra.run.smithi115.stdout:759 read 0x1e5b6ec thru 0x1e60a61 (0x5376 bytes) 2024-09-17T22:41:42.094 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:42.095 INFO:teuthology.orchestra.run.smithi115.stdout:761 flatten 2024-09-17T22:41:42.179 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:42.176+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:42.260 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:42.260 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:42.273 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:41:42.302 INFO:teuthology.orchestra.run.smithi115.stdout:762 read 0x1a31f7f thru 0x1a33c26 (0x1ca8 bytes) 2024-09-17T22:41:42.302 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:42.299+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:42.303 INFO:teuthology.orchestra.run.smithi115.stdout:763 punch from 0x92f65cb to 0x92fbdee, (0x5823 bytes) 2024-09-17T22:41:42.303 INFO:teuthology.orchestra.run.smithi115.stdout:764 punch from 0x36ad8fb to 0x36bc57f, (0xec84 bytes) 2024-09-17T22:41:42.303 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:42.303 INFO:teuthology.orchestra.run.smithi115.stdout:767 write 0xe24cef thru 0xe27039 (0x234b bytes) 2024-09-17T22:41:42.304 INFO:teuthology.orchestra.run.smithi115.stdout:768 read 0x44cd99b thru 0x44cef3d (0x15a3 bytes) 2024-09-17T22:41:42.304 INFO:teuthology.orchestra.run.smithi115.stdout:769 read 0x189ca0f thru 0x18ac3e2 (0xf9d4 bytes) 2024-09-17T22:41:42.305 INFO:teuthology.orchestra.run.smithi115.stdout:770 write 0xb36df67 thru 0xb3758c3 (0x795d bytes) 2024-09-17T22:41:42.306 INFO:teuthology.orchestra.run.smithi115.stdout:774 punch from 0xa3d1561 to 0xa3df1e0, (0xdc7f bytes) 2024-09-17T22:41:42.306 INFO:teuthology.orchestra.run.smithi115.stdout:775 writesame 0x54347d5 thru 0x544335c data_size 0x4(0xeb88 bytes) 2024-09-17T22:41:42.313 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:42.313 INFO:teuthology.orchestra.run.smithi115.stdout:778 write 0x1596dc8 thru 0x15a1822 (0xaa5b bytes) 2024-09-17T22:41:42.315 INFO:teuthology.orchestra.run.smithi115.stdout:779 write 0x1f4b1d0 thru 0x1f4f507 (0x4338 bytes) 2024-09-17T22:41:42.318 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:42.318 INFO:teuthology.orchestra.run.smithi115.stdout:781 write 0x520c43d thru 0x521480f (0x83d3 bytes) 2024-09-17T22:41:42.320 INFO:teuthology.orchestra.run.smithi115.stdout:783 write 0x6167681 thru 0x6174e21 (0xd7a1 bytes) 2024-09-17T22:41:42.322 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:42.323 INFO:teuthology.orchestra.run.smithi115.stdout:785 trunc from 0xc0ddc72 to 0xd064dfb 2024-09-17T22:41:42.329 INFO:teuthology.orchestra.run.smithi115.stdout:787 punch from 0x970a125 to 0x970b573, (0x144e bytes) 2024-09-17T22:41:42.329 INFO:teuthology.orchestra.run.smithi115.stdout:788 read 0xc700da0 thru 0xc70b4de (0xa73f bytes) 2024-09-17T22:41:42.329 INFO:teuthology.orchestra.run.smithi115.stdout:789 read 0x3cc2865 thru 0x3ccbb82 (0x931e bytes) 2024-09-17T22:41:42.330 INFO:teuthology.orchestra.run.smithi115.stdout:790 read 0x5d9e6cb thru 0x5dab3dd (0xcd13 bytes) 2024-09-17T22:41:42.330 INFO:teuthology.orchestra.run.smithi115.stdout:791 read 0x1ccb886 thru 0x1ccb90c (0x87 bytes) 2024-09-17T22:41:42.337 INFO:teuthology.orchestra.run.smithi115.stdout:792 write 0xed3a067 thru 0xed457e6 (0xb780 bytes) 2024-09-17T22:41:42.341 INFO:teuthology.orchestra.run.smithi115.stdout:793 write 0xaa10cc0 thru 0xaa14084 (0x33c5 bytes) 2024-09-17T22:41:42.342 INFO:teuthology.orchestra.run.smithi115.stdout:794 read 0xebbdfeb thru 0xebcba7d (0xda93 bytes) 2024-09-17T22:41:42.342 INFO:teuthology.orchestra.run.smithi115.stdout:797 punch from 0x360a238 to 0x360ffec, (0x5db4 bytes) 2024-09-17T22:41:42.348 INFO:teuthology.orchestra.run.smithi115.stdout:798 trunc from 0xed457e7 to 0x9690ecf 2024-09-17T22:41:42.363 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:42.363 INFO:teuthology.orchestra.run.smithi115.stdout:800 read 0x82eed06 thru 0x82fe22c (0xf527 bytes) 2024-09-17T22:41:42.363 INFO:teuthology.orchestra.run.smithi115.stdout:802 read 0xd8f8b thru 0xe7c0f (0xec85 bytes) 2024-09-17T22:41:42.364 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:42.364 INFO:teuthology.orchestra.run.smithi115.stdout:804 write 0x2728684 thru 0x27353fc (0xcd79 bytes) 2024-09-17T22:41:42.366 INFO:teuthology.orchestra.run.smithi115.stdout:805 writesame 0x74716fb thru 0x747fa53 data_size 0x79(0xe359 bytes) 2024-09-17T22:41:42.374 INFO:teuthology.orchestra.run.smithi115.stdout:806 writesame 0xb2d760d thru 0xb2de0d9 data_size 0x1(0x6acd bytes) 2024-09-17T22:41:42.376 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:42.376 INFO:teuthology.orchestra.run.smithi115.stdout:808 clone 12 order 19 su 2048 sc 3 2024-09-17T22:41:42.380 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:42.377+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:42.383 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:42.383 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:42.393 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:41:42.481 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:42.478+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:42.495 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:42.495 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:42.504 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:41:42.581 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:42.579+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:42.607 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:42.608 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:42.616 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:41:42.660 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:41:42.705 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:42.702+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:42.760 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:42.760 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:42.769 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:41:42.779 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0: {} 2024-09-17T22:41:42.779 INFO:teuthology.orchestra.run.smithi115.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:41:42.796 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:41:42.796 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:41:42.812 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:41:42.812 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:41:42.812 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:42.810+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:42.827 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:41:42.828 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:41:42.844 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:41:42.844 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:41:42.860 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:41:42.860 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:41:42.873 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:42.874 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:42.874 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:42.874 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:42.874 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:42.877 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:41:42.877 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:41:42.883 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:41:42.883 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:42.880+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:42.893 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:41:42.893 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:41:42.984 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:42.981+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:43.027 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:43.027 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:43.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:43.027 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:43.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.027 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72639 3.0 3:04644e28:::rbd_data.115fe9ac8535.0000000000000005:9 [read 4050944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.027 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.725903+0000", 2024-09-17T22:41:43.027 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2979040409999998, 2024-09-17T22:41:43.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023334941000000001, 2024-09-17T22:41:43.028 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.028 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.028 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.028 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72639 2024-09-17T22:41:43.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.028 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725903+0000", 2024-09-17T22:41:43.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725903+0000", 2024-09-17T22:41:43.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725904+0000", 2024-09-17T22:41:43.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0499999999999999e-07 2024-09-17T22:41:43.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725906+0000", 2024-09-17T22:41:43.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1430000000000001e-06 2024-09-17T22:41:43.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725912+0000", 2024-09-17T22:41:43.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8860000000000001e-06 2024-09-17T22:41:43.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725916+0000", 2024-09-17T22:41:43.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4229999999999998e-06 2024-09-17T22:41:43.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749094+0000", 2024-09-17T22:41:43.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023177729000000001 2024-09-17T22:41:43.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749167+0000", 2024-09-17T22:41:43.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2665999999999998e-05 2024-09-17T22:41:43.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749238+0000", 2024-09-17T22:41:43.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1688999999999998e-05 2024-09-17T22:41:43.032 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.032 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.032 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.032 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72640 3.18 3:1997a52f:::rbd_data.115fe9ac8535.0000000000000006:9 [read 4034560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.032 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.725936+0000", 2024-09-17T22:41:43.032 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2978713129999999, 2024-09-17T22:41:43.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023281237999999999, 2024-09-17T22:41:43.032 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.032 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.032 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.032 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.033 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72640 2024-09-17T22:41:43.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.033 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725936+0000", 2024-09-17T22:41:43.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725936+0000", 2024-09-17T22:41:43.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725937+0000", 2024-09-17T22:41:43.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1500000000000005e-07 2024-09-17T22:41:43.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725939+0000", 2024-09-17T22:41:43.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1110000000000002e-06 2024-09-17T22:41:43.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725945+0000", 2024-09-17T22:41:43.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4810000000000003e-06 2024-09-17T22:41:43.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725950+0000", 2024-09-17T22:41:43.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T22:41:43.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749083+0000", 2024-09-17T22:41:43.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023133236000000001 2024-09-17T22:41:43.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749153+0000", 2024-09-17T22:41:43.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0253000000000007e-05 2024-09-17T22:41:43.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749217+0000", 2024-09-17T22:41:43.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4084000000000005e-05 2024-09-17T22:41:43.036 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.036 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.036 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.037 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72641 3.0 3:0299d250:::rbd_data.115fe9ac8535.0000000000000000:9 [read 3502080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.037 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.725958+0000", 2024-09-17T22:41:43.037 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2978497869999996, 2024-09-17T22:41:43.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023452997999999999, 2024-09-17T22:41:43.037 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.037 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.037 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.037 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.037 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.037 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.037 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72641 2024-09-17T22:41:43.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.038 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725958+0000", 2024-09-17T22:41:43.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725958+0000", 2024-09-17T22:41:43.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725958+0000", 2024-09-17T22:41:43.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7300000000000001e-07 2024-09-17T22:41:43.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725960+0000", 2024-09-17T22:41:43.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6050000000000001e-06 2024-09-17T22:41:43.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725967+0000", 2024-09-17T22:41:43.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6720000000000004e-06 2024-09-17T22:41:43.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725972+0000", 2024-09-17T22:41:43.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3819999999999997e-06 2024-09-17T22:41:43.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749276+0000", 2024-09-17T22:41:43.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023304248 2024-09-17T22:41:43.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749344+0000", 2024-09-17T22:41:43.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7564999999999994e-05 2024-09-17T22:41:43.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749411+0000", 2024-09-17T22:41:43.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7052999999999997e-05 2024-09-17T22:41:43.041 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.041 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.041 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.041 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72643 3.18 3:1997a52f:::rbd_data.115fe9ac8535.0000000000000006:9 [read 3256320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.041 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.725978+0000", 2024-09-17T22:41:43.041 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2978293880000003, 2024-09-17T22:41:43.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023430645999999999, 2024-09-17T22:41:43.042 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.042 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.042 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.042 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.042 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.042 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.042 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72643 2024-09-17T22:41:43.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.042 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725978+0000", 2024-09-17T22:41:43.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725978+0000", 2024-09-17T22:41:43.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725979+0000", 2024-09-17T22:41:43.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:41:43.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725981+0000", 2024-09-17T22:41:43.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.294e-06 2024-09-17T22:41:43.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725989+0000", 2024-09-17T22:41:43.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7319999999999998e-06 2024-09-17T22:41:43.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725993+0000", 2024-09-17T22:41:43.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6750000000000001e-06 2024-09-17T22:41:43.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749267+0000", 2024-09-17T22:41:43.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023273540999999998 2024-09-17T22:41:43.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749340+0000", 2024-09-17T22:41:43.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2799999999999994e-05 2024-09-17T22:41:43.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749409+0000", 2024-09-17T22:41:43.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9194000000000002e-05 2024-09-17T22:41:43.045 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.045 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.045 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.046 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72647 3.0 3:04644e28:::rbd_data.115fe9ac8535.0000000000000005:9 [read 3481600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.046 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726034+0000", 2024-09-17T22:41:43.046 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2977734229999998, 2024-09-17T22:41:43.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023543489000000001, 2024-09-17T22:41:43.046 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.046 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.046 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.046 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.046 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.046 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.046 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72647 2024-09-17T22:41:43.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.046 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726034+0000", 2024-09-17T22:41:43.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726034+0000", 2024-09-17T22:41:43.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726034+0000", 2024-09-17T22:41:43.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:41:43.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726037+0000", 2024-09-17T22:41:43.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.092e-06 2024-09-17T22:41:43.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726043+0000", 2024-09-17T22:41:43.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4969999999999998e-06 2024-09-17T22:41:43.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726048+0000", 2024-09-17T22:41:43.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7129999999999996e-06 2024-09-17T22:41:43.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749436+0000", 2024-09-17T22:41:43.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023388234000000001 2024-09-17T22:41:43.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749510+0000", 2024-09-17T22:41:43.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3930000000000005e-05 2024-09-17T22:41:43.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749578+0000", 2024-09-17T22:41:43.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7606000000000003e-05 2024-09-17T22:41:43.049 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.050 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.050 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.050 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72648 3.18 3:1997a52f:::rbd_data.115fe9ac8535.0000000000000006:9 [read 3526656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.050 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726055+0000", 2024-09-17T22:41:43.050 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2977526040000003, 2024-09-17T22:41:43.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023537293000000001, 2024-09-17T22:41:43.050 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.050 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.050 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.050 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.050 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.050 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.050 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72648 2024-09-17T22:41:43.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.051 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726055+0000", 2024-09-17T22:41:43.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726055+0000", 2024-09-17T22:41:43.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726055+0000", 2024-09-17T22:41:43.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:41:43.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726057+0000", 2024-09-17T22:41:43.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0480000000000001e-06 2024-09-17T22:41:43.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726064+0000", 2024-09-17T22:41:43.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1609999999999997e-06 2024-09-17T22:41:43.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726069+0000", 2024-09-17T22:41:43.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8879999999999996e-06 2024-09-17T22:41:43.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749451+0000", 2024-09-17T22:41:43.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023381246000000001 2024-09-17T22:41:43.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749527+0000", 2024-09-17T22:41:43.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6094999999999996e-05 2024-09-17T22:41:43.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749592+0000", 2024-09-17T22:41:43.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5475000000000006e-05 2024-09-17T22:41:43.054 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.054 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.054 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.054 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72649 3.0 3:0299d250:::rbd_data.115fe9ac8535.0000000000000000:9 [read 3772416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.054 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726098+0000", 2024-09-17T22:41:43.054 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2977097190000002, 2024-09-17T22:41:43.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023657582, 2024-09-17T22:41:43.054 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.054 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.055 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.055 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72649 2024-09-17T22:41:43.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.055 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726098+0000", 2024-09-17T22:41:43.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726098+0000", 2024-09-17T22:41:43.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726098+0000", 2024-09-17T22:41:43.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0900000000000001e-07 2024-09-17T22:41:43.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726103+0000", 2024-09-17T22:41:43.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0600000000000001e-06 2024-09-17T22:41:43.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726111+0000", 2024-09-17T22:41:43.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1810000000000001e-06 2024-09-17T22:41:43.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726117+0000", 2024-09-17T22:41:43.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6030000000000001e-06 2024-09-17T22:41:43.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749603+0000", 2024-09-17T22:41:43.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023485552999999999 2024-09-17T22:41:43.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749678+0000", 2024-09-17T22:41:43.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5446999999999998e-05 2024-09-17T22:41:43.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749755+0000", 2024-09-17T22:41:43.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7028999999999997e-05 2024-09-17T22:41:43.058 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.058 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.058 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.058 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72654 3.0 3:04644e28:::rbd_data.115fe9ac8535.0000000000000005:9 [read 3751936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.059 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726150+0000", 2024-09-17T22:41:43.059 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2976572019999999, 2024-09-17T22:41:43.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023793538999999999, 2024-09-17T22:41:43.059 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.059 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.059 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.059 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.059 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.059 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.059 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72654 2024-09-17T22:41:43.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.059 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726150+0000", 2024-09-17T22:41:43.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726150+0000", 2024-09-17T22:41:43.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726151+0000", 2024-09-17T22:41:43.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.89e-07 2024-09-17T22:41:43.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726153+0000", 2024-09-17T22:41:43.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0040000000000002e-06 2024-09-17T22:41:43.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726159+0000", 2024-09-17T22:41:43.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9000000000000003e-06 2024-09-17T22:41:43.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726163+0000", 2024-09-17T22:41:43.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7559999999999999e-06 2024-09-17T22:41:43.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749793+0000", 2024-09-17T22:41:43.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023630003 2024-09-17T22:41:43.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749869+0000", 2024-09-17T22:41:43.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5401000000000005e-05 2024-09-17T22:41:43.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749944+0000", 2024-09-17T22:41:43.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5086000000000006e-05 2024-09-17T22:41:43.063 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.063 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.063 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.063 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72655 3.18 3:1997a52f:::rbd_data.115fe9ac8535.0000000000000006:9 [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.063 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726170+0000", 2024-09-17T22:41:43.063 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2976378679999998, 2024-09-17T22:41:43.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023772537, 2024-09-17T22:41:43.063 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.063 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.064 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.064 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.064 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.064 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.064 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72655 2024-09-17T22:41:43.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.064 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726170+0000", 2024-09-17T22:41:43.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726170+0000", 2024-09-17T22:41:43.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726170+0000", 2024-09-17T22:41:43.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.77e-07 2024-09-17T22:41:43.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726180+0000", 2024-09-17T22:41:43.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8199999999999992e-06 2024-09-17T22:41:43.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726186+0000", 2024-09-17T22:41:43.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3740000000000003e-06 2024-09-17T22:41:43.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726191+0000", 2024-09-17T22:41:43.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.259e-06 2024-09-17T22:41:43.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749785+0000", 2024-09-17T22:41:43.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023593139999999999 2024-09-17T22:41:43.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749867+0000", 2024-09-17T22:41:43.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2699000000000002e-05 2024-09-17T22:41:43.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749942+0000", 2024-09-17T22:41:43.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4868000000000002e-05 2024-09-17T22:41:43.067 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.067 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.067 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.068 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72656 3.18 3:1997a52f:::rbd_data.115fe9ac8535.0000000000000006:9 [read 3796992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.068 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726199+0000", 2024-09-17T22:41:43.068 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2976088580000003, 2024-09-17T22:41:43.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023907336000000001, 2024-09-17T22:41:43.068 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.068 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.068 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.068 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.068 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.068 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.068 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72656 2024-09-17T22:41:43.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.069 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726199+0000", 2024-09-17T22:41:43.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726199+0000", 2024-09-17T22:41:43.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726199+0000", 2024-09-17T22:41:43.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3399999999999999e-07 2024-09-17T22:41:43.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726201+0000", 2024-09-17T22:41:43.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0109999999999999e-06 2024-09-17T22:41:43.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726207+0000", 2024-09-17T22:41:43.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5249999999999996e-06 2024-09-17T22:41:43.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726212+0000", 2024-09-17T22:41:43.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8910000000000003e-06 2024-09-17T22:41:43.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749969+0000", 2024-09-17T22:41:43.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023757256000000001 2024-09-17T22:41:43.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750039+0000", 2024-09-17T22:41:43.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9925999999999994e-05 2024-09-17T22:41:43.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750106+0000", 2024-09-17T22:41:43.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7193000000000005e-05 2024-09-17T22:41:43.072 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.072 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.072 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.072 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72657 3.0 3:0299d250:::rbd_data.115fe9ac8535.0000000000000000:9 [read 4042752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.072 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726218+0000", 2024-09-17T22:41:43.072 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2975897600000001, 2024-09-17T22:41:43.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023899917999999999, 2024-09-17T22:41:43.072 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.072 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.072 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.073 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.073 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.073 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72657 2024-09-17T22:41:43.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.073 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726218+0000", 2024-09-17T22:41:43.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726218+0000", 2024-09-17T22:41:43.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726218+0000", 2024-09-17T22:41:43.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7099999999999997e-07 2024-09-17T22:41:43.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726220+0000", 2024-09-17T22:41:43.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.95e-06 2024-09-17T22:41:43.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726226+0000", 2024-09-17T22:41:43.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8069999999999996e-06 2024-09-17T22:41:43.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726230+0000", 2024-09-17T22:41:43.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.544e-06 2024-09-17T22:41:43.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749983+0000", 2024-09-17T22:41:43.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023752932000000001 2024-09-17T22:41:43.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750051+0000", 2024-09-17T22:41:43.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8052000000000005e-05 2024-09-17T22:41:43.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750118+0000", 2024-09-17T22:41:43.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6261999999999997e-05 2024-09-17T22:41:43.076 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.076 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.076 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.076 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72662 3.0 3:04644e28:::rbd_data.115fe9ac8535.0000000000000005:9 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.076 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726261+0000", 2024-09-17T22:41:43.076 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2975466520000003, 2024-09-17T22:41:43.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02403953, 2024-09-17T22:41:43.077 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.077 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.077 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.077 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.077 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.077 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.077 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72662 2024-09-17T22:41:43.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.077 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726261+0000", 2024-09-17T22:41:43.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726261+0000", 2024-09-17T22:41:43.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726261+0000", 2024-09-17T22:41:43.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2e-07 2024-09-17T22:41:43.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726263+0000", 2024-09-17T22:41:43.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9199999999999998e-06 2024-09-17T22:41:43.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726269+0000", 2024-09-17T22:41:43.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.767e-06 2024-09-17T22:41:43.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726274+0000", 2024-09-17T22:41:43.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6770000000000003e-06 2024-09-17T22:41:43.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750158+0000", 2024-09-17T22:41:43.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023883824000000001 2024-09-17T22:41:43.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750228+0000", 2024-09-17T22:41:43.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0813000000000001e-05 2024-09-17T22:41:43.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750300+0000", 2024-09-17T22:41:43.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2008999999999994e-05 2024-09-17T22:41:43.080 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.080 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.080 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.080 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72663 3.18 3:1997a52f:::rbd_data.115fe9ac8535.0000000000000006:9 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.080 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726280+0000", 2024-09-17T22:41:43.081 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2975275230000003, 2024-09-17T22:41:43.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023998477000000001, 2024-09-17T22:41:43.081 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.081 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.081 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.081 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.081 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.081 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.081 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72663 2024-09-17T22:41:43.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.081 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726280+0000", 2024-09-17T22:41:43.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726280+0000", 2024-09-17T22:41:43.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726281+0000", 2024-09-17T22:41:43.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4099999999999999e-07 2024-09-17T22:41:43.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726283+0000", 2024-09-17T22:41:43.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-17T22:41:43.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726289+0000", 2024-09-17T22:41:43.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8830000000000003e-06 2024-09-17T22:41:43.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726301+0000", 2024-09-17T22:41:43.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2618e-05 2024-09-17T22:41:43.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750147+0000", 2024-09-17T22:41:43.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023846201000000001 2024-09-17T22:41:43.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750213+0000", 2024-09-17T22:41:43.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5724999999999998e-05 2024-09-17T22:41:43.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750278+0000", 2024-09-17T22:41:43.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5368000000000002e-05 2024-09-17T22:41:43.085 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.085 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.085 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.085 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72664 3.18 3:1997a52f:::rbd_data.115fe9ac8535.0000000000000006:9 [read 3215360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.085 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726307+0000", 2024-09-17T22:41:43.085 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2975003259999998, 2024-09-17T22:41:43.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024121613, 2024-09-17T22:41:43.085 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.085 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.085 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.085 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.085 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.085 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.086 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72664 2024-09-17T22:41:43.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.086 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726307+0000", 2024-09-17T22:41:43.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726307+0000", 2024-09-17T22:41:43.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726308+0000", 2024-09-17T22:41:43.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4700000000000002e-07 2024-09-17T22:41:43.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726315+0000", 2024-09-17T22:41:43.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4279999999999999e-06 2024-09-17T22:41:43.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726322+0000", 2024-09-17T22:41:43.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.601e-06 2024-09-17T22:41:43.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726326+0000", 2024-09-17T22:41:43.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8509999999999998e-06 2024-09-17T22:41:43.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750301+0000", 2024-09-17T22:41:43.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023974691999999999 2024-09-17T22:41:43.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750366+0000", 2024-09-17T22:41:43.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4863000000000006e-05 2024-09-17T22:41:43.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750429+0000", 2024-09-17T22:41:43.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2731000000000006e-05 2024-09-17T22:41:43.091 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.091 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.091 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.091 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72666 3.0 3:0299d250:::rbd_data.115fe9ac8535.0000000000000000:9 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.091 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726332+0000", 2024-09-17T22:41:43.091 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2974750589999999, 2024-09-17T22:41:43.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024140720000000001, 2024-09-17T22:41:43.091 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.091 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.091 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.091 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.091 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.091 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.091 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72666 2024-09-17T22:41:43.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.092 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726332+0000", 2024-09-17T22:41:43.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726332+0000", 2024-09-17T22:41:43.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726333+0000", 2024-09-17T22:41:43.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T22:41:43.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726335+0000", 2024-09-17T22:41:43.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.767e-06 2024-09-17T22:41:43.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726340+0000", 2024-09-17T22:41:43.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5489999999999997e-06 2024-09-17T22:41:43.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726345+0000", 2024-09-17T22:41:43.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.34e-06 2024-09-17T22:41:43.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750326+0000", 2024-09-17T22:41:43.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023981201000000001 2024-09-17T22:41:43.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750395+0000", 2024-09-17T22:41:43.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9615000000000003e-05 2024-09-17T22:41:43.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750473+0000", 2024-09-17T22:41:43.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7815e-05 2024-09-17T22:41:43.095 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.095 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.095 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.095 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72671 3.18 3:1997a52f:::rbd_data.115fe9ac8535.0000000000000006:9 [read 3485696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.095 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726372+0000", 2024-09-17T22:41:43.095 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2974359209999999, 2024-09-17T22:41:43.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024212465999999998, 2024-09-17T22:41:43.095 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.095 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.095 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.096 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.096 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.096 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.096 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72671 2024-09-17T22:41:43.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.096 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726372+0000", 2024-09-17T22:41:43.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726372+0000", 2024-09-17T22:41:43.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726372+0000", 2024-09-17T22:41:43.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.27e-07 2024-09-17T22:41:43.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726374+0000", 2024-09-17T22:41:43.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.708e-06 2024-09-17T22:41:43.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726380+0000", 2024-09-17T22:41:43.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0850000000000002e-06 2024-09-17T22:41:43.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726384+0000", 2024-09-17T22:41:43.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3640000000000001e-06 2024-09-17T22:41:43.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.098 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750452+0000", 2024-09-17T22:41:43.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024067708 2024-09-17T22:41:43.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.098 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750519+0000", 2024-09-17T22:41:43.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7248000000000004e-05 2024-09-17T22:41:43.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750584+0000", 2024-09-17T22:41:43.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5025999999999997e-05 2024-09-17T22:41:43.099 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.099 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.099 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.099 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72672 3.0 3:0299d250:::rbd_data.115fe9ac8535.0000000000000000:9 [read 3792896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.099 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726390+0000", 2024-09-17T22:41:43.099 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2974178609999996, 2024-09-17T22:41:43.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024256847000000002, 2024-09-17T22:41:43.100 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.100 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.100 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.100 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.100 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.100 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.100 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72672 2024-09-17T22:41:43.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.100 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726390+0000", 2024-09-17T22:41:43.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726390+0000", 2024-09-17T22:41:43.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726390+0000", 2024-09-17T22:41:43.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:41:43.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726392+0000", 2024-09-17T22:41:43.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9209999999999999e-06 2024-09-17T22:41:43.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726399+0000", 2024-09-17T22:41:43.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5910000000000001e-06 2024-09-17T22:41:43.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726403+0000", 2024-09-17T22:41:43.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.741e-06 2024-09-17T22:41:43.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750500+0000", 2024-09-17T22:41:43.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024096362999999999 2024-09-17T22:41:43.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750569+0000", 2024-09-17T22:41:43.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9243000000000001e-05 2024-09-17T22:41:43.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750646+0000", 2024-09-17T22:41:43.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7602999999999995e-05 2024-09-17T22:41:43.103 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.103 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.103 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.104 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72677 3.0 3:04644e28:::rbd_data.115fe9ac8535.0000000000000005:9 [read 3502080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.104 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726433+0000", 2024-09-17T22:41:43.104 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2973744649999999, 2024-09-17T22:41:43.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024382343000000001, 2024-09-17T22:41:43.104 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.104 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.104 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.104 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.104 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.104 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.104 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72677 2024-09-17T22:41:43.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.104 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726433+0000", 2024-09-17T22:41:43.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726433+0000", 2024-09-17T22:41:43.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726433+0000", 2024-09-17T22:41:43.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:41:43.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726435+0000", 2024-09-17T22:41:43.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.866e-06 2024-09-17T22:41:43.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726441+0000", 2024-09-17T22:41:43.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4840000000000003e-06 2024-09-17T22:41:43.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726451+0000", 2024-09-17T22:41:43.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0101999999999999e-05 2024-09-17T22:41:43.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750681+0000", 2024-09-17T22:41:43.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024230054000000001 2024-09-17T22:41:43.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750748+0000", 2024-09-17T22:41:43.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7269999999999998e-05 2024-09-17T22:41:43.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750815+0000", 2024-09-17T22:41:43.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7156000000000006e-05 2024-09-17T22:41:43.108 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.108 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.108 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.108 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72678 3.0 3:04644e28:::rbd_data.115fe9ac8535.0000000000000005:9 [read 3772416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.108 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726458+0000", 2024-09-17T22:41:43.108 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2973493469999999, 2024-09-17T22:41:43.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024566917000000001, 2024-09-17T22:41:43.108 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.108 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.108 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.108 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.108 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.108 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.108 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72678 2024-09-17T22:41:43.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.109 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726458+0000", 2024-09-17T22:41:43.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726458+0000", 2024-09-17T22:41:43.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726459+0000", 2024-09-17T22:41:43.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9100000000000004e-07 2024-09-17T22:41:43.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726461+0000", 2024-09-17T22:41:43.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0829999999999998e-06 2024-09-17T22:41:43.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726468+0000", 2024-09-17T22:41:43.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0169999999999999e-06 2024-09-17T22:41:43.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726472+0000", 2024-09-17T22:41:43.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6789999999999996e-06 2024-09-17T22:41:43.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750861+0000", 2024-09-17T22:41:43.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024388183000000001 2024-09-17T22:41:43.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750942+0000", 2024-09-17T22:41:43.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0970999999999994e-05 2024-09-17T22:41:43.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.751025+0000", 2024-09-17T22:41:43.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3492999999999995e-05 2024-09-17T22:41:43.112 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.112 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.112 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.112 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72679 3.0 3:0299d250:::rbd_data.115fe9ac8535.0000000000000000:9 [read 4001792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:43.112 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726487+0000", 2024-09-17T22:41:43.112 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.2973200289999998, 2024-09-17T22:41:43.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024710454999999999, 2024-09-17T22:41:43.113 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.113 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.113 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.113 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.113 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.113 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.113 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72679 2024-09-17T22:41:43.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.113 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726487+0000", 2024-09-17T22:41:43.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726487+0000", 2024-09-17T22:41:43.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726488+0000", 2024-09-17T22:41:43.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2399999999999998e-07 2024-09-17T22:41:43.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726490+0000", 2024-09-17T22:41:43.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9819999999999998e-06 2024-09-17T22:41:43.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726496+0000", 2024-09-17T22:41:43.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8730000000000004e-06 2024-09-17T22:41:43.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726501+0000", 2024-09-17T22:41:43.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7620000000000003e-06 2024-09-17T22:41:43.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.751063+0000", 2024-09-17T22:41:43.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024562143000000002 2024-09-17T22:41:43.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.115 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.115 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.751135+0000", 2024-09-17T22:41:43.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2385999999999994e-05 2024-09-17T22:41:43.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.751198+0000", 2024-09-17T22:41:43.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2785000000000003e-05 2024-09-17T22:41:43.116 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.116 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.116 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.116 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.116 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.116 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:43.117 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is set 2024-09-17T22:41:43.118 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T22:41:43.119 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:41:43.119 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:43.115+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:43.120 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-17T22:41:43.218 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:43.216+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:43.254 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-17T22:41:43.266 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:43.266 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:43.266 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:41:43.266 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:43.276 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:41:43.319 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:43.317+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:43.419 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:43.417+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:43.427 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:43.427 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:43.427 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:43.427 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:43.427 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:43.436 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:41:43.463 INFO:teuthology.orchestra.run.smithi115.stdout:810 read 0x4af1e55 thru 0x4afbca8 (0x9e54 bytes) 2024-09-17T22:41:43.468 INFO:teuthology.orchestra.run.smithi115.stdout:811 write 0xabd6de5 thru 0xabdf8c0 (0x8adc bytes) 2024-09-17T22:41:43.520 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:43.517+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:43.561 INFO:teuthology.orchestra.run.smithi115.stdout:812 trunc from 0xb2de0da to 0x5981b36 2024-09-17T22:41:43.583 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:43.583 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:43.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38083 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.763873+0000", 2024-09-17T22:41:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.816469292000001, 2024-09-17T22:41:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023344747999999999, 2024-09-17T22:41:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.584 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.585 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38083 2024-09-17T22:41:43.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.585 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763873+0000", 2024-09-17T22:41:43.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763873+0000", 2024-09-17T22:41:43.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763874+0000", 2024-09-17T22:41:43.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0899999999999997e-07 2024-09-17T22:41:43.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763875+0000", 2024-09-17T22:41:43.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8530000000000001e-06 2024-09-17T22:41:43.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763881+0000", 2024-09-17T22:41:43.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4879999999999998e-06 2024-09-17T22:41:43.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763886+0000", 2024-09-17T22:41:43.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.587e-06 2024-09-17T22:41:43.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787057+0000", 2024-09-17T22:41:43.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023170611000000001 2024-09-17T22:41:43.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787127+0000", 2024-09-17T22:41:43.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0259000000000006e-05 2024-09-17T22:41:43.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787218+0000", 2024-09-17T22:41:43.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0641000000000002e-05 2024-09-17T22:41:43.588 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.588 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.588 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.588 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38105 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.588 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.763938+0000", 2024-09-17T22:41:43.588 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.816404485, 2024-09-17T22:41:43.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023471315999999999, 2024-09-17T22:41:43.588 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.588 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.589 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.589 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.589 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.589 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.589 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38105 2024-09-17T22:41:43.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.589 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763938+0000", 2024-09-17T22:41:43.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763938+0000", 2024-09-17T22:41:43.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763939+0000", 2024-09-17T22:41:43.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:41:43.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763941+0000", 2024-09-17T22:41:43.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9489999999999999e-06 2024-09-17T22:41:43.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763946+0000", 2024-09-17T22:41:43.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9139999999999996e-06 2024-09-17T22:41:43.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763951+0000", 2024-09-17T22:41:43.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6589999999999998e-06 2024-09-17T22:41:43.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787249+0000", 2024-09-17T22:41:43.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023297989000000002 2024-09-17T22:41:43.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787319+0000", 2024-09-17T22:41:43.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9836e-05 2024-09-17T22:41:43.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787409+0000", 2024-09-17T22:41:43.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0457000000000005e-05 2024-09-17T22:41:43.592 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.592 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.592 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.593 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38107 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.593 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:43.593 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.816385695999999, 2024-09-17T22:41:43.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023669550000000001, 2024-09-17T22:41:43.593 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.593 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.593 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.593 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.593 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.593 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.593 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38107 2024-09-17T22:41:43.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.593 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:43.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:43.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:43.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:41:43.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763959+0000", 2024-09-17T22:41:43.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.849e-06 2024-09-17T22:41:43.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763965+0000", 2024-09-17T22:41:43.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6110000000000001e-06 2024-09-17T22:41:43.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763969+0000", 2024-09-17T22:41:43.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-06 2024-09-17T22:41:43.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787451+0000", 2024-09-17T22:41:43.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023482281000000001 2024-09-17T22:41:43.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787516+0000", 2024-09-17T22:41:43.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4930000000000003e-05 2024-09-17T22:41:43.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787626+0000", 2024-09-17T22:41:43.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011000700000000001 2024-09-17T22:41:43.597 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.597 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.597 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.597 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38141 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.597 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764408+0000", 2024-09-17T22:41:43.597 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.815934464, 2024-09-17T22:41:43.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023422268, 2024-09-17T22:41:43.597 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.598 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.598 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.598 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.598 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.598 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.598 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38141 2024-09-17T22:41:43.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.598 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764408+0000", 2024-09-17T22:41:43.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764408+0000", 2024-09-17T22:41:43.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764409+0000", 2024-09-17T22:41:43.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:41:43.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764412+0000", 2024-09-17T22:41:43.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4470000000000002e-06 2024-09-17T22:41:43.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764421+0000", 2024-09-17T22:41:43.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7299999999999994e-06 2024-09-17T22:41:43.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764427+0000", 2024-09-17T22:41:43.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.392e-06 2024-09-17T22:41:43.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787669+0000", 2024-09-17T22:41:43.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023241385 2024-09-17T22:41:43.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787743+0000", 2024-09-17T22:41:43.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4260999999999999e-05 2024-09-17T22:41:43.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787830+0000", 2024-09-17T22:41:43.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7398999999999994e-05 2024-09-17T22:41:43.601 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.601 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.601 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.601 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38143 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.601 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:43.602 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.815904536, 2024-09-17T22:41:43.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023565691, 2024-09-17T22:41:43.602 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.602 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.602 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.602 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.602 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.602 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.602 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38143 2024-09-17T22:41:43.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.602 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:43.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:43.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:43.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6199999999999998e-07 2024-09-17T22:41:43.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764441+0000", 2024-09-17T22:41:43.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1179999999999999e-06 2024-09-17T22:41:43.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764447+0000", 2024-09-17T22:41:43.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1940000000000003e-06 2024-09-17T22:41:43.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764452+0000", 2024-09-17T22:41:43.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9150000000000004e-06 2024-09-17T22:41:43.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787881+0000", 2024-09-17T22:41:43.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023429141000000001 2024-09-17T22:41:43.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787947+0000", 2024-09-17T22:41:43.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5902999999999996e-05 2024-09-17T22:41:43.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788004+0000", 2024-09-17T22:41:43.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6957999999999999e-05 2024-09-17T22:41:43.606 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.606 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.606 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.606 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38150 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.606 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:43.606 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.815884534, 2024-09-17T22:41:43.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023741399, 2024-09-17T22:41:43.606 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.606 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.606 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.606 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.606 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.607 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.607 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38150 2024-09-17T22:41:43.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.607 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:43.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:43.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:43.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8299999999999998e-07 2024-09-17T22:41:43.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764460+0000", 2024-09-17T22:41:43.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.855e-06 2024-09-17T22:41:43.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764466+0000", 2024-09-17T22:41:43.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8200000000000002e-06 2024-09-17T22:41:43.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764471+0000", 2024-09-17T22:41:43.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8319999999999996e-06 2024-09-17T22:41:43.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788032+0000", 2024-09-17T22:41:43.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023561480999999999 2024-09-17T22:41:43.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788106+0000", 2024-09-17T22:41:43.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3406999999999997e-05 2024-09-17T22:41:43.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788199+0000", 2024-09-17T22:41:43.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3621000000000004e-05 2024-09-17T22:41:43.610 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.610 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.610 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.610 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38152 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 83968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.610 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:43.610 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.815865817000001, 2024-09-17T22:41:43.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023859726000000001, 2024-09-17T22:41:43.610 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.611 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.611 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.611 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.611 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.611 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.611 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38152 2024-09-17T22:41:43.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.611 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:43.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:43.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:43.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3000000000000001e-07 2024-09-17T22:41:43.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764479+0000", 2024-09-17T22:41:43.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8920000000000001e-06 2024-09-17T22:41:43.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764485+0000", 2024-09-17T22:41:43.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:41:43.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764490+0000", 2024-09-17T22:41:43.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6890000000000003e-06 2024-09-17T22:41:43.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788232+0000", 2024-09-17T22:41:43.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023742452000000001 2024-09-17T22:41:43.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788276+0000", 2024-09-17T22:41:43.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3689000000000001e-05 2024-09-17T22:41:43.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788336+0000", 2024-09-17T22:41:43.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0581999999999997e-05 2024-09-17T22:41:43.614 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.614 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.614 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.615 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38184 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 161792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.615 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764589+0000", 2024-09-17T22:41:43.615 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.815753424, 2024-09-17T22:41:43.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023956801, 2024-09-17T22:41:43.615 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.615 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.615 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.615 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.615 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.615 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.615 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38184 2024-09-17T22:41:43.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.615 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764589+0000", 2024-09-17T22:41:43.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764589+0000", 2024-09-17T22:41:43.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764590+0000", 2024-09-17T22:41:43.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7800000000000001e-07 2024-09-17T22:41:43.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764592+0000", 2024-09-17T22:41:43.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6929999999999999e-06 2024-09-17T22:41:43.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764600+0000", 2024-09-17T22:41:43.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1140000000000006e-06 2024-09-17T22:41:43.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764606+0000", 2024-09-17T22:41:43.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6200000000000004e-06 2024-09-17T22:41:43.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788374+0000", 2024-09-17T22:41:43.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023768083999999998 2024-09-17T22:41:43.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788447+0000", 2024-09-17T22:41:43.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2922000000000004e-05 2024-09-17T22:41:43.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788546+0000", 2024-09-17T22:41:43.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8789999999999994e-05 2024-09-17T22:41:43.618 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.618 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.618 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.619 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38186 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.619 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:43.619 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.815728474, 2024-09-17T22:41:43.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024156486000000001, 2024-09-17T22:41:43.619 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.619 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.619 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.619 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38186 2024-09-17T22:41:43.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.619 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:43.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:43.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:43.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:41:43.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764621+0000", 2024-09-17T22:41:43.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6409999999999996e-06 2024-09-17T22:41:43.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764628+0000", 2024-09-17T22:41:43.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0360000000000001e-06 2024-09-17T22:41:43.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764633+0000", 2024-09-17T22:41:43.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0300000000000001e-06 2024-09-17T22:41:43.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788594+0000", 2024-09-17T22:41:43.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023960921999999999 2024-09-17T22:41:43.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788676+0000", 2024-09-17T22:41:43.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2123e-05 2024-09-17T22:41:43.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788771+0000", 2024-09-17T22:41:43.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4284999999999996e-05 2024-09-17T22:41:43.623 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.623 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.623 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.623 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38218 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 350208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.623 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764706+0000", 2024-09-17T22:41:43.623 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.815636221, 2024-09-17T22:41:43.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024324732000000002, 2024-09-17T22:41:43.623 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.623 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.623 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.623 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.624 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.624 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.624 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38218 2024-09-17T22:41:43.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.624 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764706+0000", 2024-09-17T22:41:43.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764706+0000", 2024-09-17T22:41:43.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764707+0000", 2024-09-17T22:41:43.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7199999999999999e-07 2024-09-17T22:41:43.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764710+0000", 2024-09-17T22:41:43.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6819999999999999e-06 2024-09-17T22:41:43.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764716+0000", 2024-09-17T22:41:43.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7900000000000002e-06 2024-09-17T22:41:43.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764722+0000", 2024-09-17T22:41:43.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2560000000000002e-06 2024-09-17T22:41:43.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788867+0000", 2024-09-17T22:41:43.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024144982999999998 2024-09-17T22:41:43.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788941+0000", 2024-09-17T22:41:43.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4529000000000004e-05 2024-09-17T22:41:43.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789031+0000", 2024-09-17T22:41:43.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9920000000000006e-05 2024-09-17T22:41:43.627 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.627 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.627 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.627 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38222 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.627 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764765+0000", 2024-09-17T22:41:43.627 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.815577366999999, 2024-09-17T22:41:43.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024443910999999999, 2024-09-17T22:41:43.628 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.628 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.628 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.628 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.628 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.628 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.628 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38222 2024-09-17T22:41:43.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.628 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764765+0000", 2024-09-17T22:41:43.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764765+0000", 2024-09-17T22:41:43.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764766+0000", 2024-09-17T22:41:43.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8999999999999996e-07 2024-09-17T22:41:43.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764769+0000", 2024-09-17T22:41:43.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7659999999999999e-06 2024-09-17T22:41:43.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764776+0000", 2024-09-17T22:41:43.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4229999999999999e-06 2024-09-17T22:41:43.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764783+0000", 2024-09-17T22:41:43.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9600000000000003e-06 2024-09-17T22:41:43.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789060+0000", 2024-09-17T22:41:43.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024277343 2024-09-17T22:41:43.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789129+0000", 2024-09-17T22:41:43.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8367999999999993e-05 2024-09-17T22:41:43.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789209+0000", 2024-09-17T22:41:43.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0461000000000001e-05 2024-09-17T22:41:43.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.631 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.631 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38671 3.12 3:4a5b92ac:::rbd_data.115fe0c30ef6.0000000000000011:5 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.631 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778865+0000", 2024-09-17T22:41:43.632 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.801477307000001, 2024-09-17T22:41:43.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023227351, 2024-09-17T22:41:43.632 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.632 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.632 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.632 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38671 2024-09-17T22:41:43.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.632 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778865+0000", 2024-09-17T22:41:43.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778865+0000", 2024-09-17T22:41:43.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778866+0000", 2024-09-17T22:41:43.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0900000000000002e-07 2024-09-17T22:41:43.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778868+0000", 2024-09-17T22:41:43.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4339999999999999e-06 2024-09-17T22:41:43.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778875+0000", 2024-09-17T22:41:43.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4559999999999997e-06 2024-09-17T22:41:43.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778880+0000", 2024-09-17T22:41:43.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2689999999999999e-06 2024-09-17T22:41:43.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.801978+0000", 2024-09-17T22:41:43.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023098225999999999 2024-09-17T22:41:43.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802035+0000", 2024-09-17T22:41:43.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7123999999999997e-05 2024-09-17T22:41:43.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802093+0000", 2024-09-17T22:41:43.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7432999999999997e-05 2024-09-17T22:41:43.635 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.635 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.636 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.636 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38673 3.12 3:4a5b92ac:::rbd_data.115fe0c30ef6.0000000000000011:5 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.636 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778886+0000", 2024-09-17T22:41:43.636 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.801456159000001, 2024-09-17T22:41:43.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023389259999999999, 2024-09-17T22:41:43.636 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.636 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.636 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.636 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.636 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.636 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.636 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38673 2024-09-17T22:41:43.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.637 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778886+0000", 2024-09-17T22:41:43.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778886+0000", 2024-09-17T22:41:43.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778887+0000", 2024-09-17T22:41:43.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:41:43.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778889+0000", 2024-09-17T22:41:43.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9379999999999999e-06 2024-09-17T22:41:43.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778895+0000", 2024-09-17T22:41:43.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0360000000000003e-06 2024-09-17T22:41:43.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778900+0000", 2024-09-17T22:41:43.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1220000000000003e-06 2024-09-17T22:41:43.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802126+0000", 2024-09-17T22:41:43.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023226607 2024-09-17T22:41:43.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802208+0000", 2024-09-17T22:41:43.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1157999999999998e-05 2024-09-17T22:41:43.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802276+0000", 2024-09-17T22:41:43.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8122999999999998e-05 2024-09-17T22:41:43.640 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.640 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.640 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.640 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38675 3.12 3:4a5b92ac:::rbd_data.115fe0c30ef6.0000000000000011:5 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.640 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:43.640 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.801420953999999, 2024-09-17T22:41:43.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023484221999999999, 2024-09-17T22:41:43.640 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.640 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.640 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.640 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.640 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.641 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.641 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38675 2024-09-17T22:41:43.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.641 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:43.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:43.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:43.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9999999999999997e-07 2024-09-17T22:41:43.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778924+0000", 2024-09-17T22:41:43.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3039999999999999e-06 2024-09-17T22:41:43.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778931+0000", 2024-09-17T22:41:43.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9519999999999997e-06 2024-09-17T22:41:43.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778937+0000", 2024-09-17T22:41:43.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3639999999999998e-06 2024-09-17T22:41:43.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802300+0000", 2024-09-17T22:41:43.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023363544 2024-09-17T22:41:43.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802347+0000", 2024-09-17T22:41:43.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6897999999999997e-05 2024-09-17T22:41:43.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802406+0000", 2024-09-17T22:41:43.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8560000000000002e-05 2024-09-17T22:41:43.644 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.644 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.644 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.644 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38679 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.644 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778945+0000", 2024-09-17T22:41:43.644 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.801397337999999, 2024-09-17T22:41:43.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023578015000000001, 2024-09-17T22:41:43.645 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.645 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.645 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.645 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.645 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.645 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.645 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38679 2024-09-17T22:41:43.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.645 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778945+0000", 2024-09-17T22:41:43.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778945+0000", 2024-09-17T22:41:43.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778946+0000", 2024-09-17T22:41:43.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.44e-07 2024-09-17T22:41:43.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778948+0000", 2024-09-17T22:41:43.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-17T22:41:43.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778955+0000", 2024-09-17T22:41:43.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.528e-06 2024-09-17T22:41:43.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778960+0000", 2024-09-17T22:41:43.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1919999999999996e-06 2024-09-17T22:41:43.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802429+0000", 2024-09-17T22:41:43.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023469561999999999 2024-09-17T22:41:43.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802470+0000", 2024-09-17T22:41:43.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1068000000000001e-05 2024-09-17T22:41:43.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802523+0000", 2024-09-17T22:41:43.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2852999999999997e-05 2024-09-17T22:41:43.648 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.648 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.648 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.649 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38680 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.649 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778966+0000", 2024-09-17T22:41:43.649 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.801376380000001, 2024-09-17T22:41:43.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023670668999999998, 2024-09-17T22:41:43.649 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.649 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.649 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.649 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.649 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.649 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.649 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38680 2024-09-17T22:41:43.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.649 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778966+0000", 2024-09-17T22:41:43.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778966+0000", 2024-09-17T22:41:43.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778967+0000", 2024-09-17T22:41:43.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.89e-07 2024-09-17T22:41:43.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778969+0000", 2024-09-17T22:41:43.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0219999999999999e-06 2024-09-17T22:41:43.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778975+0000", 2024-09-17T22:41:43.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3779999999999998e-06 2024-09-17T22:41:43.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778980+0000", 2024-09-17T22:41:43.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0699999999999997e-06 2024-09-17T22:41:43.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802545+0000", 2024-09-17T22:41:43.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023565358000000002 2024-09-17T22:41:43.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802585+0000", 2024-09-17T22:41:43.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0136999999999999e-05 2024-09-17T22:41:43.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802637+0000", 2024-09-17T22:41:43.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1314999999999999e-05 2024-09-17T22:41:43.652 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.653 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.653 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.653 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38697 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 675840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.653 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.779074+0000", 2024-09-17T22:41:43.653 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.801268127, 2024-09-17T22:41:43.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023676402999999999, 2024-09-17T22:41:43.653 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.653 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.653 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.653 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.653 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.653 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.653 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38697 2024-09-17T22:41:43.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.654 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779074+0000", 2024-09-17T22:41:43.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779074+0000", 2024-09-17T22:41:43.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779075+0000", 2024-09-17T22:41:43.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:41:43.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779077+0000", 2024-09-17T22:41:43.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8810000000000001e-06 2024-09-17T22:41:43.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779084+0000", 2024-09-17T22:41:43.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.691e-06 2024-09-17T22:41:43.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779088+0000", 2024-09-17T22:41:43.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6970000000000001e-06 2024-09-17T22:41:43.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802659+0000", 2024-09-17T22:41:43.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023571214 2024-09-17T22:41:43.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802699+0000", 2024-09-17T22:41:43.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9762000000000003e-05 2024-09-17T22:41:43.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802751+0000", 2024-09-17T22:41:43.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1572e-05 2024-09-17T22:41:43.657 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.657 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.657 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.657 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38698 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 749568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.657 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.779094+0000", 2024-09-17T22:41:43.657 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.801248190000001, 2024-09-17T22:41:43.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023778845999999999, 2024-09-17T22:41:43.658 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.658 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.658 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.658 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.658 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.658 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.658 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38698 2024-09-17T22:41:43.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.658 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779094+0000", 2024-09-17T22:41:43.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779094+0000", 2024-09-17T22:41:43.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779095+0000", 2024-09-17T22:41:43.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8799999999999998e-07 2024-09-17T22:41:43.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779097+0000", 2024-09-17T22:41:43.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8530000000000001e-06 2024-09-17T22:41:43.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779103+0000", 2024-09-17T22:41:43.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0410000000000003e-06 2024-09-17T22:41:43.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779107+0000", 2024-09-17T22:41:43.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T22:41:43.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802774+0000", 2024-09-17T22:41:43.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023666348 2024-09-17T22:41:43.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802814+0000", 2024-09-17T22:41:43.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0173000000000003e-05 2024-09-17T22:41:43.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802873+0000", 2024-09-17T22:41:43.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9484999999999997e-05 2024-09-17T22:41:43.661 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.661 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.661 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.661 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38825 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.661 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.781521+0000", 2024-09-17T22:41:43.662 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.798821123, 2024-09-17T22:41:43.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023214104999999999, 2024-09-17T22:41:43.662 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.662 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.662 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:43.662 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.662 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.662 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.662 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38825 2024-09-17T22:41:43.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.662 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781521+0000", 2024-09-17T22:41:43.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781521+0000", 2024-09-17T22:41:43.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781522+0000", 2024-09-17T22:41:43.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.53e-07 2024-09-17T22:41:43.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781524+0000", 2024-09-17T22:41:43.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0389999999999999e-06 2024-09-17T22:41:43.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781530+0000", 2024-09-17T22:41:43.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8209999999999998e-06 2024-09-17T22:41:43.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781534+0000", 2024-09-17T22:41:43.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6419999999999998e-06 2024-09-17T22:41:43.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804570+0000", 2024-09-17T22:41:43.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023035951999999998 2024-09-17T22:41:43.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:43.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804645+0000", 2024-09-17T22:41:43.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4696000000000004e-05 2024-09-17T22:41:43.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804736+0000", 2024-09-17T22:41:43.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0601999999999998e-05 2024-09-17T22:41:43.666 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.666 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.666 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.666 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38843 3.12 3.c4845672 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:43.666 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:43.666 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.798698912999999, 2024-09-17T22:41:43.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023236265999999998, 2024-09-17T22:41:43.666 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:43.666 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:43.666 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:43.667 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:43.667 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:43.667 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:43.667 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38843 2024-09-17T22:41:43.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.667 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:43.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:43.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:43.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:43.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:43.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:43.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:43.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:43.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:41:43.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:43.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781646+0000", 2024-09-17T22:41:43.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9290000000000001e-06 2024-09-17T22:41:43.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781652+0000", 2024-09-17T22:41:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7389999999999996e-06 2024-09-17T22:41:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781656+0000", 2024-09-17T22:41:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3830000000000002e-06 2024-09-17T22:41:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804871+0000", 2024-09-17T22:41:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023215257999999999 2024-09-17T22:41:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:43.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:43.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:43.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804880+0000", 2024-09-17T22:41:43.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5769999999999994e-06 2024-09-17T22:41:43.670 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.670 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.670 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.670 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:43.670 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:43.670 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:43.672 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:41:43.672 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:43.669+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:43.687 INFO:teuthology.orchestra.run.smithi115.stdout:813 write 0x7a8719a thru 0x7a8fbf1 (0x8a58 bytes) 2024-09-17T22:41:43.696 INFO:teuthology.orchestra.run.smithi115.stdout:817 punch from 0x154292c to 0x15493f6, (0x6aca bytes) 2024-09-17T22:41:43.706 INFO:teuthology.orchestra.run.smithi115.stdout:818 write 0xb5c1dfc thru 0xb5c25ae (0x7b3 bytes) 2024-09-17T22:41:43.709 INFO:teuthology.orchestra.run.smithi115.stdout:819 read 0xaa7de44 thru 0xaa85184 (0x7341 bytes) 2024-09-17T22:41:43.709 INFO:teuthology.orchestra.run.smithi115.stdout:820 read 0x16294f2 thru 0x1632924 (0x9433 bytes) 2024-09-17T22:41:43.712 INFO:teuthology.orchestra.run.smithi115.stdout:821 read 0x9390dd0 thru 0x939b558 (0xa789 bytes) 2024-09-17T22:41:43.712 INFO:teuthology.orchestra.run.smithi115.stdout:822 write 0x61f6261 thru 0x62028eb (0xc68b bytes) 2024-09-17T22:41:43.717 INFO:teuthology.orchestra.run.smithi115.stdout:824 read 0x2f759ac thru 0x2f7e083 (0x86d8 bytes) 2024-09-17T22:41:43.719 INFO:teuthology.orchestra.run.smithi115.stdout:825 punch from 0xf4cd8e to 0xf586ba, (0xb92c bytes) 2024-09-17T22:41:43.719 INFO:teuthology.orchestra.run.smithi115.stdout:826 writesame 0x11c482 thru 0x12746c data_size 0x1(0xafeb bytes) 2024-09-17T22:41:43.773 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:43.770+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:43.778 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:43.778 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:43.778 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:41:43.778 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:43.787 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:41:43.832 INFO:teuthology.orchestra.run.smithi115.stdout:828 trunc from 0xb5c25af to 0xd38849f 2024-09-17T22:41:43.837 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:43.837 INFO:teuthology.orchestra.run.smithi115.stdout:830 write 0x875c87e thru 0x876b3f7 (0xeb7a bytes) 2024-09-17T22:41:43.841 INFO:teuthology.orchestra.run.smithi115.stdout:831 punch from 0x95b7096 to 0x95bc13c, (0x50a6 bytes) 2024-09-17T22:41:43.842 INFO:teuthology.orchestra.run.smithi115.stdout:832 clone 13 order 22 su 131072 sc 14 2024-09-17T22:41:43.874 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:43.872+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:43.891 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:43.892 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:43.900 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:41:43.975 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:43.973+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:44.005 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:44.005 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:44.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:44.005 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:44.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.005 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36437 3.b 3:d64eaf12:::rbd_data.115f86d8437d.000000000000001c:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:44.005 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:41:44.005 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.286893320000001, 2024-09-17T22:41:44.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068296390000000002, 2024-09-17T22:41:44.006 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.006 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.006 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:44.006 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:44.006 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:44.006 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.006 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36437 2024-09-17T22:41:44.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.006 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:41:44.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:41:44.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715035+0000", 2024-09-17T22:41:44.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.288e-06 2024-09-17T22:41:44.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715041+0000", 2024-09-17T22:41:44.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.643e-06 2024-09-17T22:41:44.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715051+0000", 2024-09-17T22:41:44.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0597e-05 2024-09-17T22:41:44.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715056+0000", 2024-09-17T22:41:44.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1379999999999998e-06 2024-09-17T22:41:44.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715101+0000", 2024-09-17T22:41:44.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4433e-05 2024-09-17T22:41:44.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715187+0000", 2024-09-17T22:41:44.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6148000000000003e-05 2024-09-17T22:41:44.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:44.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.716264+0000", 2024-09-17T22:41:44.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010770179999999999 2024-09-17T22:41:44.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:44.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721735+0000", 2024-09-17T22:41:44.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0054709019999999997 2024-09-17T22:41:44.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:41:44.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721754+0000", 2024-09-17T22:41:44.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9133000000000001e-05 2024-09-17T22:41:44.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:44.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721807+0000", 2024-09-17T22:41:44.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2831999999999998e-05 2024-09-17T22:41:44.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.010 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.010 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721863+0000", 2024-09-17T22:41:44.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6507e-05 2024-09-17T22:41:44.011 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.011 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.011 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.011 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36743 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 188416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:44.011 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735018+0000", 2024-09-17T22:41:44.011 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.266909204999999, 2024-09-17T22:41:44.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0060955260000000004, 2024-09-17T22:41:44.011 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.011 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.011 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:44.011 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:44.011 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:44.012 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.012 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36743 2024-09-17T22:41:44.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.012 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735018+0000", 2024-09-17T22:41:44.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735018+0000", 2024-09-17T22:41:44.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735018+0000", 2024-09-17T22:41:44.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7199999999999999e-07 2024-09-17T22:41:44.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735020+0000", 2024-09-17T22:41:44.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9889999999999999e-06 2024-09-17T22:41:44.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735027+0000", 2024-09-17T22:41:44.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3409999999999999e-06 2024-09-17T22:41:44.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735032+0000", 2024-09-17T22:41:44.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.904e-06 2024-09-17T22:41:44.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.740910+0000", 2024-09-17T22:41:44.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.005878266 2024-09-17T22:41:44.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741009+0000", 2024-09-17T22:41:44.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8754999999999999e-05 2024-09-17T22:41:44.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741113+0000", 2024-09-17T22:41:44.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000104799 2024-09-17T22:41:44.015 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.015 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.015 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.015 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36744 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:44.015 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:41:44.015 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.266889178, 2024-09-17T22:41:44.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063082559999999999, 2024-09-17T22:41:44.016 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.016 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.016 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:44.016 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:44.016 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:44.016 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.016 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36744 2024-09-17T22:41:44.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.016 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:41:44.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:41:44.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:41:44.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:41:44.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735040+0000", 2024-09-17T22:41:44.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9630000000000001e-06 2024-09-17T22:41:44.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735047+0000", 2024-09-17T22:41:44.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3439999999999997e-06 2024-09-17T22:41:44.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735052+0000", 2024-09-17T22:41:44.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9250000000000003e-06 2024-09-17T22:41:44.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741147+0000", 2024-09-17T22:41:44.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0060954210000000002 2024-09-17T22:41:44.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741243+0000", 2024-09-17T22:41:44.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5674999999999995e-05 2024-09-17T22:41:44.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741346+0000", 2024-09-17T22:41:44.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103522 2024-09-17T22:41:44.019 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.019 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.019 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.019 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36745 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:44.019 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:41:44.019 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.266869194, 2024-09-17T22:41:44.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065186740000000003, 2024-09-17T22:41:44.020 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.020 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.020 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:44.020 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:44.020 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:44.020 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.020 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36745 2024-09-17T22:41:44.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.020 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:41:44.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:41:44.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:41:44.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2500000000000001e-07 2024-09-17T22:41:44.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735061+0000", 2024-09-17T22:41:44.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.824e-06 2024-09-17T22:41:44.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735068+0000", 2024-09-17T22:41:44.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8580000000000002e-06 2024-09-17T22:41:44.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735074+0000", 2024-09-17T22:41:44.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5829999999999997e-06 2024-09-17T22:41:44.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741379+0000", 2024-09-17T22:41:44.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063051909999999999 2024-09-17T22:41:44.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741472+0000", 2024-09-17T22:41:44.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3238000000000004e-05 2024-09-17T22:41:44.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741577+0000", 2024-09-17T22:41:44.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000104555 2024-09-17T22:41:44.023 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.023 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.023 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.024 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36746 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:44.024 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:41:44.024 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.266846228, 2024-09-17T22:41:44.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067275240000000004, 2024-09-17T22:41:44.024 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.024 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.024 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:44.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:44.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:44.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.024 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36746 2024-09-17T22:41:44.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.025 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:41:44.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:41:44.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:41:44.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.03e-07 2024-09-17T22:41:44.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735083+0000", 2024-09-17T22:41:44.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T22:41:44.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735090+0000", 2024-09-17T22:41:44.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7800000000000003e-06 2024-09-17T22:41:44.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735095+0000", 2024-09-17T22:41:44.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8500000000000002e-06 2024-09-17T22:41:44.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741608+0000", 2024-09-17T22:41:44.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065131809999999998 2024-09-17T22:41:44.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741703+0000", 2024-09-17T22:41:44.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4914999999999995e-05 2024-09-17T22:41:44.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741808+0000", 2024-09-17T22:41:44.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000105448 2024-09-17T22:41:44.028 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.028 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.028 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.028 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36747 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:44.028 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:41:44.028 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.266825675, 2024-09-17T22:41:44.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069468790000000004, 2024-09-17T22:41:44.028 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.028 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.029 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:44.029 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:44.029 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:44.029 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.029 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36747 2024-09-17T22:41:44.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.029 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:41:44.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:41:44.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735102+0000", 2024-09-17T22:41:44.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:41:44.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735104+0000", 2024-09-17T22:41:44.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7400000000000001e-06 2024-09-17T22:41:44.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735111+0000", 2024-09-17T22:41:44.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4189999999999996e-06 2024-09-17T22:41:44.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735116+0000", 2024-09-17T22:41:44.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0860000000000001e-06 2024-09-17T22:41:44.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741847+0000", 2024-09-17T22:41:44.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067306859999999996 2024-09-17T22:41:44.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741942+0000", 2024-09-17T22:41:44.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5180999999999996e-05 2024-09-17T22:41:44.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742048+0000", 2024-09-17T22:41:44.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106387 2024-09-17T22:41:44.032 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.032 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.032 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.033 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36748 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:44.033 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:41:44.033 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.266804354, 2024-09-17T22:41:44.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071864119999999997, 2024-09-17T22:41:44.033 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.033 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.033 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:44.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:44.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:44.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.033 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36748 2024-09-17T22:41:44.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.033 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:41:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:41:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:41:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.34e-07 2024-09-17T22:41:44.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735125+0000", 2024-09-17T22:41:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0020000000000001e-06 2024-09-17T22:41:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735132+0000", 2024-09-17T22:41:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3489999999999997e-06 2024-09-17T22:41:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735138+0000", 2024-09-17T22:41:44.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9709999999999998e-06 2024-09-17T22:41:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742087+0000", 2024-09-17T22:41:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069480669999999996 2024-09-17T22:41:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742188+0000", 2024-09-17T22:41:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101279 2024-09-17T22:41:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742309+0000", 2024-09-17T22:41:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012131 2024-09-17T22:41:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36749 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:41:44.037 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.266781667, 2024-09-17T22:41:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0074112570000000001, 2024-09-17T22:41:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36749 2024-09-17T22:41:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:41:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:41:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735146+0000", 2024-09-17T22:41:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9200000000000002e-07 2024-09-17T22:41:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735158+0000", 2024-09-17T22:41:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2508e-05 2024-09-17T22:41:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735166+0000", 2024-09-17T22:41:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6290000000000001e-06 2024-09-17T22:41:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735174+0000", 2024-09-17T22:41:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1000000000000004e-06 2024-09-17T22:41:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742354+0000", 2024-09-17T22:41:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071795660000000001 2024-09-17T22:41:44.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742454+0000", 2024-09-17T22:41:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100408 2024-09-17T22:41:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742557+0000", 2024-09-17T22:41:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102654 2024-09-17T22:41:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.041 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36750 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:41:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.266744541, 2024-09-17T22:41:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0077413890000000004, 2024-09-17T22:41:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36750 2024-09-17T22:41:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:41:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:41:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:41:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2399999999999998e-07 2024-09-17T22:41:44.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735185+0000", 2024-09-17T22:41:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1710000000000001e-06 2024-09-17T22:41:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735192+0000", 2024-09-17T22:41:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.81e-06 2024-09-17T22:41:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735197+0000", 2024-09-17T22:41:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9080000000000003e-06 2024-09-17T22:41:44.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742587+0000", 2024-09-17T22:41:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.007390359 2024-09-17T22:41:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742694+0000", 2024-09-17T22:41:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106901 2024-09-17T22:41:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742924+0000", 2024-09-17T22:41:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022961600000000001 2024-09-17T22:41:44.045 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36751 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:41:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.266722567, 2024-09-17T22:41:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079273859999999998, 2024-09-17T22:41:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.046 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36751 2024-09-17T22:41:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:41:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:41:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:41:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1600000000000001e-07 2024-09-17T22:41:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735207+0000", 2024-09-17T22:41:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1909999999999999e-06 2024-09-17T22:41:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735214+0000", 2024-09-17T22:41:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9269999999999999e-06 2024-09-17T22:41:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735220+0000", 2024-09-17T22:41:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3739999999999997e-06 2024-09-17T22:41:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742957+0000", 2024-09-17T22:41:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0077371699999999998 2024-09-17T22:41:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.743037+0000", 2024-09-17T22:41:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0370000000000005e-05 2024-09-17T22:41:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.743132+0000", 2024-09-17T22:41:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4838000000000002e-05 2024-09-17T22:41:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37658 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749751+0000", 2024-09-17T22:41:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.252176487, 2024-09-17T22:41:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061048459999999997, 2024-09-17T22:41:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.050 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37658 2024-09-17T22:41:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749751+0000", 2024-09-17T22:41:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749751+0000", 2024-09-17T22:41:44.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749751+0000", 2024-09-17T22:41:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8500000000000002e-07 2024-09-17T22:41:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749753+0000", 2024-09-17T22:41:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2579999999999998e-06 2024-09-17T22:41:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749760+0000", 2024-09-17T22:41:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9099999999999999e-06 2024-09-17T22:41:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749765+0000", 2024-09-17T22:41:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1170000000000003e-06 2024-09-17T22:41:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755677+0000", 2024-09-17T22:41:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059120140000000002 2024-09-17T22:41:44.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755756+0000", 2024-09-17T22:41:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9013999999999999e-05 2024-09-17T22:41:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755855+0000", 2024-09-17T22:41:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9048000000000004e-05 2024-09-17T22:41:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37659 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:44.054 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:41:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.252154923999999, 2024-09-17T22:41:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063144200000000003, 2024-09-17T22:41:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37659 2024-09-17T22:41:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:41:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:41:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749773+0000", 2024-09-17T22:41:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:41:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749774+0000", 2024-09-17T22:41:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:41:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749781+0000", 2024-09-17T22:41:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5740000000000001e-06 2024-09-17T22:41:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749786+0000", 2024-09-17T22:41:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9099999999999996e-06 2024-09-17T22:41:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755889+0000", 2024-09-17T22:41:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061034260000000003 2024-09-17T22:41:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755972+0000", 2024-09-17T22:41:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2795999999999998e-05 2024-09-17T22:41:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756087+0000", 2024-09-17T22:41:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000114418 2024-09-17T22:41:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37660 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:41:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.252125085999999, 2024-09-17T22:41:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065587800000000002, 2024-09-17T22:41:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37660 2024-09-17T22:41:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:41:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:41:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:41:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:41:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749805+0000", 2024-09-17T22:41:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-17T22:41:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749812+0000", 2024-09-17T22:41:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270000000000002e-06 2024-09-17T22:41:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749819+0000", 2024-09-17T22:41:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.759e-06 2024-09-17T22:41:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756153+0000", 2024-09-17T22:41:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063345809999999997 2024-09-17T22:41:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756257+0000", 2024-09-17T22:41:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103583 2024-09-17T22:41:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756361+0000", 2024-09-17T22:41:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103853 2024-09-17T22:41:44.062 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37661 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:41:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.252096492, 2024-09-17T22:41:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067442300000000004, 2024-09-17T22:41:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.063 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37661 2024-09-17T22:41:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:41:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:41:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:41:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9399999999999995e-07 2024-09-17T22:41:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749835+0000", 2024-09-17T22:41:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1049999999999997e-06 2024-09-17T22:41:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749855+0000", 2024-09-17T22:41:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9343000000000001e-05 2024-09-17T22:41:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749860+0000", 2024-09-17T22:41:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:41:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756395+0000", 2024-09-17T22:41:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065349900000000001 2024-09-17T22:41:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756479+0000", 2024-09-17T22:41:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4030999999999996e-05 2024-09-17T22:41:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756575+0000", 2024-09-17T22:41:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5835999999999995e-05 2024-09-17T22:41:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37662 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:41:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.252042685999999, 2024-09-17T22:41:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006927295, 2024-09-17T22:41:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:44.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37662 2024-09-17T22:41:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:41:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:41:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749885+0000", 2024-09-17T22:41:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2699999999999999e-07 2024-09-17T22:41:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749888+0000", 2024-09-17T22:41:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6599999999999999e-06 2024-09-17T22:41:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749895+0000", 2024-09-17T22:41:44.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4200000000000001e-06 2024-09-17T22:41:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749901+0000", 2024-09-17T22:41:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5090000000000001e-06 2024-09-17T22:41:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756605+0000", 2024-09-17T22:41:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067040629999999997 2024-09-17T22:41:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756691+0000", 2024-09-17T22:41:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5846000000000004e-05 2024-09-17T22:41:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756812+0000", 2024-09-17T22:41:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012116999999999999 2024-09-17T22:41:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37663 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:41:44.071 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.252018476, 2024-09-17T22:41:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071811440000000004, 2024-09-17T22:41:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37663 2024-09-17T22:41:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:41:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:41:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:41:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:41:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749911+0000", 2024-09-17T22:41:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.035e-06 2024-09-17T22:41:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749917+0000", 2024-09-17T22:41:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4370000000000004e-06 2024-09-17T22:41:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749922+0000", 2024-09-17T22:41:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6249999999999998e-06 2024-09-17T22:41:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756845+0000", 2024-09-17T22:41:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069227280000000004 2024-09-17T22:41:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756994+0000", 2024-09-17T22:41:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000149068 2024-09-17T22:41:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757090+0000", 2024-09-17T22:41:44.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5851000000000001e-05 2024-09-17T22:41:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37664 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:41:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.251997631, 2024-09-17T22:41:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.007384743, 2024-09-17T22:41:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:44.076 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37664 2024-09-17T22:41:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:41:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:41:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749930+0000", 2024-09-17T22:41:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:41:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749933+0000", 2024-09-17T22:41:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8430000000000001e-06 2024-09-17T22:41:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749939+0000", 2024-09-17T22:41:44.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:41:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749944+0000", 2024-09-17T22:41:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9230000000000001e-06 2024-09-17T22:41:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757124+0000", 2024-09-17T22:41:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071800020000000004 2024-09-17T22:41:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757212+0000", 2024-09-17T22:41:44.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7383e-05 2024-09-17T22:41:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757314+0000", 2024-09-17T22:41:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010247399999999999 2024-09-17T22:41:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:39363 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:41:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.210009292000001, 2024-09-17T22:41:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006226245, 2024-09-17T22:41:44.080 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 39363 2024-09-17T22:41:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:41:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:41:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:41:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3500000000000002e-07 2024-09-17T22:41:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791920+0000", 2024-09-17T22:41:44.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.869e-06 2024-09-17T22:41:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791926+0000", 2024-09-17T22:41:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8490000000000002e-06 2024-09-17T22:41:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791931+0000", 2024-09-17T22:41:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4269999999999999e-06 2024-09-17T22:41:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.797912+0000", 2024-09-17T22:41:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059808819999999999 2024-09-17T22:41:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.798020+0000", 2024-09-17T22:41:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010783800000000001 2024-09-17T22:41:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.798144+0000", 2024-09-17T22:41:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012404500000000001 2024-09-17T22:41:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.084 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4447.0:51869 3.6 e35/23 3:6037e35f:::rbd_object_map.115fe9ac8535:head v 35'71, mlcod=35'66)", 2024-09-17T22:41:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:41:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 10.224686992000001, 2024-09-17T22:41:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0066208669999999999, 2024-09-17T22:41:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:41:44.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:41:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777241+0000", 2024-09-17T22:41:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4000000000000001e-07 2024-09-17T22:41:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777246+0000", 2024-09-17T22:41:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2789999999999998e-06 2024-09-17T22:41:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777255+0000", 2024-09-17T22:41:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6209999999999993e-06 2024-09-17T22:41:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777259+0000", 2024-09-17T22:41:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7650000000000001e-06 2024-09-17T22:41:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777298+0000", 2024-09-17T22:41:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8797999999999997e-05 2024-09-17T22:41:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777311+0000", 2024-09-17T22:41:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2364999999999999e-05 2024-09-17T22:41:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.783814+0000", 2024-09-17T22:41:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065039169999999997 2024-09-17T22:41:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.783861+0000", 2024-09-17T22:41:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6482e-05 2024-09-17T22:41:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:105336 3.b 3:d3549b8d:::rbd_data.115ff9e8d63c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:41:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:43.787529+0000", 2024-09-17T22:41:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.214397687, 2024-09-17T22:41:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0099864259999999996, 2024-09-17T22:41:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:44.089 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 105336 2024-09-17T22:41:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.787529+0000", 2024-09-17T22:41:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.787529+0000", 2024-09-17T22:41:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:44.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.787531+0000", 2024-09-17T22:41:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9759999999999998e-06 2024-09-17T22:41:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788120+0000", 2024-09-17T22:41:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00058899799999999997 2024-09-17T22:41:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788138+0000", 2024-09-17T22:41:44.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7393000000000001e-05 2024-09-17T22:41:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788146+0000", 2024-09-17T22:41:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8220000000000007e-06 2024-09-17T22:41:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788199+0000", 2024-09-17T22:41:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3718000000000003e-05 2024-09-17T22:41:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:44.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788316+0000", 2024-09-17T22:41:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000116999 2024-09-17T22:41:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.791608+0000", 2024-09-17T22:41:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003291188 2024-09-17T22:41:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797379+0000", 2024-09-17T22:41:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057716829999999997 2024-09-17T22:41:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:41:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797401+0000", 2024-09-17T22:41:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1729000000000002e-05 2024-09-17T22:41:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797459+0000", 2024-09-17T22:41:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8115000000000002e-05 2024-09-17T22:41:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797516+0000", 2024-09-17T22:41:44.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6805000000000002e-05 2024-09-17T22:41:44.095 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.095 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.095 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.095 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:44.095 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:44.095 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:44.097 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T22:41:44.097 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:44.094+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:44.097 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-17T22:41:44.098 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:41:44.111 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone12 from 0xd38849f (overlap 0x5981b36) to 0x3ea7972 2024-09-17T22:41:44.197 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:44.194+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:44.292 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:44.292 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:41:44.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.292 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106746 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.292 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.226123+0000", 2024-09-17T22:41:44.293 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.044301447000000001, 2024-09-17T22:41:44.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044326962999999997, 2024-09-17T22:41:44.293 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.293 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.293 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.293 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.293 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.293 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.293 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106746 2024-09-17T22:41:44.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.293 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226123+0000", 2024-09-17T22:41:44.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226123+0000", 2024-09-17T22:41:44.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226123+0000", 2024-09-17T22:41:44.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7800000000000002e-07 2024-09-17T22:41:44.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226125+0000", 2024-09-17T22:41:44.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1569999999999998e-06 2024-09-17T22:41:44.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226132+0000", 2024-09-17T22:41:44.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9210000000000003e-06 2024-09-17T22:41:44.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226137+0000", 2024-09-17T22:41:44.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8940000000000001e-06 2024-09-17T22:41:44.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.296 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.296 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.296 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106778 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.296 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.232143+0000", 2024-09-17T22:41:44.296 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.038280852999999997, 2024-09-17T22:41:44.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038369254999999998, 2024-09-17T22:41:44.296 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.296 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.296 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.297 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.297 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106778 2024-09-17T22:41:44.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.297 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232143+0000", 2024-09-17T22:41:44.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232143+0000", 2024-09-17T22:41:44.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232143+0000", 2024-09-17T22:41:44.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1e-07 2024-09-17T22:41:44.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232144+0000", 2024-09-17T22:41:44.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0359999999999999e-06 2024-09-17T22:41:44.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232147+0000", 2024-09-17T22:41:44.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.903e-06 2024-09-17T22:41:44.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232150+0000", 2024-09-17T22:41:44.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000001e-06 2024-09-17T22:41:44.299 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.299 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.299 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.299 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106810 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.299 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.238146+0000", 2024-09-17T22:41:44.299 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.032278326000000003, 2024-09-17T22:41:44.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032421358999999997, 2024-09-17T22:41:44.300 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.300 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.300 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.300 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106810 2024-09-17T22:41:44.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.300 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238146+0000", 2024-09-17T22:41:44.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238146+0000", 2024-09-17T22:41:44.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238146+0000", 2024-09-17T22:41:44.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:41:44.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238148+0000", 2024-09-17T22:41:44.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9120000000000001e-06 2024-09-17T22:41:44.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238153+0000", 2024-09-17T22:41:44.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4140000000000002e-06 2024-09-17T22:41:44.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238156+0000", 2024-09-17T22:41:44.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8669999999999998e-06 2024-09-17T22:41:44.302 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.303 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.303 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.303 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106842 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.303 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.244125+0000", 2024-09-17T22:41:44.303 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.026298957000000001, 2024-09-17T22:41:44.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026477302000000001, 2024-09-17T22:41:44.303 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.303 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.303 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.303 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.303 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.304 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106842 2024-09-17T22:41:44.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.304 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244125+0000", 2024-09-17T22:41:44.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244125+0000", 2024-09-17T22:41:44.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244125+0000", 2024-09-17T22:41:44.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1800000000000002e-07 2024-09-17T22:41:44.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244126+0000", 2024-09-17T22:41:44.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.097e-06 2024-09-17T22:41:44.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244129+0000", 2024-09-17T22:41:44.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:41:44.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244132+0000", 2024-09-17T22:41:44.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.621e-06 2024-09-17T22:41:44.306 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.306 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.306 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.307 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106874 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.307 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.250172+0000", 2024-09-17T22:41:44.307 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020252005, 2024-09-17T22:41:44.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020457619, 2024-09-17T22:41:44.307 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.307 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.307 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.307 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.307 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106874 2024-09-17T22:41:44.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.307 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250172+0000", 2024-09-17T22:41:44.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250172+0000", 2024-09-17T22:41:44.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250172+0000", 2024-09-17T22:41:44.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:41:44.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250175+0000", 2024-09-17T22:41:44.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2510000000000001e-06 2024-09-17T22:41:44.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250181+0000", 2024-09-17T22:41:44.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1650000000000003e-06 2024-09-17T22:41:44.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250186+0000", 2024-09-17T22:41:44.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.074e-06 2024-09-17T22:41:44.310 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.310 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.310 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.310 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106906 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.310 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.254194+0000", 2024-09-17T22:41:44.310 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.016230474000000002, 2024-09-17T22:41:44.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016472519000000001, 2024-09-17T22:41:44.311 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.311 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.311 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.311 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.311 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106906 2024-09-17T22:41:44.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.311 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254194+0000", 2024-09-17T22:41:44.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254194+0000", 2024-09-17T22:41:44.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254194+0000", 2024-09-17T22:41:44.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8700000000000001e-07 2024-09-17T22:41:44.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254196+0000", 2024-09-17T22:41:44.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.007e-06 2024-09-17T22:41:44.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254202+0000", 2024-09-17T22:41:44.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8540000000000002e-06 2024-09-17T22:41:44.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254207+0000", 2024-09-17T22:41:44.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7530000000000001e-06 2024-09-17T22:41:44.314 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.314 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.314 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.314 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106936 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.314 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.260133+0000", 2024-09-17T22:41:44.314 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.010290818, 2024-09-17T22:41:44.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010585171000000001, 2024-09-17T22:41:44.315 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.315 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.315 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.315 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.315 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.315 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.315 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106936 2024-09-17T22:41:44.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.315 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260133+0000", 2024-09-17T22:41:44.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260133+0000", 2024-09-17T22:41:44.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260133+0000", 2024-09-17T22:41:44.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.35e-07 2024-09-17T22:41:44.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260135+0000", 2024-09-17T22:41:44.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.145e-06 2024-09-17T22:41:44.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260138+0000", 2024-09-17T22:41:44.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.709e-06 2024-09-17T22:41:44.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260142+0000", 2024-09-17T22:41:44.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8519999999999997e-06 2024-09-17T22:41:44.317 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.317 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.318 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.318 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106968 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.318 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.268131+0000", 2024-09-17T22:41:44.318 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0022924849999999999, 2024-09-17T22:41:44.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026407800000000001, 2024-09-17T22:41:44.318 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.318 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.318 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.319 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106968 2024-09-17T22:41:44.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.319 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268131+0000", 2024-09-17T22:41:44.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268131+0000", 2024-09-17T22:41:44.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268132+0000", 2024-09-17T22:41:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.28e-07 2024-09-17T22:41:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268133+0000", 2024-09-17T22:41:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3549999999999999e-06 2024-09-17T22:41:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268136+0000", 2024-09-17T22:41:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1599999999999998e-06 2024-09-17T22:41:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268139+0000", 2024-09-17T22:41:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6979999999999998e-06 2024-09-17T22:41:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106747 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.226143+0000", 2024-09-17T22:41:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.044280818999999999, 2024-09-17T22:41:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044681653000000002, 2024-09-17T22:41:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106747 2024-09-17T22:41:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226143+0000", 2024-09-17T22:41:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226143+0000", 2024-09-17T22:41:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226144+0000", 2024-09-17T22:41:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8700000000000001e-07 2024-09-17T22:41:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226146+0000", 2024-09-17T22:41:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2919999999999998e-06 2024-09-17T22:41:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226152+0000", 2024-09-17T22:41:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7169999999999996e-06 2024-09-17T22:41:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226157+0000", 2024-09-17T22:41:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3530000000000002e-06 2024-09-17T22:41:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106779 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.232154+0000", 2024-09-17T22:41:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.038270072000000002, 2024-09-17T22:41:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038723019999999997, 2024-09-17T22:41:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106779 2024-09-17T22:41:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232154+0000", 2024-09-17T22:41:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232154+0000", 2024-09-17T22:41:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232154+0000", 2024-09-17T22:41:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:41:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232156+0000", 2024-09-17T22:41:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.581e-06 2024-09-17T22:41:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232159+0000", 2024-09-17T22:41:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0070000000000002e-06 2024-09-17T22:41:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232161+0000", 2024-09-17T22:41:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6139999999999999e-06 2024-09-17T22:41:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106811 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.238160+0000", 2024-09-17T22:41:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.032263978999999998, 2024-09-17T22:41:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032745440000000001, 2024-09-17T22:41:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106811 2024-09-17T22:41:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238160+0000", 2024-09-17T22:41:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238160+0000", 2024-09-17T22:41:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238160+0000", 2024-09-17T22:41:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.65e-07 2024-09-17T22:41:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238162+0000", 2024-09-17T22:41:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4899999999999999e-06 2024-09-17T22:41:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238165+0000", 2024-09-17T22:41:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0029999999999999e-06 2024-09-17T22:41:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238168+0000", 2024-09-17T22:41:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.146e-06 2024-09-17T22:41:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106843 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.244135+0000", 2024-09-17T22:41:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.02628904, 2024-09-17T22:41:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026797329000000002, 2024-09-17T22:41:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106843 2024-09-17T22:41:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244135+0000", 2024-09-17T22:41:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244135+0000", 2024-09-17T22:41:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244135+0000", 2024-09-17T22:41:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-09-17T22:41:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244137+0000", 2024-09-17T22:41:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.449e-06 2024-09-17T22:41:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244140+0000", 2024-09-17T22:41:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9399999999999998e-06 2024-09-17T22:41:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244142+0000", 2024-09-17T22:41:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6809999999999998e-06 2024-09-17T22:41:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106875 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.250193+0000", 2024-09-17T22:41:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020231262, 2024-09-17T22:41:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020774279, 2024-09-17T22:41:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106875 2024-09-17T22:41:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250193+0000", 2024-09-17T22:41:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250193+0000", 2024-09-17T22:41:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250193+0000", 2024-09-17T22:41:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9999999999999998e-07 2024-09-17T22:41:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250196+0000", 2024-09-17T22:41:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5809999999999999e-06 2024-09-17T22:41:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250202+0000", 2024-09-17T22:41:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2430000000000002e-06 2024-09-17T22:41:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250208+0000", 2024-09-17T22:41:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3149999999999997e-06 2024-09-17T22:41:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106907 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.254212+0000", 2024-09-17T22:41:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.016211936, 2024-09-17T22:41:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016807083, 2024-09-17T22:41:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106907 2024-09-17T22:41:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254212+0000", 2024-09-17T22:41:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254212+0000", 2024-09-17T22:41:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254212+0000", 2024-09-17T22:41:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:41:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254215+0000", 2024-09-17T22:41:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2900000000000001e-06 2024-09-17T22:41:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254220+0000", 2024-09-17T22:41:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.614e-06 2024-09-17T22:41:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254767+0000", 2024-09-17T22:41:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054656300000000002 2024-09-17T22:41:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106937 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.260148+0000", 2024-09-17T22:41:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.010276260000000001, 2024-09-17T22:41:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010928836000000001, 2024-09-17T22:41:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106937 2024-09-17T22:41:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260148+0000", 2024-09-17T22:41:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260148+0000", 2024-09-17T22:41:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260148+0000", 2024-09-17T22:41:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-09-17T22:41:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260150+0000", 2024-09-17T22:41:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.973e-06 2024-09-17T22:41:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260180+0000", 2024-09-17T22:41:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9496e-05 2024-09-17T22:41:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260187+0000", 2024-09-17T22:41:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1709999999999996e-06 2024-09-17T22:41:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106969 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.268142+0000", 2024-09-17T22:41:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0022820140000000002, 2024-09-17T22:41:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029919959999999998, 2024-09-17T22:41:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106969 2024-09-17T22:41:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268142+0000", 2024-09-17T22:41:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268142+0000", 2024-09-17T22:41:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268142+0000", 2024-09-17T22:41:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.28e-07 2024-09-17T22:41:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268143+0000", 2024-09-17T22:41:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1319999999999999e-06 2024-09-17T22:41:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268146+0000", 2024-09-17T22:41:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9790000000000002e-06 2024-09-17T22:41:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268149+0000", 2024-09-17T22:41:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6730000000000001e-06 2024-09-17T22:41:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106748 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.226163+0000", 2024-09-17T22:41:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.044261314000000003, 2024-09-17T22:41:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045014838000000001, 2024-09-17T22:41:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106748 2024-09-17T22:41:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226163+0000", 2024-09-17T22:41:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226163+0000", 2024-09-17T22:41:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226163+0000", 2024-09-17T22:41:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:41:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226165+0000", 2024-09-17T22:41:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0200000000000001e-06 2024-09-17T22:41:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226171+0000", 2024-09-17T22:41:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5570000000000003e-06 2024-09-17T22:41:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226176+0000", 2024-09-17T22:41:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1780000000000002e-06 2024-09-17T22:41:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106780 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.232164+0000", 2024-09-17T22:41:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.038259561999999997, 2024-09-17T22:41:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039045976000000003, 2024-09-17T22:41:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106780 2024-09-17T22:41:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232164+0000", 2024-09-17T22:41:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232164+0000", 2024-09-17T22:41:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232165+0000", 2024-09-17T22:41:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-09-17T22:41:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232166+0000", 2024-09-17T22:41:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0780000000000001e-06 2024-09-17T22:41:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232169+0000", 2024-09-17T22:41:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.937e-06 2024-09-17T22:41:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232171+0000", 2024-09-17T22:41:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6419999999999999e-06 2024-09-17T22:41:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106812 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.238171+0000", 2024-09-17T22:41:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.032253113999999999, 2024-09-17T22:41:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033066326, 2024-09-17T22:41:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106812 2024-09-17T22:41:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238171+0000", 2024-09-17T22:41:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238171+0000", 2024-09-17T22:41:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238171+0000", 2024-09-17T22:41:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.28e-07 2024-09-17T22:41:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238172+0000", 2024-09-17T22:41:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.145e-06 2024-09-17T22:41:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238175+0000", 2024-09-17T22:41:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.021e-06 2024-09-17T22:41:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238178+0000", 2024-09-17T22:41:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5299999999999999e-06 2024-09-17T22:41:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106844 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.244158+0000", 2024-09-17T22:41:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.026265892999999998, 2024-09-17T22:41:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027105536, 2024-09-17T22:41:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106844 2024-09-17T22:41:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244158+0000", 2024-09-17T22:41:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244158+0000", 2024-09-17T22:41:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244158+0000", 2024-09-17T22:41:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.96e-07 2024-09-17T22:41:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244160+0000", 2024-09-17T22:41:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7039999999999999e-06 2024-09-17T22:41:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244165+0000", 2024-09-17T22:41:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4159999999999997e-06 2024-09-17T22:41:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244168+0000", 2024-09-17T22:41:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3139999999999999e-06 2024-09-17T22:41:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106876 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.250215+0000", 2024-09-17T22:41:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020208667999999999, 2024-09-17T22:41:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021098206000000001, 2024-09-17T22:41:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106876 2024-09-17T22:41:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250215+0000", 2024-09-17T22:41:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250215+0000", 2024-09-17T22:41:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250216+0000", 2024-09-17T22:41:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3499999999999996e-07 2024-09-17T22:41:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250218+0000", 2024-09-17T22:41:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3410000000000001e-06 2024-09-17T22:41:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250225+0000", 2024-09-17T22:41:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4880000000000004e-06 2024-09-17T22:41:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250231+0000", 2024-09-17T22:41:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7020000000000004e-06 2024-09-17T22:41:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106908 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.254781+0000", 2024-09-17T22:41:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015642527, 2024-09-17T22:41:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016584840999999999, 2024-09-17T22:41:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106908 2024-09-17T22:41:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254781+0000", 2024-09-17T22:41:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254781+0000", 2024-09-17T22:41:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254782+0000", 2024-09-17T22:41:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4399999999999999e-07 2024-09-17T22:41:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254786+0000", 2024-09-17T22:41:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1860000000000002e-06 2024-09-17T22:41:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254796+0000", 2024-09-17T22:41:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0083e-05 2024-09-17T22:41:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254804+0000", 2024-09-17T22:41:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3660000000000003e-06 2024-09-17T22:41:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106938 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.260225+0000", 2024-09-17T22:41:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.010199107000000001, 2024-09-17T22:41:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011195785999999999, 2024-09-17T22:41:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106938 2024-09-17T22:41:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260225+0000", 2024-09-17T22:41:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260225+0000", 2024-09-17T22:41:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260226+0000", 2024-09-17T22:41:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7000000000000004e-07 2024-09-17T22:41:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260228+0000", 2024-09-17T22:41:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8729999999999999e-06 2024-09-17T22:41:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260236+0000", 2024-09-17T22:41:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3379999999999999e-06 2024-09-17T22:41:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260241+0000", 2024-09-17T22:41:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6670000000000001e-06 2024-09-17T22:41:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106970 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.268152+0000", 2024-09-17T22:41:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0022721009999999999, 2024-09-17T22:41:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033206490000000002, 2024-09-17T22:41:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106970 2024-09-17T22:41:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268152+0000", 2024-09-17T22:41:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268152+0000", 2024-09-17T22:41:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268152+0000", 2024-09-17T22:41:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3799999999999999e-07 2024-09-17T22:41:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268153+0000", 2024-09-17T22:41:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1170000000000001e-06 2024-09-17T22:41:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268156+0000", 2024-09-17T22:41:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.892e-06 2024-09-17T22:41:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268159+0000", 2024-09-17T22:41:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5890000000000001e-06 2024-09-17T22:41:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106749 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.226181+0000", 2024-09-17T22:41:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.044242521, 2024-09-17T22:41:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045320216000000003, 2024-09-17T22:41:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106749 2024-09-17T22:41:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226181+0000", 2024-09-17T22:41:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226181+0000", 2024-09-17T22:41:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226182+0000", 2024-09-17T22:41:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2000000000000001e-07 2024-09-17T22:41:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226184+0000", 2024-09-17T22:41:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.841e-06 2024-09-17T22:41:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226189+0000", 2024-09-17T22:41:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7849999999999997e-06 2024-09-17T22:41:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226194+0000", 2024-09-17T22:41:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.853e-06 2024-09-17T22:41:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106781 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.232174+0000", 2024-09-17T22:41:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.038249765999999998, 2024-09-17T22:41:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039355257999999997, 2024-09-17T22:41:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106781 2024-09-17T22:41:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232174+0000", 2024-09-17T22:41:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232174+0000", 2024-09-17T22:41:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232174+0000", 2024-09-17T22:41:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:41:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232176+0000", 2024-09-17T22:41:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2270000000000001e-06 2024-09-17T22:41:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232179+0000", 2024-09-17T22:41:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.892e-06 2024-09-17T22:41:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232181+0000", 2024-09-17T22:41:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5519999999999999e-06 2024-09-17T22:41:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106813 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.238181+0000", 2024-09-17T22:41:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.032243168000000003, 2024-09-17T22:41:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033375228999999999, 2024-09-17T22:41:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106813 2024-09-17T22:41:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238181+0000", 2024-09-17T22:41:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238181+0000", 2024-09-17T22:41:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238181+0000", 2024-09-17T22:41:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:41:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238182+0000", 2024-09-17T22:41:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1739999999999999e-06 2024-09-17T22:41:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238185+0000", 2024-09-17T22:41:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9359999999999999e-06 2024-09-17T22:41:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238188+0000", 2024-09-17T22:41:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5229999999999998e-06 2024-09-17T22:41:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106845 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.244172+0000", 2024-09-17T22:41:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.02625216, 2024-09-17T22:41:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027423280000000001, 2024-09-17T22:41:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106845 2024-09-17T22:41:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244172+0000", 2024-09-17T22:41:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244172+0000", 2024-09-17T22:41:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244172+0000", 2024-09-17T22:41:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2600000000000001e-07 2024-09-17T22:41:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244173+0000", 2024-09-17T22:41:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.252e-06 2024-09-17T22:41:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244176+0000", 2024-09-17T22:41:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0570000000000001e-06 2024-09-17T22:41:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244179+0000", 2024-09-17T22:41:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0419999999999998e-06 2024-09-17T22:41:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106877 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.250238+0000", 2024-09-17T22:41:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020186065999999999, 2024-09-17T22:41:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021410202999999999, 2024-09-17T22:41:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106877 2024-09-17T22:41:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250238+0000", 2024-09-17T22:41:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250238+0000", 2024-09-17T22:41:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250238+0000", 2024-09-17T22:41:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:41:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250241+0000", 2024-09-17T22:41:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2970000000000002e-06 2024-09-17T22:41:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250247+0000", 2024-09-17T22:41:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.392e-06 2024-09-17T22:41:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250252+0000", 2024-09-17T22:41:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0980000000000001e-06 2024-09-17T22:41:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106909 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.254824+0000", 2024-09-17T22:41:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015599955, 2024-09-17T22:41:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016876563000000001, 2024-09-17T22:41:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106909 2024-09-17T22:41:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254824+0000", 2024-09-17T22:41:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254824+0000", 2024-09-17T22:41:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254825+0000", 2024-09-17T22:41:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0800000000000004e-07 2024-09-17T22:41:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254827+0000", 2024-09-17T22:41:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-09-17T22:41:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254835+0000", 2024-09-17T22:41:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2899999999999997e-06 2024-09-17T22:41:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254843+0000", 2024-09-17T22:41:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1049999999999995e-06 2024-09-17T22:41:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106939 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.260247+0000", 2024-09-17T22:41:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.010176727, 2024-09-17T22:41:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011505471, 2024-09-17T22:41:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106939 2024-09-17T22:41:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260247+0000", 2024-09-17T22:41:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260247+0000", 2024-09-17T22:41:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260248+0000", 2024-09-17T22:41:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4999999999999999e-07 2024-09-17T22:41:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260254+0000", 2024-09-17T22:41:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1970000000000001e-06 2024-09-17T22:41:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260257+0000", 2024-09-17T22:41:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5810000000000001e-06 2024-09-17T22:41:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260260+0000", 2024-09-17T22:41:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8279999999999999e-06 2024-09-17T22:41:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106971 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.268162+0000", 2024-09-17T22:41:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0022622139999999998, 2024-09-17T22:41:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036372150000000001, 2024-09-17T22:41:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106971 2024-09-17T22:41:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268162+0000", 2024-09-17T22:41:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268162+0000", 2024-09-17T22:41:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268162+0000", 2024-09-17T22:41:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:41:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268164+0000", 2024-09-17T22:41:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4559999999999999e-06 2024-09-17T22:41:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268167+0000", 2024-09-17T22:41:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9720000000000001e-06 2024-09-17T22:41:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268169+0000", 2024-09-17T22:41:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4889999999999998e-06 2024-09-17T22:41:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106750 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.226200+0000", 2024-09-17T22:41:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.044223998, 2024-09-17T22:41:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045627110999999998, 2024-09-17T22:41:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106750 2024-09-17T22:41:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226200+0000", 2024-09-17T22:41:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226200+0000", 2024-09-17T22:41:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226200+0000", 2024-09-17T22:41:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.27e-07 2024-09-17T22:41:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226206+0000", 2024-09-17T22:41:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8629999999999996e-06 2024-09-17T22:41:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226212+0000", 2024-09-17T22:41:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2670000000000003e-06 2024-09-17T22:41:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226218+0000", 2024-09-17T22:41:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4960000000000004e-06 2024-09-17T22:41:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106782 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.232184+0000", 2024-09-17T22:41:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.038239810999999999, 2024-09-17T22:41:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039670416999999999, 2024-09-17T22:41:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106782 2024-09-17T22:41:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232184+0000", 2024-09-17T22:41:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232184+0000", 2024-09-17T22:41:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232184+0000", 2024-09-17T22:41:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.16e-07 2024-09-17T22:41:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232185+0000", 2024-09-17T22:41:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0699999999999999e-06 2024-09-17T22:41:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232188+0000", 2024-09-17T22:41:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.01e-06 2024-09-17T22:41:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232191+0000", 2024-09-17T22:41:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6919999999999998e-06 2024-09-17T22:41:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106814 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.238191+0000", 2024-09-17T22:41:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.032233352, 2024-09-17T22:41:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033765575999999999, 2024-09-17T22:41:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106814 2024-09-17T22:41:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238191+0000", 2024-09-17T22:41:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238191+0000", 2024-09-17T22:41:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238191+0000", 2024-09-17T22:41:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9400000000000001e-07 2024-09-17T22:41:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238192+0000", 2024-09-17T22:41:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1510000000000001e-06 2024-09-17T22:41:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238196+0000", 2024-09-17T22:41:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7500000000000001e-06 2024-09-17T22:41:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238199+0000", 2024-09-17T22:41:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6809999999999998e-06 2024-09-17T22:41:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106846 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.244182+0000", 2024-09-17T22:41:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.026241704000000001, 2024-09-17T22:41:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027828604, 2024-09-17T22:41:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106846 2024-09-17T22:41:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244182+0000", 2024-09-17T22:41:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244182+0000", 2024-09-17T22:41:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244182+0000", 2024-09-17T22:41:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9500000000000001e-07 2024-09-17T22:41:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244183+0000", 2024-09-17T22:41:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001e-06 2024-09-17T22:41:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244186+0000", 2024-09-17T22:41:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8399999999999999e-06 2024-09-17T22:41:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244189+0000", 2024-09-17T22:41:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5979999999999999e-06 2024-09-17T22:41:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106878 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.250260+0000", 2024-09-17T22:41:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020164159000000001, 2024-09-17T22:41:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021815771000000001, 2024-09-17T22:41:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106878 2024-09-17T22:41:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250260+0000", 2024-09-17T22:41:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250260+0000", 2024-09-17T22:41:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250260+0000", 2024-09-17T22:41:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-09-17T22:41:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250262+0000", 2024-09-17T22:41:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1409999999999999e-06 2024-09-17T22:41:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250270+0000", 2024-09-17T22:41:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6290000000000001e-06 2024-09-17T22:41:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250277+0000", 2024-09-17T22:41:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5189999999999998e-06 2024-09-17T22:41:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106910 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.254850+0000", 2024-09-17T22:41:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015574298, 2024-09-17T22:41:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017255626999999999, 2024-09-17T22:41:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106910 2024-09-17T22:41:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254850+0000", 2024-09-17T22:41:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254850+0000", 2024-09-17T22:41:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254850+0000", 2024-09-17T22:41:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3000000000000001e-07 2024-09-17T22:41:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254852+0000", 2024-09-17T22:41:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1959999999999998e-06 2024-09-17T22:41:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254859+0000", 2024-09-17T22:41:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2979999999999997e-06 2024-09-17T22:41:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254864+0000", 2024-09-17T22:41:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3369999999999999e-06 2024-09-17T22:41:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106940 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.260263+0000", 2024-09-17T22:41:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.010160567000000001, 2024-09-17T22:41:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011868896, 2024-09-17T22:41:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106940 2024-09-17T22:41:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260263+0000", 2024-09-17T22:41:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260263+0000", 2024-09-17T22:41:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262050+0000", 2024-09-17T22:41:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017868770000000001 2024-09-17T22:41:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262054+0000", 2024-09-17T22:41:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0740000000000003e-06 2024-09-17T22:41:44.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262063+0000", 2024-09-17T22:41:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5359999999999992e-06 2024-09-17T22:41:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262069+0000", 2024-09-17T22:41:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7509999999999997e-06 2024-09-17T22:41:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106972 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.268181+0000", 2024-09-17T22:41:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.002243278, 2024-09-17T22:41:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003982167, 2024-09-17T22:41:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106972 2024-09-17T22:41:44.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268181+0000", 2024-09-17T22:41:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268181+0000", 2024-09-17T22:41:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268181+0000", 2024-09-17T22:41:44.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5400000000000002e-07 2024-09-17T22:41:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268183+0000", 2024-09-17T22:41:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6810000000000001e-06 2024-09-17T22:41:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268186+0000", 2024-09-17T22:41:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4460000000000001e-06 2024-09-17T22:41:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268189+0000", 2024-09-17T22:41:44.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7489999999999999e-06 2024-09-17T22:41:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106751 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.226224+0000", 2024-09-17T22:41:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.044199745999999998, 2024-09-17T22:41:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045980198999999999, 2024-09-17T22:41:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106751 2024-09-17T22:41:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226224+0000", 2024-09-17T22:41:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226224+0000", 2024-09-17T22:41:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226225+0000", 2024-09-17T22:41:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8599999999999999e-07 2024-09-17T22:41:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226227+0000", 2024-09-17T22:41:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.305e-06 2024-09-17T22:41:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226240+0000", 2024-09-17T22:41:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2870999999999999e-05 2024-09-17T22:41:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226248+0000", 2024-09-17T22:41:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2179999999999999e-06 2024-09-17T22:41:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106783 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.435 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.232194+0000", 2024-09-17T22:41:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.038229805999999998, 2024-09-17T22:41:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040066081000000003, 2024-09-17T22:41:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106783 2024-09-17T22:41:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232194+0000", 2024-09-17T22:41:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232194+0000", 2024-09-17T22:41:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232194+0000", 2024-09-17T22:41:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-09-17T22:41:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232195+0000", 2024-09-17T22:41:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0809999999999999e-06 2024-09-17T22:41:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232198+0000", 2024-09-17T22:41:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8729999999999999e-06 2024-09-17T22:41:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232201+0000", 2024-09-17T22:41:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7980000000000001e-06 2024-09-17T22:41:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.438 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106815 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.238201+0000", 2024-09-17T22:41:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.032222608999999999, 2024-09-17T22:41:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034114819999999997, 2024-09-17T22:41:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106815 2024-09-17T22:41:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.439 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238201+0000", 2024-09-17T22:41:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238201+0000", 2024-09-17T22:41:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238202+0000", 2024-09-17T22:41:44.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:41:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238203+0000", 2024-09-17T22:41:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9099999999999991e-07 2024-09-17T22:41:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238206+0000", 2024-09-17T22:41:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.687e-06 2024-09-17T22:41:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238209+0000", 2024-09-17T22:41:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:41:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106847 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.244192+0000", 2024-09-17T22:41:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.026232338000000001, 2024-09-17T22:41:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028179808000000001, 2024-09-17T22:41:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.442 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106847 2024-09-17T22:41:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244192+0000", 2024-09-17T22:41:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244192+0000", 2024-09-17T22:41:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244192+0000", 2024-09-17T22:41:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9999999999999999e-07 2024-09-17T22:41:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244193+0000", 2024-09-17T22:41:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1379999999999999e-06 2024-09-17T22:41:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244196+0000", 2024-09-17T22:41:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8090000000000001e-06 2024-09-17T22:41:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244200+0000", 2024-09-17T22:41:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8e-06 2024-09-17T22:41:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.445 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106879 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.250283+0000", 2024-09-17T22:41:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020140754, 2024-09-17T22:41:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022120114999999999, 2024-09-17T22:41:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106879 2024-09-17T22:41:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250283+0000", 2024-09-17T22:41:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250283+0000", 2024-09-17T22:41:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250284+0000", 2024-09-17T22:41:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:41:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250290+0000", 2024-09-17T22:41:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4590000000000003e-06 2024-09-17T22:41:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250296+0000", 2024-09-17T22:41:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9479999999999996e-06 2024-09-17T22:41:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250302+0000", 2024-09-17T22:41:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9950000000000002e-06 2024-09-17T22:41:44.448 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106911 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.254870+0000", 2024-09-17T22:41:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015554414000000001, 2024-09-17T22:41:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01756307, 2024-09-17T22:41:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106911 2024-09-17T22:41:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254870+0000", 2024-09-17T22:41:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254870+0000", 2024-09-17T22:41:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254870+0000", 2024-09-17T22:41:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5199999999999998e-07 2024-09-17T22:41:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254872+0000", 2024-09-17T22:41:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.886e-06 2024-09-17T22:41:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254884+0000", 2024-09-17T22:41:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2393e-05 2024-09-17T22:41:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254890+0000", 2024-09-17T22:41:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.195e-06 2024-09-17T22:41:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106941 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.262078+0000", 2024-09-17T22:41:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0083460010000000005, 2024-09-17T22:41:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010381069999999999, 2024-09-17T22:41:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.452 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106941 2024-09-17T22:41:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262078+0000", 2024-09-17T22:41:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262078+0000", 2024-09-17T22:41:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262078+0000", 2024-09-17T22:41:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-17T22:41:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262080+0000", 2024-09-17T22:41:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.635e-06 2024-09-17T22:41:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262084+0000", 2024-09-17T22:41:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6770000000000001e-06 2024-09-17T22:41:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262086+0000", 2024-09-17T22:41:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9189999999999999e-06 2024-09-17T22:41:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106973 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.268192+0000", 2024-09-17T22:41:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0022315550000000001, 2024-09-17T22:41:44.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0043052539999999997, 2024-09-17T22:41:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106973 2024-09-17T22:41:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268192+0000", 2024-09-17T22:41:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268192+0000", 2024-09-17T22:41:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268193+0000", 2024-09-17T22:41:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0699999999999999e-07 2024-09-17T22:41:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268194+0000", 2024-09-17T22:41:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.201e-06 2024-09-17T22:41:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268197+0000", 2024-09-17T22:41:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8959999999999999e-06 2024-09-17T22:41:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268199+0000", 2024-09-17T22:41:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7250000000000002e-06 2024-09-17T22:41:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.458 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106752 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.226277+0000", 2024-09-17T22:41:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.04414742, 2024-09-17T22:41:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046274049999999997, 2024-09-17T22:41:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106752 2024-09-17T22:41:44.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226277+0000", 2024-09-17T22:41:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226277+0000", 2024-09-17T22:41:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226277+0000", 2024-09-17T22:41:44.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1399999999999997e-07 2024-09-17T22:41:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226280+0000", 2024-09-17T22:41:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4949999999999998e-06 2024-09-17T22:41:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226286+0000", 2024-09-17T22:41:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6780000000000003e-06 2024-09-17T22:41:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226291+0000", 2024-09-17T22:41:44.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0769999999999998e-06 2024-09-17T22:41:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106784 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.232205+0000", 2024-09-17T22:41:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.038219347000000001, 2024-09-17T22:41:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040404105000000003, 2024-09-17T22:41:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.462 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106784 2024-09-17T22:41:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232205+0000", 2024-09-17T22:41:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.232205+0000", 2024-09-17T22:41:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234051+0000", 2024-09-17T22:41:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018467900000000001 2024-09-17T22:41:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234055+0000", 2024-09-17T22:41:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3179999999999998e-06 2024-09-17T22:41:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234061+0000", 2024-09-17T22:41:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.579e-06 2024-09-17T22:41:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234066+0000", 2024-09-17T22:41:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2989999999999998e-06 2024-09-17T22:41:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106816 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.238212+0000", 2024-09-17T22:41:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.032212076999999999, 2024-09-17T22:41:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034449436, 2024-09-17T22:41:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.465 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106816 2024-09-17T22:41:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238212+0000", 2024-09-17T22:41:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.238212+0000", 2024-09-17T22:41:44.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240049+0000", 2024-09-17T22:41:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018367050000000001 2024-09-17T22:41:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240052+0000", 2024-09-17T22:41:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.444e-06 2024-09-17T22:41:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240060+0000", 2024-09-17T22:41:44.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4760000000000001e-06 2024-09-17T22:41:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240064+0000", 2024-09-17T22:41:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9529999999999998e-06 2024-09-17T22:41:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106848 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.244203+0000", 2024-09-17T22:41:44.468 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.026221451999999999, 2024-09-17T22:41:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028503923, 2024-09-17T22:41:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106848 2024-09-17T22:41:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244203+0000", 2024-09-17T22:41:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244203+0000", 2024-09-17T22:41:44.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246073+0000", 2024-09-17T22:41:44.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001870626 2024-09-17T22:41:44.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246077+0000", 2024-09-17T22:41:44.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7139999999999999e-06 2024-09-17T22:41:44.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246114+0000", 2024-09-17T22:41:44.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7407000000000003e-05 2024-09-17T22:41:44.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246120+0000", 2024-09-17T22:41:44.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5090000000000001e-06 2024-09-17T22:41:44.471 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.471 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.471 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.472 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106880 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.472 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.250317+0000", 2024-09-17T22:41:44.472 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020106914, 2024-09-17T22:41:44.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022417323999999999, 2024-09-17T22:41:44.472 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.472 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.472 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.472 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.472 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.472 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.472 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106880 2024-09-17T22:41:44.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.473 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250317+0000", 2024-09-17T22:41:44.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250317+0000", 2024-09-17T22:41:44.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250318+0000", 2024-09-17T22:41:44.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8500000000000002e-07 2024-09-17T22:41:44.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250320+0000", 2024-09-17T22:41:44.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1509999999999998e-06 2024-09-17T22:41:44.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250328+0000", 2024-09-17T22:41:44.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8140000000000001e-06 2024-09-17T22:41:44.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250333+0000", 2024-09-17T22:41:44.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3079999999999998e-06 2024-09-17T22:41:44.475 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.475 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.475 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.475 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106912 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.475 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.254897+0000", 2024-09-17T22:41:44.475 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015527269, 2024-09-17T22:41:44.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017866908000000001, 2024-09-17T22:41:44.475 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.475 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.475 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.475 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.476 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.476 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.476 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106912 2024-09-17T22:41:44.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.476 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254897+0000", 2024-09-17T22:41:44.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254897+0000", 2024-09-17T22:41:44.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254897+0000", 2024-09-17T22:41:44.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.77e-07 2024-09-17T22:41:44.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254899+0000", 2024-09-17T22:41:44.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.103e-06 2024-09-17T22:41:44.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254906+0000", 2024-09-17T22:41:44.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3110000000000002e-06 2024-09-17T22:41:44.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254911+0000", 2024-09-17T22:41:44.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0189999999999997e-06 2024-09-17T22:41:44.478 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.478 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.478 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.478 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106942 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.478 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.262090+0000", 2024-09-17T22:41:44.478 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0083337979999999999, 2024-09-17T22:41:44.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010722376, 2024-09-17T22:41:44.479 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.479 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.479 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.479 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.479 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.479 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.479 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106942 2024-09-17T22:41:44.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.479 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262090+0000", 2024-09-17T22:41:44.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262090+0000", 2024-09-17T22:41:44.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262090+0000", 2024-09-17T22:41:44.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.17e-07 2024-09-17T22:41:44.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262092+0000", 2024-09-17T22:41:44.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.195e-06 2024-09-17T22:41:44.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262095+0000", 2024-09-17T22:41:44.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0860000000000002e-06 2024-09-17T22:41:44.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262098+0000", 2024-09-17T22:41:44.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0139999999999999e-06 2024-09-17T22:41:44.481 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.481 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.481 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.481 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106974 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.481 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.268202+0000", 2024-09-17T22:41:44.482 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0022215009999999999, 2024-09-17T22:41:44.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0046636330000000004, 2024-09-17T22:41:44.482 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.482 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.482 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.482 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.482 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.482 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.482 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106974 2024-09-17T22:41:44.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.482 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268202+0000", 2024-09-17T22:41:44.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268202+0000", 2024-09-17T22:41:44.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268203+0000", 2024-09-17T22:41:44.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9700000000000003e-07 2024-09-17T22:41:44.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268207+0000", 2024-09-17T22:41:44.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9230000000000004e-06 2024-09-17T22:41:44.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268210+0000", 2024-09-17T22:41:44.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.157e-06 2024-09-17T22:41:44.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268213+0000", 2024-09-17T22:41:44.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.836e-06 2024-09-17T22:41:44.484 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.485 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.485 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.485 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106753 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.485 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.226298+0000", 2024-09-17T22:41:44.485 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.044125831999999997, 2024-09-17T22:41:44.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046623038999999998, 2024-09-17T22:41:44.485 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.485 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.485 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.485 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.485 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.485 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.486 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106753 2024-09-17T22:41:44.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.486 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226298+0000", 2024-09-17T22:41:44.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226298+0000", 2024-09-17T22:41:44.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226299+0000", 2024-09-17T22:41:44.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5900000000000002e-07 2024-09-17T22:41:44.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226301+0000", 2024-09-17T22:41:44.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1059999999999998e-06 2024-09-17T22:41:44.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226307+0000", 2024-09-17T22:41:44.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8429999999999998e-06 2024-09-17T22:41:44.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.226311+0000", 2024-09-17T22:41:44.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.78e-06 2024-09-17T22:41:44.488 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.488 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.488 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.488 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106785 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.488 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.234074+0000", 2024-09-17T22:41:44.488 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.036349528999999998, 2024-09-17T22:41:44.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038897473000000002, 2024-09-17T22:41:44.488 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.488 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.489 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.489 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.489 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.489 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.489 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106785 2024-09-17T22:41:44.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.489 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234074+0000", 2024-09-17T22:41:44.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234074+0000", 2024-09-17T22:41:44.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234075+0000", 2024-09-17T22:41:44.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9299999999999999e-07 2024-09-17T22:41:44.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234096+0000", 2024-09-17T22:41:44.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1228000000000001e-05 2024-09-17T22:41:44.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234100+0000", 2024-09-17T22:41:44.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5029999999999999e-06 2024-09-17T22:41:44.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234104+0000", 2024-09-17T22:41:44.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2600000000000001e-06 2024-09-17T22:41:44.491 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.491 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.491 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.491 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106817 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.491 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.240076+0000", 2024-09-17T22:41:44.491 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.030348040999999999, 2024-09-17T22:41:44.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032931566000000002, 2024-09-17T22:41:44.492 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.492 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.492 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.492 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.492 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.492 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.492 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106817 2024-09-17T22:41:44.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.492 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240076+0000", 2024-09-17T22:41:44.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240076+0000", 2024-09-17T22:41:44.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240076+0000", 2024-09-17T22:41:44.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.79e-07 2024-09-17T22:41:44.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240078+0000", 2024-09-17T22:41:44.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.491e-06 2024-09-17T22:41:44.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240081+0000", 2024-09-17T22:41:44.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5619999999999999e-06 2024-09-17T22:41:44.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240084+0000", 2024-09-17T22:41:44.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8660000000000002e-06 2024-09-17T22:41:44.494 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.495 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.495 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.495 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106849 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.495 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.246129+0000", 2024-09-17T22:41:44.495 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.024295135999999998, 2024-09-17T22:41:44.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026905661000000001, 2024-09-17T22:41:44.495 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.495 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.495 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.495 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.495 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.495 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.495 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106849 2024-09-17T22:41:44.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.496 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246129+0000", 2024-09-17T22:41:44.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246129+0000", 2024-09-17T22:41:44.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246129+0000", 2024-09-17T22:41:44.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0899999999999997e-07 2024-09-17T22:41:44.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246131+0000", 2024-09-17T22:41:44.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5850000000000001e-06 2024-09-17T22:41:44.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246134+0000", 2024-09-17T22:41:44.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7050000000000001e-06 2024-09-17T22:41:44.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246137+0000", 2024-09-17T22:41:44.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.875e-06 2024-09-17T22:41:44.498 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.498 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.498 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.498 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106881 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.498 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.250339+0000", 2024-09-17T22:41:44.498 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020084980999999998, 2024-09-17T22:41:44.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022733692, 2024-09-17T22:41:44.498 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.498 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.498 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.498 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.499 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.499 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.499 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106881 2024-09-17T22:41:44.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.499 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250339+0000", 2024-09-17T22:41:44.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250339+0000", 2024-09-17T22:41:44.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250339+0000", 2024-09-17T22:41:44.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:41:44.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250341+0000", 2024-09-17T22:41:44.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9149999999999999e-06 2024-09-17T22:41:44.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250347+0000", 2024-09-17T22:41:44.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8710000000000002e-06 2024-09-17T22:41:44.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250352+0000", 2024-09-17T22:41:44.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8230000000000002e-06 2024-09-17T22:41:44.501 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.501 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.501 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.501 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106913 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.501 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.254916+0000", 2024-09-17T22:41:44.501 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015507953, 2024-09-17T22:41:44.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018203652000000001, 2024-09-17T22:41:44.502 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.502 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.502 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.502 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.502 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.502 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.502 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106913 2024-09-17T22:41:44.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.502 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254916+0000", 2024-09-17T22:41:44.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254916+0000", 2024-09-17T22:41:44.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254916+0000", 2024-09-17T22:41:44.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:41:44.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254918+0000", 2024-09-17T22:41:44.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.784e-06 2024-09-17T22:41:44.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254924+0000", 2024-09-17T22:41:44.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3269999999999997e-06 2024-09-17T22:41:44.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254929+0000", 2024-09-17T22:41:44.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.865e-06 2024-09-17T22:41:44.504 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.504 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.504 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.504 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106943 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.505 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.262101+0000", 2024-09-17T22:41:44.505 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0083233119999999994, 2024-09-17T22:41:44.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011075382, 2024-09-17T22:41:44.505 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.505 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.505 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.505 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.505 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.505 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.505 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106943 2024-09-17T22:41:44.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.505 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262101+0000", 2024-09-17T22:41:44.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262101+0000", 2024-09-17T22:41:44.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262101+0000", 2024-09-17T22:41:44.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:41:44.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262102+0000", 2024-09-17T22:41:44.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.497e-06 2024-09-17T22:41:44.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262105+0000", 2024-09-17T22:41:44.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9670000000000001e-06 2024-09-17T22:41:44.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262108+0000", 2024-09-17T22:41:44.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6970000000000002e-06 2024-09-17T22:41:44.508 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.508 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.508 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.508 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106975 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.508 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.268216+0000", 2024-09-17T22:41:44.508 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0022080950000000002, 2024-09-17T22:41:44.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050136829999999997, 2024-09-17T22:41:44.508 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.508 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.508 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.508 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.509 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.509 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.509 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106975 2024-09-17T22:41:44.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.509 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268216+0000", 2024-09-17T22:41:44.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268216+0000", 2024-09-17T22:41:44.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268216+0000", 2024-09-17T22:41:44.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3699999999999999e-07 2024-09-17T22:41:44.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268217+0000", 2024-09-17T22:41:44.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2249999999999999e-06 2024-09-17T22:41:44.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268220+0000", 2024-09-17T22:41:44.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9699999999999999e-06 2024-09-17T22:41:44.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268223+0000", 2024-09-17T22:41:44.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.818e-06 2024-09-17T22:41:44.511 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.511 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.511 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.511 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106786 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.511 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.234110+0000", 2024-09-17T22:41:44.511 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.036314218000000002, 2024-09-17T22:41:44.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039176018999999999, 2024-09-17T22:41:44.512 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.512 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.512 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.512 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.512 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.512 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.512 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106786 2024-09-17T22:41:44.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.512 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234110+0000", 2024-09-17T22:41:44.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234110+0000", 2024-09-17T22:41:44.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234110+0000", 2024-09-17T22:41:44.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3799999999999997e-07 2024-09-17T22:41:44.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234112+0000", 2024-09-17T22:41:44.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.739e-06 2024-09-17T22:41:44.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234121+0000", 2024-09-17T22:41:44.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3000000000000007e-06 2024-09-17T22:41:44.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234126+0000", 2024-09-17T22:41:44.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6249999999999998e-06 2024-09-17T22:41:44.514 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.514 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.514 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.515 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106818 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.515 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.240088+0000", 2024-09-17T22:41:44.515 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.030336170999999999, 2024-09-17T22:41:44.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033251279000000002, 2024-09-17T22:41:44.515 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.515 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.515 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.515 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.515 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.515 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.515 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106818 2024-09-17T22:41:44.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.516 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240088+0000", 2024-09-17T22:41:44.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240088+0000", 2024-09-17T22:41:44.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240088+0000", 2024-09-17T22:41:44.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3900000000000001e-07 2024-09-17T22:41:44.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240089+0000", 2024-09-17T22:41:44.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1599999999999999e-06 2024-09-17T22:41:44.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240092+0000", 2024-09-17T22:41:44.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0520000000000002e-06 2024-09-17T22:41:44.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240095+0000", 2024-09-17T22:41:44.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6079999999999998e-06 2024-09-17T22:41:44.518 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.518 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.518 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.518 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106850 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.518 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.246141+0000", 2024-09-17T22:41:44.519 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.024282967999999999, 2024-09-17T22:41:44.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027254235000000002, 2024-09-17T22:41:44.519 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.519 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.519 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.519 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.519 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.519 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.519 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106850 2024-09-17T22:41:44.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.519 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246141+0000", 2024-09-17T22:41:44.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246141+0000", 2024-09-17T22:41:44.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246141+0000", 2024-09-17T22:41:44.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0499999999999999e-07 2024-09-17T22:41:44.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246142+0000", 2024-09-17T22:41:44.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1710000000000001e-06 2024-09-17T22:41:44.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246146+0000", 2024-09-17T22:41:44.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0299999999999998e-06 2024-09-17T22:41:44.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246148+0000", 2024-09-17T22:41:44.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.605e-06 2024-09-17T22:41:44.522 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.522 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.522 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.522 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106882 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.522 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.250359+0000", 2024-09-17T22:41:44.522 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020065383999999999, 2024-09-17T22:41:44.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023089228, 2024-09-17T22:41:44.522 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.522 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.522 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.522 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.523 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.523 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.523 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106882 2024-09-17T22:41:44.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.523 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250359+0000", 2024-09-17T22:41:44.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250359+0000", 2024-09-17T22:41:44.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250359+0000", 2024-09-17T22:41:44.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:41:44.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250361+0000", 2024-09-17T22:41:44.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.951e-06 2024-09-17T22:41:44.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250368+0000", 2024-09-17T22:41:44.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9e-06 2024-09-17T22:41:44.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250373+0000", 2024-09-17T22:41:44.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-09-17T22:41:44.525 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.525 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.525 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.526 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106944 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.526 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.262111+0000", 2024-09-17T22:41:44.526 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0083125860000000003, 2024-09-17T22:41:44.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011411696000000001, 2024-09-17T22:41:44.526 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.526 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.526 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.526 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106944 2024-09-17T22:41:44.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.527 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262111+0000", 2024-09-17T22:41:44.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262111+0000", 2024-09-17T22:41:44.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262112+0000", 2024-09-17T22:41:44.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.29e-07 2024-09-17T22:41:44.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262113+0000", 2024-09-17T22:41:44.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2270000000000001e-06 2024-09-17T22:41:44.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262116+0000", 2024-09-17T22:41:44.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.954e-06 2024-09-17T22:41:44.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262118+0000", 2024-09-17T22:41:44.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6960000000000001e-06 2024-09-17T22:41:44.529 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.529 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.529 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.529 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106976 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.529 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.268227+0000", 2024-09-17T22:41:44.529 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0021972039999999999, 2024-09-17T22:41:44.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053246550000000002, 2024-09-17T22:41:44.530 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.530 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.530 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.530 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.530 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.530 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.530 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106976 2024-09-17T22:41:44.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.530 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268227+0000", 2024-09-17T22:41:44.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268227+0000", 2024-09-17T22:41:44.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268227+0000", 2024-09-17T22:41:44.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0599999999999999e-07 2024-09-17T22:41:44.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268228+0000", 2024-09-17T22:41:44.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.062e-06 2024-09-17T22:41:44.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268231+0000", 2024-09-17T22:41:44.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9830000000000001e-06 2024-09-17T22:41:44.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268234+0000", 2024-09-17T22:41:44.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.74e-06 2024-09-17T22:41:44.533 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.533 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.533 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.533 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106787 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.533 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.234140+0000", 2024-09-17T22:41:44.533 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.036284467000000001, 2024-09-17T22:41:44.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.03946289, 2024-09-17T22:41:44.533 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.533 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.533 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.534 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.534 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.534 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.534 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106787 2024-09-17T22:41:44.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.534 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234140+0000", 2024-09-17T22:41:44.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234140+0000", 2024-09-17T22:41:44.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234140+0000", 2024-09-17T22:41:44.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6899999999999998e-07 2024-09-17T22:41:44.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234142+0000", 2024-09-17T22:41:44.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-09-17T22:41:44.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234148+0000", 2024-09-17T22:41:44.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1539999999999999e-06 2024-09-17T22:41:44.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234153+0000", 2024-09-17T22:41:44.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8049999999999997e-06 2024-09-17T22:41:44.536 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.536 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.536 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.536 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106819 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.537 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.240098+0000", 2024-09-17T22:41:44.537 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.030325892, 2024-09-17T22:41:44.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033557499999999997, 2024-09-17T22:41:44.537 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.537 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.537 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.537 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.537 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.537 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.537 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106819 2024-09-17T22:41:44.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.537 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240098+0000", 2024-09-17T22:41:44.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240098+0000", 2024-09-17T22:41:44.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240098+0000", 2024-09-17T22:41:44.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:41:44.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240099+0000", 2024-09-17T22:41:44.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1459999999999999e-06 2024-09-17T22:41:44.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240102+0000", 2024-09-17T22:41:44.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.96e-06 2024-09-17T22:41:44.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240105+0000", 2024-09-17T22:41:44.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5950000000000001e-06 2024-09-17T22:41:44.539 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.539 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.540 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.540 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106851 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.540 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.246151+0000", 2024-09-17T22:41:44.540 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.024272716999999999, 2024-09-17T22:41:44.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027557120000000001, 2024-09-17T22:41:44.540 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.540 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.540 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.540 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.540 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.540 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.540 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106851 2024-09-17T22:41:44.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.541 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246151+0000", 2024-09-17T22:41:44.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246151+0000", 2024-09-17T22:41:44.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246151+0000", 2024-09-17T22:41:44.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-09-17T22:41:44.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246152+0000", 2024-09-17T22:41:44.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6099999999999999e-07 2024-09-17T22:41:44.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246155+0000", 2024-09-17T22:41:44.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0139999999999999e-06 2024-09-17T22:41:44.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246158+0000", 2024-09-17T22:41:44.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6180000000000002e-06 2024-09-17T22:41:44.543 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.543 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.543 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.543 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106883 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.543 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.250378+0000", 2024-09-17T22:41:44.543 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020045759999999999, 2024-09-17T22:41:44.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023381669000000001, 2024-09-17T22:41:44.544 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.544 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.544 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.544 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106883 2024-09-17T22:41:44.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.544 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250378+0000", 2024-09-17T22:41:44.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250378+0000", 2024-09-17T22:41:44.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250379+0000", 2024-09-17T22:41:44.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2599999999999998e-07 2024-09-17T22:41:44.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250380+0000", 2024-09-17T22:41:44.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.858e-06 2024-09-17T22:41:44.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250386+0000", 2024-09-17T22:41:44.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8699999999999997e-06 2024-09-17T22:41:44.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250391+0000", 2024-09-17T22:41:44.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2710000000000003e-06 2024-09-17T22:41:44.546 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.547 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.547 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.547 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106914 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.547 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.254935+0000", 2024-09-17T22:41:44.547 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015489324, 2024-09-17T22:41:44.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01886034, 2024-09-17T22:41:44.547 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.547 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.547 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.547 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.548 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.548 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.548 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106914 2024-09-17T22:41:44.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.548 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254935+0000", 2024-09-17T22:41:44.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254935+0000", 2024-09-17T22:41:44.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256058+0000", 2024-09-17T22:41:44.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011229199999999999 2024-09-17T22:41:44.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256063+0000", 2024-09-17T22:41:44.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8390000000000003e-06 2024-09-17T22:41:44.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256073+0000", 2024-09-17T22:41:44.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5699999999999999e-06 2024-09-17T22:41:44.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256080+0000", 2024-09-17T22:41:44.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.922e-06 2024-09-17T22:41:44.550 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.550 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.550 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.550 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106945 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.551 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.262133+0000", 2024-09-17T22:41:44.551 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0082906109999999998, 2024-09-17T22:41:44.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01168903, 2024-09-17T22:41:44.551 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.551 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.551 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.551 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.551 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.551 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.551 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106945 2024-09-17T22:41:44.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.551 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262133+0000", 2024-09-17T22:41:44.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262133+0000", 2024-09-17T22:41:44.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262134+0000", 2024-09-17T22:41:44.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:41:44.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262135+0000", 2024-09-17T22:41:44.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6840000000000001e-06 2024-09-17T22:41:44.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262140+0000", 2024-09-17T22:41:44.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4780000000000002e-06 2024-09-17T22:41:44.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262143+0000", 2024-09-17T22:41:44.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.757e-06 2024-09-17T22:41:44.554 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.554 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.554 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.554 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106977 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.554 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.268237+0000", 2024-09-17T22:41:44.554 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0021870969999999998, 2024-09-17T22:41:44.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056129220000000002, 2024-09-17T22:41:44.554 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.554 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.554 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.555 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.555 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.555 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.555 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106977 2024-09-17T22:41:44.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.555 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268237+0000", 2024-09-17T22:41:44.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268237+0000", 2024-09-17T22:41:44.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268237+0000", 2024-09-17T22:41:44.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0100000000000001e-07 2024-09-17T22:41:44.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268238+0000", 2024-09-17T22:41:44.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0419999999999999e-06 2024-09-17T22:41:44.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268242+0000", 2024-09-17T22:41:44.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4589999999999998e-06 2024-09-17T22:41:44.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268244+0000", 2024-09-17T22:41:44.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6869999999999999e-06 2024-09-17T22:41:44.557 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.557 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.557 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.558 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106788 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.558 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.234159+0000", 2024-09-17T22:41:44.558 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.036264810000000001, 2024-09-17T22:41:44.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039717980999999999, 2024-09-17T22:41:44.558 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.558 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.558 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.558 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.558 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.558 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.558 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106788 2024-09-17T22:41:44.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.558 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234159+0000", 2024-09-17T22:41:44.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234159+0000", 2024-09-17T22:41:44.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234160+0000", 2024-09-17T22:41:44.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0900000000000002e-07 2024-09-17T22:41:44.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234161+0000", 2024-09-17T22:41:44.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.872e-06 2024-09-17T22:41:44.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234167+0000", 2024-09-17T22:41:44.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3929999999999999e-06 2024-09-17T22:41:44.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234172+0000", 2024-09-17T22:41:44.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.826e-06 2024-09-17T22:41:44.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.561 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.561 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106820 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.561 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.240108+0000", 2024-09-17T22:41:44.561 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.030315953999999999, 2024-09-17T22:41:44.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033812070999999999, 2024-09-17T22:41:44.561 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.561 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.562 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.562 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106820 2024-09-17T22:41:44.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.562 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240108+0000", 2024-09-17T22:41:44.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240108+0000", 2024-09-17T22:41:44.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240108+0000", 2024-09-17T22:41:44.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:41:44.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240113+0000", 2024-09-17T22:41:44.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4769999999999997e-06 2024-09-17T22:41:44.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240116+0000", 2024-09-17T22:41:44.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.269e-06 2024-09-17T22:41:44.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240119+0000", 2024-09-17T22:41:44.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6630000000000002e-06 2024-09-17T22:41:44.564 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.564 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.564 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.565 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106852 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.565 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.246161+0000", 2024-09-17T22:41:44.565 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.024262967999999999, 2024-09-17T22:41:44.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027811195, 2024-09-17T22:41:44.565 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.565 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.565 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.565 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.565 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.565 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.565 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106852 2024-09-17T22:41:44.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.565 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246161+0000", 2024-09-17T22:41:44.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246161+0000", 2024-09-17T22:41:44.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246161+0000", 2024-09-17T22:41:44.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:41:44.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246162+0000", 2024-09-17T22:41:44.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2299999999999999e-07 2024-09-17T22:41:44.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246165+0000", 2024-09-17T22:41:44.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9500000000000001e-06 2024-09-17T22:41:44.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246167+0000", 2024-09-17T22:41:44.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3709999999999998e-06 2024-09-17T22:41:44.568 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.568 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.568 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.568 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106884 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.568 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.250398+0000", 2024-09-17T22:41:44.568 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020026451000000001, 2024-09-17T22:41:44.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023627563000000001, 2024-09-17T22:41:44.568 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.568 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.568 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.568 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.568 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.568 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.569 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106884 2024-09-17T22:41:44.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.569 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250398+0000", 2024-09-17T22:41:44.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250398+0000", 2024-09-17T22:41:44.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252049+0000", 2024-09-17T22:41:44.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001651926 2024-09-17T22:41:44.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252054+0000", 2024-09-17T22:41:44.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.279e-06 2024-09-17T22:41:44.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252065+0000", 2024-09-17T22:41:44.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1345999999999999e-05 2024-09-17T22:41:44.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252073+0000", 2024-09-17T22:41:44.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5340000000000002e-06 2024-09-17T22:41:44.571 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.571 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.571 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.571 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106915 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.571 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.256094+0000", 2024-09-17T22:41:44.571 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.014329754, 2024-09-17T22:41:44.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017986992, 2024-09-17T22:41:44.571 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.572 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.572 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.572 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.572 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.572 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.572 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106915 2024-09-17T22:41:44.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.572 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256094+0000", 2024-09-17T22:41:44.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256094+0000", 2024-09-17T22:41:44.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256095+0000", 2024-09-17T22:41:44.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3399999999999999e-07 2024-09-17T22:41:44.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256102+0000", 2024-09-17T22:41:44.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.985e-06 2024-09-17T22:41:44.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256120+0000", 2024-09-17T22:41:44.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8127000000000001e-05 2024-09-17T22:41:44.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256126+0000", 2024-09-17T22:41:44.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1340000000000001e-06 2024-09-17T22:41:44.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.574 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.575 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106946 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.575 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.262152+0000", 2024-09-17T22:41:44.575 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.008271947, 2024-09-17T22:41:44.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011980349, 2024-09-17T22:41:44.575 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.575 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.575 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.575 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106946 2024-09-17T22:41:44.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.575 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262152+0000", 2024-09-17T22:41:44.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262152+0000", 2024-09-17T22:41:44.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262152+0000", 2024-09-17T22:41:44.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4400000000000001e-07 2024-09-17T22:41:44.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262154+0000", 2024-09-17T22:41:44.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.626e-06 2024-09-17T22:41:44.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262158+0000", 2024-09-17T22:41:44.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2640000000000002e-06 2024-09-17T22:41:44.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262161+0000", 2024-09-17T22:41:44.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8009999999999999e-06 2024-09-17T22:41:44.578 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.578 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.578 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.578 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106978 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.578 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.268247+0000", 2024-09-17T22:41:44.578 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0021766450000000001, 2024-09-17T22:41:44.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059371909999999996, 2024-09-17T22:41:44.578 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.578 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.578 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.578 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.578 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.578 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.579 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106978 2024-09-17T22:41:44.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.579 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268247+0000", 2024-09-17T22:41:44.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268247+0000", 2024-09-17T22:41:44.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270049+0000", 2024-09-17T22:41:44.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001801361 2024-09-17T22:41:44.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270053+0000", 2024-09-17T22:41:44.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9960000000000004e-06 2024-09-17T22:41:44.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270069+0000", 2024-09-17T22:41:44.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6283999999999999e-05 2024-09-17T22:41:44.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270075+0000", 2024-09-17T22:41:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.426e-06 2024-09-17T22:41:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106789 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.234178+0000", 2024-09-17T22:41:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.036245882, 2024-09-17T22:41:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040058536999999998, 2024-09-17T22:41:44.581 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106789 2024-09-17T22:41:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234178+0000", 2024-09-17T22:41:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234178+0000", 2024-09-17T22:41:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234178+0000", 2024-09-17T22:41:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:41:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234180+0000", 2024-09-17T22:41:44.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8330000000000001e-06 2024-09-17T22:41:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234186+0000", 2024-09-17T22:41:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7660000000000004e-06 2024-09-17T22:41:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234191+0000", 2024-09-17T22:41:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8829999999999997e-06 2024-09-17T22:41:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.584 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106821 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.240122+0000", 2024-09-17T22:41:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.030302051999999999, 2024-09-17T22:41:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034164430000000003, 2024-09-17T22:41:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.585 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106821 2024-09-17T22:41:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240122+0000", 2024-09-17T22:41:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240122+0000", 2024-09-17T22:41:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240122+0000", 2024-09-17T22:41:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3200000000000001e-07 2024-09-17T22:41:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240123+0000", 2024-09-17T22:41:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.06e-06 2024-09-17T22:41:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240126+0000", 2024-09-17T22:41:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9679999999999998e-06 2024-09-17T22:41:44.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240129+0000", 2024-09-17T22:41:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5330000000000001e-06 2024-09-17T22:41:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106853 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.246170+0000", 2024-09-17T22:41:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.024253607999999999, 2024-09-17T22:41:44.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028167757000000002, 2024-09-17T22:41:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106853 2024-09-17T22:41:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246170+0000", 2024-09-17T22:41:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246170+0000", 2024-09-17T22:41:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246171+0000", 2024-09-17T22:41:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:41:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246172+0000", 2024-09-17T22:41:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4e-07 2024-09-17T22:41:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246174+0000", 2024-09-17T22:41:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.942e-06 2024-09-17T22:41:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246177+0000", 2024-09-17T22:41:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7750000000000001e-06 2024-09-17T22:41:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106885 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.252086+0000", 2024-09-17T22:41:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018337815, 2024-09-17T22:41:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022303611000000001, 2024-09-17T22:41:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106885 2024-09-17T22:41:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252086+0000", 2024-09-17T22:41:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252086+0000", 2024-09-17T22:41:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252087+0000", 2024-09-17T22:41:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8500000000000002e-07 2024-09-17T22:41:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252089+0000", 2024-09-17T22:41:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3329999999999999e-06 2024-09-17T22:41:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252096+0000", 2024-09-17T22:41:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1960000000000002e-06 2024-09-17T22:41:44.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252103+0000", 2024-09-17T22:41:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0559999999999999e-06 2024-09-17T22:41:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106916 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.256134+0000", 2024-09-17T22:41:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.01428959, 2024-09-17T22:41:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018320645, 2024-09-17T22:41:44.595 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106916 2024-09-17T22:41:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256134+0000", 2024-09-17T22:41:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256134+0000", 2024-09-17T22:41:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256135+0000", 2024-09-17T22:41:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5299999999999999e-07 2024-09-17T22:41:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256138+0000", 2024-09-17T22:41:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7020000000000001e-06 2024-09-17T22:41:44.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256145+0000", 2024-09-17T22:41:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6299999999999998e-06 2024-09-17T22:41:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256150+0000", 2024-09-17T22:41:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0889999999999999e-06 2024-09-17T22:41:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106947 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.262165+0000", 2024-09-17T22:41:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0082593270000000003, 2024-09-17T22:41:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012343975, 2024-09-17T22:41:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.599 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106947 2024-09-17T22:41:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262165+0000", 2024-09-17T22:41:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262165+0000", 2024-09-17T22:41:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262165+0000", 2024-09-17T22:41:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.17e-07 2024-09-17T22:41:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262166+0000", 2024-09-17T22:41:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.187e-06 2024-09-17T22:41:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262170+0000", 2024-09-17T22:41:44.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1509999999999997e-06 2024-09-17T22:41:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262173+0000", 2024-09-17T22:41:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7240000000000001e-06 2024-09-17T22:41:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106979 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.270085+0000", 2024-09-17T22:41:44.602 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00033872699999999998, 2024-09-17T22:41:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044758050000000002, 2024-09-17T22:41:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106979 2024-09-17T22:41:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270085+0000", 2024-09-17T22:41:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270085+0000", 2024-09-17T22:41:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270086+0000", 2024-09-17T22:41:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3099999999999998e-07 2024-09-17T22:41:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270088+0000", 2024-09-17T22:41:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8819999999999999e-06 2024-09-17T22:41:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270091+0000", 2024-09-17T22:41:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9199999999999997e-06 2024-09-17T22:41:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270094+0000", 2024-09-17T22:41:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.897e-06 2024-09-17T22:41:44.605 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106790 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.234197+0000", 2024-09-17T22:41:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.036227236000000003, 2024-09-17T22:41:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040419012999999997, 2024-09-17T22:41:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106790 2024-09-17T22:41:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234197+0000", 2024-09-17T22:41:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234197+0000", 2024-09-17T22:41:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234197+0000", 2024-09-17T22:41:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:41:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234199+0000", 2024-09-17T22:41:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.854e-06 2024-09-17T22:41:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234205+0000", 2024-09-17T22:41:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-09-17T22:41:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234210+0000", 2024-09-17T22:41:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3789999999999997e-06 2024-09-17T22:41:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106822 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.609 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.240132+0000", 2024-09-17T22:41:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.030292458000000001, 2024-09-17T22:41:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034537709, 2024-09-17T22:41:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106822 2024-09-17T22:41:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240132+0000", 2024-09-17T22:41:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240132+0000", 2024-09-17T22:41:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240132+0000", 2024-09-17T22:41:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0599999999999999e-07 2024-09-17T22:41:44.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240133+0000", 2024-09-17T22:41:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1069999999999999e-06 2024-09-17T22:41:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240136+0000", 2024-09-17T22:41:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9519999999999999e-06 2024-09-17T22:41:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240138+0000", 2024-09-17T22:41:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4710000000000001e-06 2024-09-17T22:41:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106854 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.246180+0000", 2024-09-17T22:41:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.024243882000000001, 2024-09-17T22:41:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028541679, 2024-09-17T22:41:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.613 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106854 2024-09-17T22:41:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246180+0000", 2024-09-17T22:41:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246180+0000", 2024-09-17T22:41:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246180+0000", 2024-09-17T22:41:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.05e-07 2024-09-17T22:41:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246181+0000", 2024-09-17T22:41:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0219999999999999e-06 2024-09-17T22:41:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246184+0000", 2024-09-17T22:41:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9799999999999998e-06 2024-09-17T22:41:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246187+0000", 2024-09-17T22:41:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5799999999999999e-06 2024-09-17T22:41:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106886 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.252131+0000", 2024-09-17T22:41:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018293416, 2024-09-17T22:41:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022643941000000001, 2024-09-17T22:41:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106886 2024-09-17T22:41:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.617 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252131+0000", 2024-09-17T22:41:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252131+0000", 2024-09-17T22:41:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252131+0000", 2024-09-17T22:41:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6100000000000001e-07 2024-09-17T22:41:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252133+0000", 2024-09-17T22:41:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3700000000000002e-06 2024-09-17T22:41:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252143+0000", 2024-09-17T22:41:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2399999999999996e-06 2024-09-17T22:41:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252150+0000", 2024-09-17T22:41:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2699999999999999e-06 2024-09-17T22:41:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106917 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.256157+0000", 2024-09-17T22:41:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.014266620000000001, 2024-09-17T22:41:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018671030000000002, 2024-09-17T22:41:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.620 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106917 2024-09-17T22:41:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256157+0000", 2024-09-17T22:41:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256157+0000", 2024-09-17T22:41:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256158+0000", 2024-09-17T22:41:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:41:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256160+0000", 2024-09-17T22:41:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1629999999999999e-06 2024-09-17T22:41:44.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256166+0000", 2024-09-17T22:41:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0090000000000004e-06 2024-09-17T22:41:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256224+0000", 2024-09-17T22:41:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8278e-05 2024-09-17T22:41:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106948 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.262176+0000", 2024-09-17T22:41:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0082480130000000002, 2024-09-17T22:41:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012705696000000001, 2024-09-17T22:41:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106948 2024-09-17T22:41:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262176+0000", 2024-09-17T22:41:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262176+0000", 2024-09-17T22:41:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262176+0000", 2024-09-17T22:41:44.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0899999999999997e-07 2024-09-17T22:41:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262178+0000", 2024-09-17T22:41:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.235e-06 2024-09-17T22:41:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262181+0000", 2024-09-17T22:41:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9709999999999998e-06 2024-09-17T22:41:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262184+0000", 2024-09-17T22:41:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6579999999999998e-06 2024-09-17T22:41:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106980 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.270098+0000", 2024-09-17T22:41:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00032576600000000001, 2024-09-17T22:41:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048370089999999998, 2024-09-17T22:41:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.627 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106980 2024-09-17T22:41:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270098+0000", 2024-09-17T22:41:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270098+0000", 2024-09-17T22:41:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270098+0000", 2024-09-17T22:41:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:41:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270100+0000", 2024-09-17T22:41:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1739999999999999e-06 2024-09-17T22:41:44.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270103+0000", 2024-09-17T22:41:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.089e-06 2024-09-17T22:41:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270105+0000", 2024-09-17T22:41:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7099999999999999e-06 2024-09-17T22:41:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106791 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.234216+0000", 2024-09-17T22:41:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.036208157999999997, 2024-09-17T22:41:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040775245000000002, 2024-09-17T22:41:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106791 2024-09-17T22:41:44.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234216+0000", 2024-09-17T22:41:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234216+0000", 2024-09-17T22:41:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234216+0000", 2024-09-17T22:41:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000002e-07 2024-09-17T22:41:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234218+0000", 2024-09-17T22:41:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.832e-06 2024-09-17T22:41:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234224+0000", 2024-09-17T22:41:44.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7620000000000001e-06 2024-09-17T22:41:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234230+0000", 2024-09-17T22:41:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8479999999999997e-06 2024-09-17T22:41:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106823 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.240141+0000", 2024-09-17T22:41:44.634 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.030282851999999999, 2024-09-17T22:41:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034906617000000001, 2024-09-17T22:41:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106823 2024-09-17T22:41:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240141+0000", 2024-09-17T22:41:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240141+0000", 2024-09-17T22:41:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240141+0000", 2024-09-17T22:41:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:41:44.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240142+0000", 2024-09-17T22:41:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0020000000000001e-06 2024-09-17T22:41:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240145+0000", 2024-09-17T22:41:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9830000000000001e-06 2024-09-17T22:41:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240148+0000", 2024-09-17T22:41:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3939999999999999e-06 2024-09-17T22:41:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106855 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.246191+0000", 2024-09-17T22:41:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.024232997999999999, 2024-09-17T22:41:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028911405000000001, 2024-09-17T22:41:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106855 2024-09-17T22:41:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246191+0000", 2024-09-17T22:41:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246191+0000", 2024-09-17T22:41:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246191+0000", 2024-09-17T22:41:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:41:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246195+0000", 2024-09-17T22:41:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0099999999999997e-06 2024-09-17T22:41:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246198+0000", 2024-09-17T22:41:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0869999999999998e-06 2024-09-17T22:41:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246201+0000", 2024-09-17T22:41:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7259999999999998e-06 2024-09-17T22:41:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106887 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.641 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.252158+0000", 2024-09-17T22:41:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018265685, 2024-09-17T22:41:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022997159999999999, 2024-09-17T22:41:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106887 2024-09-17T22:41:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252158+0000", 2024-09-17T22:41:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252158+0000", 2024-09-17T22:41:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252159+0000", 2024-09-17T22:41:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:41:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252161+0000", 2024-09-17T22:41:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.029e-06 2024-09-17T22:41:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252167+0000", 2024-09-17T22:41:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8880000000000002e-06 2024-09-17T22:41:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252171+0000", 2024-09-17T22:41:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6639999999999997e-06 2024-09-17T22:41:44.644 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106918 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.256235+0000", 2024-09-17T22:41:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.014189451, 2024-09-17T22:41:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018974807, 2024-09-17T22:41:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.645 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106918 2024-09-17T22:41:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256235+0000", 2024-09-17T22:41:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256235+0000", 2024-09-17T22:41:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256235+0000", 2024-09-17T22:41:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4199999999999996e-07 2024-09-17T22:41:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256238+0000", 2024-09-17T22:41:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8590000000000001e-06 2024-09-17T22:41:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256245+0000", 2024-09-17T22:41:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1409999999999999e-06 2024-09-17T22:41:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256251+0000", 2024-09-17T22:41:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7529999999999998e-06 2024-09-17T22:41:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.648 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106949 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.262187+0000", 2024-09-17T22:41:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0082368189999999994, 2024-09-17T22:41:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013078803999999999, 2024-09-17T22:41:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106949 2024-09-17T22:41:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262187+0000", 2024-09-17T22:41:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.262187+0000", 2024-09-17T22:41:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264051+0000", 2024-09-17T22:41:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001864267 2024-09-17T22:41:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264058+0000", 2024-09-17T22:41:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.562e-06 2024-09-17T22:41:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264070+0000", 2024-09-17T22:41:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1559000000000001e-05 2024-09-17T22:41:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264078+0000", 2024-09-17T22:41:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2160000000000006e-06 2024-09-17T22:41:44.651 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106981 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.270109+0000", 2024-09-17T22:41:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00031492800000000002, 2024-09-17T22:41:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052094239999999998, 2024-09-17T22:41:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.652 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106981 2024-09-17T22:41:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270109+0000", 2024-09-17T22:41:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270109+0000", 2024-09-17T22:41:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270109+0000", 2024-09-17T22:41:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:41:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270110+0000", 2024-09-17T22:41:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.116e-06 2024-09-17T22:41:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270113+0000", 2024-09-17T22:41:44.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0520000000000002e-06 2024-09-17T22:41:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270116+0000", 2024-09-17T22:41:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.57e-06 2024-09-17T22:41:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106792 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.234236+0000", 2024-09-17T22:41:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.036188410999999997, 2024-09-17T22:41:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041139694999999997, 2024-09-17T22:41:44.655 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106792 2024-09-17T22:41:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234236+0000", 2024-09-17T22:41:44.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234236+0000", 2024-09-17T22:41:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234236+0000", 2024-09-17T22:41:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8100000000000003e-07 2024-09-17T22:41:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234238+0000", 2024-09-17T22:41:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.866e-06 2024-09-17T22:41:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234243+0000", 2024-09-17T22:41:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5629999999999999e-06 2024-09-17T22:41:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234248+0000", 2024-09-17T22:41:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5539999999999999e-06 2024-09-17T22:41:44.658 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106824 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.240151+0000", 2024-09-17T22:41:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.030273343000000001, 2024-09-17T22:41:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.035277443999999998, 2024-09-17T22:41:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106824 2024-09-17T22:41:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240151+0000", 2024-09-17T22:41:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240151+0000", 2024-09-17T22:41:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240151+0000", 2024-09-17T22:41:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:41:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240152+0000", 2024-09-17T22:41:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5099999999999998e-07 2024-09-17T22:41:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240155+0000", 2024-09-17T22:41:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8849999999999999e-06 2024-09-17T22:41:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240157+0000", 2024-09-17T22:41:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4949999999999998e-06 2024-09-17T22:41:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106856 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.246204+0000", 2024-09-17T22:41:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.024219774999999999, 2024-09-17T22:41:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029277752000000001, 2024-09-17T22:41:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106856 2024-09-17T22:41:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246204+0000", 2024-09-17T22:41:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246204+0000", 2024-09-17T22:41:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246204+0000", 2024-09-17T22:41:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-09-17T22:41:44.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246206+0000", 2024-09-17T22:41:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.13e-06 2024-09-17T22:41:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246208+0000", 2024-09-17T22:41:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8700000000000001e-06 2024-09-17T22:41:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246211+0000", 2024-09-17T22:41:44.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5390000000000002e-06 2024-09-17T22:41:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106888 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.252178+0000", 2024-09-17T22:41:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018246471, 2024-09-17T22:41:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023357170999999999, 2024-09-17T22:41:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106888 2024-09-17T22:41:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252178+0000", 2024-09-17T22:41:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252178+0000", 2024-09-17T22:41:44.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252178+0000", 2024-09-17T22:41:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.01e-07 2024-09-17T22:41:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252180+0000", 2024-09-17T22:41:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.091e-06 2024-09-17T22:41:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252186+0000", 2024-09-17T22:41:44.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4690000000000002e-06 2024-09-17T22:41:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252192+0000", 2024-09-17T22:41:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1699999999999996e-06 2024-09-17T22:41:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106919 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.256257+0000", 2024-09-17T22:41:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.014166728999999999, 2024-09-17T22:41:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019329466, 2024-09-17T22:41:44.669 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106919 2024-09-17T22:41:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256257+0000", 2024-09-17T22:41:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256257+0000", 2024-09-17T22:41:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256258+0000", 2024-09-17T22:41:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3000000000000001e-07 2024-09-17T22:41:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256260+0000", 2024-09-17T22:41:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0169999999999999e-06 2024-09-17T22:41:44.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256266+0000", 2024-09-17T22:41:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8710000000000002e-06 2024-09-17T22:41:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256270+0000", 2024-09-17T22:41:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.775e-06 2024-09-17T22:41:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106950 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.264092+0000", 2024-09-17T22:41:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0063316650000000002, 2024-09-17T22:41:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011546232999999999, 2024-09-17T22:41:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106950 2024-09-17T22:41:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264092+0000", 2024-09-17T22:41:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264092+0000", 2024-09-17T22:41:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264093+0000", 2024-09-17T22:41:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:41:44.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264095+0000", 2024-09-17T22:41:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.39e-06 2024-09-17T22:41:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264102+0000", 2024-09-17T22:41:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6409999999999996e-06 2024-09-17T22:41:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264107+0000", 2024-09-17T22:41:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6879999999999996e-06 2024-09-17T22:41:44.675 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106982 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.270119+0000", 2024-09-17T22:41:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00030493600000000001, 2024-09-17T22:41:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0055716840000000004, 2024-09-17T22:41:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.676 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106982 2024-09-17T22:41:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270119+0000", 2024-09-17T22:41:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270119+0000", 2024-09-17T22:41:44.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270119+0000", 2024-09-17T22:41:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9999999999999999e-07 2024-09-17T22:41:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270120+0000", 2024-09-17T22:41:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0699999999999999e-06 2024-09-17T22:41:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270123+0000", 2024-09-17T22:41:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-09-17T22:41:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270126+0000", 2024-09-17T22:41:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.52e-06 2024-09-17T22:41:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106793 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.234254+0000", 2024-09-17T22:41:44.679 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.036170178999999997, 2024-09-17T22:41:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041491338000000003, 2024-09-17T22:41:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106793 2024-09-17T22:41:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234254+0000", 2024-09-17T22:41:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234254+0000", 2024-09-17T22:41:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234254+0000", 2024-09-17T22:41:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:41:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234256+0000", 2024-09-17T22:41:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0020000000000001e-06 2024-09-17T22:41:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234262+0000", 2024-09-17T22:41:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5679999999999999e-06 2024-09-17T22:41:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234266+0000", 2024-09-17T22:41:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5399999999999997e-06 2024-09-17T22:41:44.682 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106825 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.240160+0000", 2024-09-17T22:41:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.030263785000000001, 2024-09-17T22:41:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.035639279000000003, 2024-09-17T22:41:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.683 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106825 2024-09-17T22:41:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240160+0000", 2024-09-17T22:41:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240160+0000", 2024-09-17T22:41:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240160+0000", 2024-09-17T22:41:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9999999999999999e-07 2024-09-17T22:41:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240161+0000", 2024-09-17T22:41:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7499999999999998e-07 2024-09-17T22:41:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240164+0000", 2024-09-17T22:41:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9380000000000001e-06 2024-09-17T22:41:44.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240167+0000", 2024-09-17T22:41:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.463e-06 2024-09-17T22:41:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106857 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.246214+0000", 2024-09-17T22:41:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.024210124, 2024-09-17T22:41:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029638326, 2024-09-17T22:41:44.686 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.687 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.687 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.687 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106857 2024-09-17T22:41:44.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.687 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246214+0000", 2024-09-17T22:41:44.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246214+0000", 2024-09-17T22:41:44.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246214+0000", 2024-09-17T22:41:44.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:41:44.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246215+0000", 2024-09-17T22:41:44.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0109999999999999e-06 2024-09-17T22:41:44.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246218+0000", 2024-09-17T22:41:44.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9050000000000001e-06 2024-09-17T22:41:44.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246221+0000", 2024-09-17T22:41:44.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5770000000000001e-06 2024-09-17T22:41:44.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.689 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.690 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106889 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.690 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.252199+0000", 2024-09-17T22:41:44.690 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018225407999999998, 2024-09-17T22:41:44.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023705819, 2024-09-17T22:41:44.690 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.690 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.690 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.690 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.690 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106889 2024-09-17T22:41:44.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.690 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252199+0000", 2024-09-17T22:41:44.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252199+0000", 2024-09-17T22:41:44.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252199+0000", 2024-09-17T22:41:44.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:41:44.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252201+0000", 2024-09-17T22:41:44.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.266e-06 2024-09-17T22:41:44.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252208+0000", 2024-09-17T22:41:44.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5139999999999998e-06 2024-09-17T22:41:44.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252219+0000", 2024-09-17T22:41:44.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1685000000000001e-05 2024-09-17T22:41:44.693 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.693 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.693 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.693 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106951 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.693 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.264115+0000", 2024-09-17T22:41:44.693 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0063088520000000002, 2024-09-17T22:41:44.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011841866, 2024-09-17T22:41:44.693 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.693 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.694 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.694 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.694 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.694 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.694 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106951 2024-09-17T22:41:44.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.694 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264115+0000", 2024-09-17T22:41:44.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264115+0000", 2024-09-17T22:41:44.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264116+0000", 2024-09-17T22:41:44.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1099999999999996e-07 2024-09-17T22:41:44.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264123+0000", 2024-09-17T22:41:44.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4900000000000003e-06 2024-09-17T22:41:44.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264130+0000", 2024-09-17T22:41:44.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0600000000000002e-06 2024-09-17T22:41:44.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264135+0000", 2024-09-17T22:41:44.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7149999999999997e-06 2024-09-17T22:41:44.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.696 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.696 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106983 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.697 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.270129+0000", 2024-09-17T22:41:44.697 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00029540700000000003, 2024-09-17T22:41:44.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0058812880000000001, 2024-09-17T22:41:44.697 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.697 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.697 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.697 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106983 2024-09-17T22:41:44.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.697 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270129+0000", 2024-09-17T22:41:44.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270129+0000", 2024-09-17T22:41:44.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270129+0000", 2024-09-17T22:41:44.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1299999999999999e-07 2024-09-17T22:41:44.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270130+0000", 2024-09-17T22:41:44.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0979999999999999e-06 2024-09-17T22:41:44.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270133+0000", 2024-09-17T22:41:44.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8760000000000001e-06 2024-09-17T22:41:44.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270135+0000", 2024-09-17T22:41:44.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5809999999999999e-06 2024-09-17T22:41:44.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.699 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.700 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106794 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.700 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.234272+0000", 2024-09-17T22:41:44.700 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.036152062999999998, 2024-09-17T22:41:44.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041797204999999997, 2024-09-17T22:41:44.700 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.700 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.700 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.700 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106794 2024-09-17T22:41:44.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.701 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234272+0000", 2024-09-17T22:41:44.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234272+0000", 2024-09-17T22:41:44.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234272+0000", 2024-09-17T22:41:44.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-09-17T22:41:44.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234274+0000", 2024-09-17T22:41:44.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7069999999999999e-06 2024-09-17T22:41:44.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234280+0000", 2024-09-17T22:41:44.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5890000000000002e-06 2024-09-17T22:41:44.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234284+0000", 2024-09-17T22:41:44.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999998e-06 2024-09-17T22:41:44.703 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.703 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.703 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.703 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106826 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.703 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.240170+0000", 2024-09-17T22:41:44.703 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.030254260000000002, 2024-09-17T22:41:44.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.035953311000000002, 2024-09-17T22:41:44.703 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.703 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.704 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.704 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.704 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.704 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.704 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106826 2024-09-17T22:41:44.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.704 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240170+0000", 2024-09-17T22:41:44.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.240170+0000", 2024-09-17T22:41:44.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242049+0000", 2024-09-17T22:41:44.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001878846 2024-09-17T22:41:44.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242052+0000", 2024-09-17T22:41:44.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4489999999999999e-06 2024-09-17T22:41:44.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242063+0000", 2024-09-17T22:41:44.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0633e-05 2024-09-17T22:41:44.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242067+0000", 2024-09-17T22:41:44.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5970000000000002e-06 2024-09-17T22:41:44.706 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.706 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.706 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.706 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106858 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.707 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.246223+0000", 2024-09-17T22:41:44.707 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.024200597000000001, 2024-09-17T22:41:44.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029955187000000001, 2024-09-17T22:41:44.707 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.707 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.707 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.707 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106858 2024-09-17T22:41:44.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.707 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246223+0000", 2024-09-17T22:41:44.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246223+0000", 2024-09-17T22:41:44.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246224+0000", 2024-09-17T22:41:44.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0100000000000001e-07 2024-09-17T22:41:44.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246225+0000", 2024-09-17T22:41:44.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0020000000000001e-06 2024-09-17T22:41:44.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246227+0000", 2024-09-17T22:41:44.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8150000000000002e-06 2024-09-17T22:41:44.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246230+0000", 2024-09-17T22:41:44.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6740000000000001e-06 2024-09-17T22:41:44.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.710 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.710 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106890 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.710 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.252226+0000", 2024-09-17T22:41:44.710 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018197775999999999, 2024-09-17T22:41:44.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024005340999999999, 2024-09-17T22:41:44.710 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.710 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.710 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.711 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106890 2024-09-17T22:41:44.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.711 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252226+0000", 2024-09-17T22:41:44.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252226+0000", 2024-09-17T22:41:44.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252227+0000", 2024-09-17T22:41:44.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5499999999999999e-07 2024-09-17T22:41:44.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252229+0000", 2024-09-17T22:41:44.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0509999999999999e-06 2024-09-17T22:41:44.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252240+0000", 2024-09-17T22:41:44.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0927999999999999e-05 2024-09-17T22:41:44.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252251+0000", 2024-09-17T22:41:44.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1463e-05 2024-09-17T22:41:44.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.713 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.713 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106920 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.714 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.256276+0000", 2024-09-17T22:41:44.714 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.014148189, 2024-09-17T22:41:44.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020009474999999999, 2024-09-17T22:41:44.714 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.714 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.714 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.714 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.714 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.714 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.714 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106920 2024-09-17T22:41:44.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.714 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256276+0000", 2024-09-17T22:41:44.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.256276+0000", 2024-09-17T22:41:44.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258051+0000", 2024-09-17T22:41:44.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017756180000000001 2024-09-17T22:41:44.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258056+0000", 2024-09-17T22:41:44.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8539999999999996e-06 2024-09-17T22:41:44.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258064+0000", 2024-09-17T22:41:44.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.729e-06 2024-09-17T22:41:44.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258070+0000", 2024-09-17T22:41:44.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2979999999999997e-06 2024-09-17T22:41:44.717 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.717 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.717 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.717 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106952 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.717 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.264142+0000", 2024-09-17T22:41:44.717 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.006281895, 2024-09-17T22:41:44.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012196289000000001, 2024-09-17T22:41:44.717 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.717 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.717 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.718 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.718 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.718 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106952 2024-09-17T22:41:44.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.718 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264142+0000", 2024-09-17T22:41:44.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264142+0000", 2024-09-17T22:41:44.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264143+0000", 2024-09-17T22:41:44.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1900000000000003e-07 2024-09-17T22:41:44.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264145+0000", 2024-09-17T22:41:44.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1780000000000002e-06 2024-09-17T22:41:44.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264151+0000", 2024-09-17T22:41:44.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6850000000000004e-06 2024-09-17T22:41:44.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264157+0000", 2024-09-17T22:41:44.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1189999999999996e-06 2024-09-17T22:41:44.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.720 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.720 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106984 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.720 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.270138+0000", 2024-09-17T22:41:44.721 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00028571899999999998, 2024-09-17T22:41:44.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062522109999999997, 2024-09-17T22:41:44.721 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.721 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.721 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.721 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106984 2024-09-17T22:41:44.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.721 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270138+0000", 2024-09-17T22:41:44.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270138+0000", 2024-09-17T22:41:44.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270138+0000", 2024-09-17T22:41:44.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-09-17T22:41:44.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270139+0000", 2024-09-17T22:41:44.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.02e-06 2024-09-17T22:41:44.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270142+0000", 2024-09-17T22:41:44.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8990000000000001e-06 2024-09-17T22:41:44.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270145+0000", 2024-09-17T22:41:44.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7e-06 2024-09-17T22:41:44.723 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.724 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.724 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106795 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.724 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.234290+0000", 2024-09-17T22:41:44.724 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.036133512, 2024-09-17T22:41:44.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042154279000000003, 2024-09-17T22:41:44.724 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.724 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.724 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.724 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106795 2024-09-17T22:41:44.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.725 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234290+0000", 2024-09-17T22:41:44.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234290+0000", 2024-09-17T22:41:44.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234326+0000", 2024-09-17T22:41:44.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5709e-05 2024-09-17T22:41:44.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234329+0000", 2024-09-17T22:41:44.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7180000000000001e-06 2024-09-17T22:41:44.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234335+0000", 2024-09-17T22:41:44.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4740000000000002e-06 2024-09-17T22:41:44.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234341+0000", 2024-09-17T22:41:44.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.356e-06 2024-09-17T22:41:44.727 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.727 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.727 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.727 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106827 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.727 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.242090+0000", 2024-09-17T22:41:44.727 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.028333855000000002, 2024-09-17T22:41:44.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034407205000000003, 2024-09-17T22:41:44.727 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.727 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.727 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.728 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106827 2024-09-17T22:41:44.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.728 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242090+0000", 2024-09-17T22:41:44.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242090+0000", 2024-09-17T22:41:44.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242091+0000", 2024-09-17T22:41:44.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0800000000000004e-07 2024-09-17T22:41:44.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242093+0000", 2024-09-17T22:41:44.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4710000000000001e-06 2024-09-17T22:41:44.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242099+0000", 2024-09-17T22:41:44.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9730000000000002e-06 2024-09-17T22:41:44.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242104+0000", 2024-09-17T22:41:44.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1479999999999997e-06 2024-09-17T22:41:44.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.730 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.730 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106859 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.730 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.246233+0000", 2024-09-17T22:41:44.730 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.024191125000000001, 2024-09-17T22:41:44.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030317179, 2024-09-17T22:41:44.730 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.730 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.730 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.731 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106859 2024-09-17T22:41:44.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.731 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246233+0000", 2024-09-17T22:41:44.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.246233+0000", 2024-09-17T22:41:44.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248049+0000", 2024-09-17T22:41:44.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018162370000000001 2024-09-17T22:41:44.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248053+0000", 2024-09-17T22:41:44.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3429999999999998e-06 2024-09-17T22:41:44.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248064+0000", 2024-09-17T22:41:44.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0723999999999999e-05 2024-09-17T22:41:44.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248071+0000", 2024-09-17T22:41:44.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2389999999999996e-06 2024-09-17T22:41:44.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.733 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.733 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106891 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.733 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.252258+0000", 2024-09-17T22:41:44.733 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018165559000000001, 2024-09-17T22:41:44.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0243447, 2024-09-17T22:41:44.734 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.734 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.734 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.734 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106891 2024-09-17T22:41:44.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.734 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252258+0000", 2024-09-17T22:41:44.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252258+0000", 2024-09-17T22:41:44.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252259+0000", 2024-09-17T22:41:44.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.39e-07 2024-09-17T22:41:44.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252265+0000", 2024-09-17T22:41:44.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3090000000000001e-06 2024-09-17T22:41:44.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252272+0000", 2024-09-17T22:41:44.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3000000000000004e-06 2024-09-17T22:41:44.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252283+0000", 2024-09-17T22:41:44.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.007e-05 2024-09-17T22:41:44.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.736 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.736 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106921 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.737 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.258085+0000", 2024-09-17T22:41:44.737 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.012338711, 2024-09-17T22:41:44.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018571284, 2024-09-17T22:41:44.737 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.737 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.737 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.737 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106921 2024-09-17T22:41:44.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.737 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258085+0000", 2024-09-17T22:41:44.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258085+0000", 2024-09-17T22:41:44.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258086+0000", 2024-09-17T22:41:44.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:41:44.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258088+0000", 2024-09-17T22:41:44.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7709999999999998e-06 2024-09-17T22:41:44.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258095+0000", 2024-09-17T22:41:44.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0119999999999999e-06 2024-09-17T22:41:44.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258101+0000", 2024-09-17T22:41:44.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3000000000000001e-06 2024-09-17T22:41:44.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.739 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.740 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106953 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.740 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.264164+0000", 2024-09-17T22:41:44.740 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0062602990000000004, 2024-09-17T22:41:44.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012545275, 2024-09-17T22:41:44.740 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.740 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.740 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.740 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106953 2024-09-17T22:41:44.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.741 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264164+0000", 2024-09-17T22:41:44.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264164+0000", 2024-09-17T22:41:44.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264164+0000", 2024-09-17T22:41:44.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.65e-07 2024-09-17T22:41:44.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264166+0000", 2024-09-17T22:41:44.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2120000000000002e-06 2024-09-17T22:41:44.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264173+0000", 2024-09-17T22:41:44.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7270000000000001e-06 2024-09-17T22:41:44.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264178+0000", 2024-09-17T22:41:44.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2390000000000002e-06 2024-09-17T22:41:44.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.743 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.743 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106985 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.743 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.270148+0000", 2024-09-17T22:41:44.743 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00027604700000000002, 2024-09-17T22:41:44.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0066131419999999998, 2024-09-17T22:41:44.743 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.743 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.744 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106985 2024-09-17T22:41:44.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.744 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270148+0000", 2024-09-17T22:41:44.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270148+0000", 2024-09-17T22:41:44.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270148+0000", 2024-09-17T22:41:44.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-09-17T22:41:44.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270149+0000", 2024-09-17T22:41:44.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-17T22:41:44.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270153+0000", 2024-09-17T22:41:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4860000000000002e-06 2024-09-17T22:41:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270155+0000", 2024-09-17T22:41:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6010000000000002e-06 2024-09-17T22:41:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.746 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106796 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.234349+0000", 2024-09-17T22:41:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.036074510999999997, 2024-09-17T22:41:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042466867999999998, 2024-09-17T22:41:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106796 2024-09-17T22:41:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234349+0000", 2024-09-17T22:41:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234349+0000", 2024-09-17T22:41:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234538+0000", 2024-09-17T22:41:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018850900000000001 2024-09-17T22:41:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234544+0000", 2024-09-17T22:41:44.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9370000000000001e-06 2024-09-17T22:41:44.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234551+0000", 2024-09-17T22:41:44.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7530000000000004e-06 2024-09-17T22:41:44.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234556+0000", 2024-09-17T22:41:44.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5269999999999998e-06 2024-09-17T22:41:44.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.750 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.750 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106828 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.750 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.242111+0000", 2024-09-17T22:41:44.750 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.028312826999999999, 2024-09-17T22:41:44.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034758095000000003, 2024-09-17T22:41:44.750 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.750 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.750 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.750 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106828 2024-09-17T22:41:44.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.751 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242111+0000", 2024-09-17T22:41:44.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242111+0000", 2024-09-17T22:41:44.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242112+0000", 2024-09-17T22:41:44.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:41:44.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242113+0000", 2024-09-17T22:41:44.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.81e-06 2024-09-17T22:41:44.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242119+0000", 2024-09-17T22:41:44.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6690000000000003e-06 2024-09-17T22:41:44.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242125+0000", 2024-09-17T22:41:44.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6250000000000004e-06 2024-09-17T22:41:44.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.753 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.753 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106860 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.753 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.248082+0000", 2024-09-17T22:41:44.753 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.022342016999999999, 2024-09-17T22:41:44.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028841615000000001, 2024-09-17T22:41:44.753 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.753 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.753 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.753 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.753 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.754 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106860 2024-09-17T22:41:44.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.754 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248082+0000", 2024-09-17T22:41:44.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248082+0000", 2024-09-17T22:41:44.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248082+0000", 2024-09-17T22:41:44.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9100000000000004e-07 2024-09-17T22:41:44.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248085+0000", 2024-09-17T22:41:44.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1670000000000002e-06 2024-09-17T22:41:44.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248097+0000", 2024-09-17T22:41:44.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2051e-05 2024-09-17T22:41:44.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248102+0000", 2024-09-17T22:41:44.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7590000000000003e-06 2024-09-17T22:41:44.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.756 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.756 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106892 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.756 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.252290+0000", 2024-09-17T22:41:44.756 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018133513, 2024-09-17T22:41:44.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024688605999999998, 2024-09-17T22:41:44.756 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.756 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.756 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.757 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106892 2024-09-17T22:41:44.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.757 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252290+0000", 2024-09-17T22:41:44.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252290+0000", 2024-09-17T22:41:44.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252291+0000", 2024-09-17T22:41:44.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.75e-07 2024-09-17T22:41:44.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252293+0000", 2024-09-17T22:41:44.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.114e-06 2024-09-17T22:41:44.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252299+0000", 2024-09-17T22:41:44.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2299999999999996e-06 2024-09-17T22:41:44.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252305+0000", 2024-09-17T22:41:44.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5149999999999997e-06 2024-09-17T22:41:44.759 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.759 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.759 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.759 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106922 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.759 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.258109+0000", 2024-09-17T22:41:44.759 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.012314973, 2024-09-17T22:41:44.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018960536, 2024-09-17T22:41:44.760 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.760 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.760 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.760 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.760 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.760 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.760 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106922 2024-09-17T22:41:44.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.760 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258109+0000", 2024-09-17T22:41:44.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258109+0000", 2024-09-17T22:41:44.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258109+0000", 2024-09-17T22:41:44.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1600000000000002e-07 2024-09-17T22:41:44.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258112+0000", 2024-09-17T22:41:44.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5490000000000001e-06 2024-09-17T22:41:44.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258118+0000", 2024-09-17T22:41:44.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1890000000000004e-06 2024-09-17T22:41:44.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258123+0000", 2024-09-17T22:41:44.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8640000000000004e-06 2024-09-17T22:41:44.762 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.762 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.762 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.762 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106954 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.763 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.264185+0000", 2024-09-17T22:41:44.763 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0062390170000000003, 2024-09-17T22:41:44.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012938201999999999, 2024-09-17T22:41:44.763 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.763 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.763 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.763 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.763 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.763 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.763 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106954 2024-09-17T22:41:44.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.763 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264185+0000", 2024-09-17T22:41:44.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264185+0000", 2024-09-17T22:41:44.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264185+0000", 2024-09-17T22:41:44.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3099999999999998e-07 2024-09-17T22:41:44.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264188+0000", 2024-09-17T22:41:44.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3120000000000001e-06 2024-09-17T22:41:44.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264194+0000", 2024-09-17T22:41:44.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1290000000000001e-06 2024-09-17T22:41:44.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264199+0000", 2024-09-17T22:41:44.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.023e-06 2024-09-17T22:41:44.766 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.766 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.766 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.766 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106986 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.766 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.270158+0000", 2024-09-17T22:41:44.766 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00026571100000000002, 2024-09-17T22:41:44.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0070184499999999999, 2024-09-17T22:41:44.766 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.766 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.766 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.766 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.767 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106986 2024-09-17T22:41:44.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.767 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270158+0000", 2024-09-17T22:41:44.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270158+0000", 2024-09-17T22:41:44.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270158+0000", 2024-09-17T22:41:44.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.05e-07 2024-09-17T22:41:44.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270162+0000", 2024-09-17T22:41:44.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9140000000000001e-06 2024-09-17T22:41:44.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270166+0000", 2024-09-17T22:41:44.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2320000000000001e-06 2024-09-17T22:41:44.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270168+0000", 2024-09-17T22:41:44.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6690000000000002e-06 2024-09-17T22:41:44.769 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.769 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.769 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.770 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106797 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.770 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.234565+0000", 2024-09-17T22:41:44.770 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.035859004, 2024-09-17T22:41:44.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042666215, 2024-09-17T22:41:44.770 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.770 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.770 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.770 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.770 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.770 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.770 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106797 2024-09-17T22:41:44.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.770 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234565+0000", 2024-09-17T22:41:44.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.234565+0000", 2024-09-17T22:41:44.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236050+0000", 2024-09-17T22:41:44.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001485203 2024-09-17T22:41:44.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236054+0000", 2024-09-17T22:41:44.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0409999999999999e-06 2024-09-17T22:41:44.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236062+0000", 2024-09-17T22:41:44.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6969999999999993e-06 2024-09-17T22:41:44.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236068+0000", 2024-09-17T22:41:44.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8440000000000003e-06 2024-09-17T22:41:44.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.773 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.773 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106829 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.773 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.242131+0000", 2024-09-17T22:41:44.773 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.028293308999999999, 2024-09-17T22:41:44.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.035153644999999997, 2024-09-17T22:41:44.773 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.773 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.773 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.774 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106829 2024-09-17T22:41:44.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.774 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242131+0000", 2024-09-17T22:41:44.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242131+0000", 2024-09-17T22:41:44.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242131+0000", 2024-09-17T22:41:44.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7099999999999997e-07 2024-09-17T22:41:44.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242133+0000", 2024-09-17T22:41:44.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9649999999999998e-06 2024-09-17T22:41:44.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242139+0000", 2024-09-17T22:41:44.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5999999999999997e-06 2024-09-17T22:41:44.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242143+0000", 2024-09-17T22:41:44.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5850000000000001e-06 2024-09-17T22:41:44.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.776 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.776 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106861 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.776 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.248109+0000", 2024-09-17T22:41:44.776 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.022314975000000001, 2024-09-17T22:41:44.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029228391999999999, 2024-09-17T22:41:44.777 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.777 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.777 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.777 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106861 2024-09-17T22:41:44.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.777 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248109+0000", 2024-09-17T22:41:44.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248109+0000", 2024-09-17T22:41:44.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248109+0000", 2024-09-17T22:41:44.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2899999999999999e-07 2024-09-17T22:41:44.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248112+0000", 2024-09-17T22:41:44.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0779999999999998e-06 2024-09-17T22:41:44.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248121+0000", 2024-09-17T22:41:44.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5019999999999998e-06 2024-09-17T22:41:44.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248126+0000", 2024-09-17T22:41:44.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.079e-06 2024-09-17T22:41:44.779 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.779 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.779 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.780 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106893 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.780 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.252311+0000", 2024-09-17T22:41:44.780 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018113212, 2024-09-17T22:41:44.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025080126000000001, 2024-09-17T22:41:44.780 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.780 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.780 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.780 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106893 2024-09-17T22:41:44.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.781 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252311+0000", 2024-09-17T22:41:44.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252311+0000", 2024-09-17T22:41:44.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252311+0000", 2024-09-17T22:41:44.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3000000000000001e-07 2024-09-17T22:41:44.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252313+0000", 2024-09-17T22:41:44.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.889e-06 2024-09-17T22:41:44.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252319+0000", 2024-09-17T22:41:44.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4110000000000001e-06 2024-09-17T22:41:44.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252324+0000", 2024-09-17T22:41:44.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8609999999999997e-06 2024-09-17T22:41:44.783 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.783 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.783 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.783 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106923 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.783 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.258129+0000", 2024-09-17T22:41:44.783 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.012295278999999999, 2024-09-17T22:41:44.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019314695999999999, 2024-09-17T22:41:44.783 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.783 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.783 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.784 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.784 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.784 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.784 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106923 2024-09-17T22:41:44.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.784 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258129+0000", 2024-09-17T22:41:44.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258129+0000", 2024-09-17T22:41:44.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258129+0000", 2024-09-17T22:41:44.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:41:44.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258131+0000", 2024-09-17T22:41:44.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0930000000000001e-06 2024-09-17T22:41:44.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258137+0000", 2024-09-17T22:41:44.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.088e-06 2024-09-17T22:41:44.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258142+0000", 2024-09-17T22:41:44.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0420000000000002e-06 2024-09-17T22:41:44.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.786 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.786 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106955 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.786 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.264205+0000", 2024-09-17T22:41:44.786 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0062188629999999998, 2024-09-17T22:41:44.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013290618000000001, 2024-09-17T22:41:44.787 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.787 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.787 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.787 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106955 2024-09-17T22:41:44.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.787 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264205+0000", 2024-09-17T22:41:44.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.264205+0000", 2024-09-17T22:41:44.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266050+0000", 2024-09-17T22:41:44.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018450070000000001 2024-09-17T22:41:44.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266055+0000", 2024-09-17T22:41:44.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2499999999999997e-06 2024-09-17T22:41:44.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266070+0000", 2024-09-17T22:41:44.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4221e-05 2024-09-17T22:41:44.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266075+0000", 2024-09-17T22:41:44.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5740000000000003e-06 2024-09-17T22:41:44.789 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.789 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.789 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106987 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.270172+0000", 2024-09-17T22:41:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00025243900000000003, 2024-09-17T22:41:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073762719999999997, 2024-09-17T22:41:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106987 2024-09-17T22:41:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270172+0000", 2024-09-17T22:41:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270172+0000", 2024-09-17T22:41:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270172+0000", 2024-09-17T22:41:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2700000000000001e-07 2024-09-17T22:41:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270173+0000", 2024-09-17T22:41:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.145e-06 2024-09-17T22:41:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270176+0000", 2024-09-17T22:41:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8619999999999999e-06 2024-09-17T22:41:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270178+0000", 2024-09-17T22:41:44.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6029999999999999e-06 2024-09-17T22:41:44.793 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.793 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.793 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.793 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106798 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.793 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.236079+0000", 2024-09-17T22:41:44.793 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.034345002999999999, 2024-09-17T22:41:44.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041522699000000003, 2024-09-17T22:41:44.793 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.793 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.793 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.793 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.793 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.793 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.794 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106798 2024-09-17T22:41:44.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.794 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236079+0000", 2024-09-17T22:41:44.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236079+0000", 2024-09-17T22:41:44.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236079+0000", 2024-09-17T22:41:44.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:41:44.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236081+0000", 2024-09-17T22:41:44.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0660000000000002e-06 2024-09-17T22:41:44.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236088+0000", 2024-09-17T22:41:44.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0639999999999999e-06 2024-09-17T22:41:44.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236093+0000", 2024-09-17T22:41:44.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0390000000000004e-06 2024-09-17T22:41:44.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.796 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.796 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106830 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.796 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.242149+0000", 2024-09-17T22:41:44.796 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.028274814999999998, 2024-09-17T22:41:44.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.035504668000000003, 2024-09-17T22:41:44.796 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.796 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.796 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.797 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.797 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.797 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.797 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106830 2024-09-17T22:41:44.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.797 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242149+0000", 2024-09-17T22:41:44.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242149+0000", 2024-09-17T22:41:44.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242149+0000", 2024-09-17T22:41:44.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3200000000000001e-07 2024-09-17T22:41:44.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242151+0000", 2024-09-17T22:41:44.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.863e-06 2024-09-17T22:41:44.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242157+0000", 2024-09-17T22:41:44.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5999999999999997e-06 2024-09-17T22:41:44.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242162+0000", 2024-09-17T22:41:44.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6269999999999999e-06 2024-09-17T22:41:44.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.799 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.799 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106862 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.799 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.248132+0000", 2024-09-17T22:41:44.799 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.022291992, 2024-09-17T22:41:44.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029574064000000001, 2024-09-17T22:41:44.800 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.800 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.800 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.800 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106862 2024-09-17T22:41:44.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.800 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248132+0000", 2024-09-17T22:41:44.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248132+0000", 2024-09-17T22:41:44.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248132+0000", 2024-09-17T22:41:44.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2e-07 2024-09-17T22:41:44.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248134+0000", 2024-09-17T22:41:44.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0059999999999999e-06 2024-09-17T22:41:44.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248143+0000", 2024-09-17T22:41:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1340000000000004e-06 2024-09-17T22:41:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248148+0000", 2024-09-17T22:41:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1320000000000002e-06 2024-09-17T22:41:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106894 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.252332+0000", 2024-09-17T22:41:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018092225, 2024-09-17T22:41:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025426639000000001, 2024-09-17T22:41:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106894 2024-09-17T22:41:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252332+0000", 2024-09-17T22:41:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252332+0000", 2024-09-17T22:41:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252332+0000", 2024-09-17T22:41:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8200000000000001e-07 2024-09-17T22:41:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252334+0000", 2024-09-17T22:41:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-09-17T22:41:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252343+0000", 2024-09-17T22:41:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6799999999999999e-06 2024-09-17T22:41:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252349+0000", 2024-09-17T22:41:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4509999999999997e-06 2024-09-17T22:41:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106924 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.258148+0000", 2024-09-17T22:41:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.012275698999999999, 2024-09-17T22:41:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019663428, 2024-09-17T22:41:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106924 2024-09-17T22:41:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258148+0000", 2024-09-17T22:41:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258148+0000", 2024-09-17T22:41:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258149+0000", 2024-09-17T22:41:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0699999999999998e-07 2024-09-17T22:41:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258151+0000", 2024-09-17T22:41:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0090000000000002e-06 2024-09-17T22:41:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258157+0000", 2024-09-17T22:41:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0000000000000002e-06 2024-09-17T22:41:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258162+0000", 2024-09-17T22:41:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8810000000000004e-06 2024-09-17T22:41:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106956 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.266086+0000", 2024-09-17T22:41:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0043383069999999996, 2024-09-17T22:41:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011781811999999999, 2024-09-17T22:41:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106956 2024-09-17T22:41:44.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266086+0000", 2024-09-17T22:41:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266086+0000", 2024-09-17T22:41:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266086+0000", 2024-09-17T22:41:44.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:41:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266088+0000", 2024-09-17T22:41:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8050000000000001e-06 2024-09-17T22:41:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266092+0000", 2024-09-17T22:41:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9949999999999999e-06 2024-09-17T22:41:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266095+0000", 2024-09-17T22:41:44.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9239999999999999e-06 2024-09-17T22:41:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106988 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.270181+0000", 2024-09-17T22:41:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.000242725, 2024-09-17T22:41:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0077392340000000002, 2024-09-17T22:41:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.812 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106988 2024-09-17T22:41:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270181+0000", 2024-09-17T22:41:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270181+0000", 2024-09-17T22:41:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270181+0000", 2024-09-17T22:41:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.04e-07 2024-09-17T22:41:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270183+0000", 2024-09-17T22:41:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0759999999999999e-06 2024-09-17T22:41:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270185+0000", 2024-09-17T22:41:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8839999999999998e-06 2024-09-17T22:41:44.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270188+0000", 2024-09-17T22:41:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6759999999999999e-06 2024-09-17T22:41:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106799 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.236099+0000", 2024-09-17T22:41:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.034324873999999998, 2024-09-17T22:41:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041874524000000003, 2024-09-17T22:41:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.815 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106799 2024-09-17T22:41:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236099+0000", 2024-09-17T22:41:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236099+0000", 2024-09-17T22:41:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236099+0000", 2024-09-17T22:41:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:41:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236101+0000", 2024-09-17T22:41:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.849e-06 2024-09-17T22:41:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236113+0000", 2024-09-17T22:41:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1345999999999999e-05 2024-09-17T22:41:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236118+0000", 2024-09-17T22:41:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9760000000000003e-06 2024-09-17T22:41:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106831 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.242167+0000", 2024-09-17T22:41:44.818 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.028256874000000001, 2024-09-17T22:41:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.035862167, 2024-09-17T22:41:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106831 2024-09-17T22:41:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242167+0000", 2024-09-17T22:41:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242167+0000", 2024-09-17T22:41:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242167+0000", 2024-09-17T22:41:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1800000000000002e-07 2024-09-17T22:41:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242169+0000", 2024-09-17T22:41:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7269999999999999e-06 2024-09-17T22:41:44.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242175+0000", 2024-09-17T22:41:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.682e-06 2024-09-17T22:41:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242180+0000", 2024-09-17T22:41:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8319999999999996e-06 2024-09-17T22:41:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106863 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.248158+0000", 2024-09-17T22:41:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.022266240999999999, 2024-09-17T22:41:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029927404000000001, 2024-09-17T22:41:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.822 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106863 2024-09-17T22:41:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248158+0000", 2024-09-17T22:41:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248158+0000", 2024-09-17T22:41:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248158+0000", 2024-09-17T22:41:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2899999999999999e-07 2024-09-17T22:41:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248160+0000", 2024-09-17T22:41:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0109999999999999e-06 2024-09-17T22:41:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248167+0000", 2024-09-17T22:41:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1949999999999997e-06 2024-09-17T22:41:44.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248172+0000", 2024-09-17T22:41:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0139999999999998e-06 2024-09-17T22:41:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106895 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.252355+0000", 2024-09-17T22:41:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018068638000000001, 2024-09-17T22:41:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025778170999999999, 2024-09-17T22:41:44.825 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106895 2024-09-17T22:41:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252355+0000", 2024-09-17T22:41:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252355+0000", 2024-09-17T22:41:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252356+0000", 2024-09-17T22:41:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1900000000000003e-07 2024-09-17T22:41:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252358+0000", 2024-09-17T22:41:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9570000000000001e-06 2024-09-17T22:41:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252364+0000", 2024-09-17T22:41:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5019999999999998e-06 2024-09-17T22:41:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252370+0000", 2024-09-17T22:41:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6999999999999996e-06 2024-09-17T22:41:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.828 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106925 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.258168+0000", 2024-09-17T22:41:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.012256240999999999, 2024-09-17T22:41:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020019706000000002, 2024-09-17T22:41:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106925 2024-09-17T22:41:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258168+0000", 2024-09-17T22:41:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258168+0000", 2024-09-17T22:41:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258168+0000", 2024-09-17T22:41:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8799999999999998e-07 2024-09-17T22:41:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258170+0000", 2024-09-17T22:41:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1239999999999999e-06 2024-09-17T22:41:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258176+0000", 2024-09-17T22:41:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0680000000000002e-06 2024-09-17T22:41:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258181+0000", 2024-09-17T22:41:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9019999999999998e-06 2024-09-17T22:41:44.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106957 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.266099+0000", 2024-09-17T22:41:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0043250989999999998, 2024-09-17T22:41:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012144654, 2024-09-17T22:41:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.832 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106957 2024-09-17T22:41:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266099+0000", 2024-09-17T22:41:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266099+0000", 2024-09-17T22:41:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266099+0000", 2024-09-17T22:41:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7000000000000001e-07 2024-09-17T22:41:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266100+0000", 2024-09-17T22:41:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1999999999999999e-06 2024-09-17T22:41:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266104+0000", 2024-09-17T22:41:44.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2310000000000001e-06 2024-09-17T22:41:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266106+0000", 2024-09-17T22:41:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6709999999999999e-06 2024-09-17T22:41:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106989 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.270191+0000", 2024-09-17T22:41:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00023304, 2024-09-17T22:41:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0081056099999999992, 2024-09-17T22:41:44.835 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106989 2024-09-17T22:41:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270191+0000", 2024-09-17T22:41:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270191+0000", 2024-09-17T22:41:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270191+0000", 2024-09-17T22:41:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:41:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270192+0000", 2024-09-17T22:41:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.006e-06 2024-09-17T22:41:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270195+0000", 2024-09-17T22:41:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8849999999999999e-06 2024-09-17T22:41:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270198+0000", 2024-09-17T22:41:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.638e-06 2024-09-17T22:41:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106800 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.236125+0000", 2024-09-17T22:41:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.034298649, 2024-09-17T22:41:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042230500999999997, 2024-09-17T22:41:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106800 2024-09-17T22:41:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.839 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236125+0000", 2024-09-17T22:41:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236125+0000", 2024-09-17T22:41:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236126+0000", 2024-09-17T22:41:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.84e-07 2024-09-17T22:41:44.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236128+0000", 2024-09-17T22:41:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1409999999999999e-06 2024-09-17T22:41:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236134+0000", 2024-09-17T22:41:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.224e-06 2024-09-17T22:41:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.236139+0000", 2024-09-17T22:41:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8080000000000004e-06 2024-09-17T22:41:44.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106832 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.242185+0000", 2024-09-17T22:41:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.028238866000000001, 2024-09-17T22:41:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036226492999999999, 2024-09-17T22:41:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106832 2024-09-17T22:41:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242185+0000", 2024-09-17T22:41:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242185+0000", 2024-09-17T22:41:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242185+0000", 2024-09-17T22:41:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2599999999999998e-07 2024-09-17T22:41:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242192+0000", 2024-09-17T22:41:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1160000000000004e-06 2024-09-17T22:41:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242198+0000", 2024-09-17T22:41:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0100000000000001e-06 2024-09-17T22:41:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242203+0000", 2024-09-17T22:41:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9570000000000001e-06 2024-09-17T22:41:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106864 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.248194+0000", 2024-09-17T22:41:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.022230427000000001, 2024-09-17T22:41:44.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030280476000000001, 2024-09-17T22:41:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106864 2024-09-17T22:41:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248194+0000", 2024-09-17T22:41:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248194+0000", 2024-09-17T22:41:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248194+0000", 2024-09-17T22:41:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3300000000000002e-07 2024-09-17T22:41:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248196+0000", 2024-09-17T22:41:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1639999999999999e-06 2024-09-17T22:41:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248203+0000", 2024-09-17T22:41:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3779999999999998e-06 2024-09-17T22:41:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248209+0000", 2024-09-17T22:41:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2389999999999999e-06 2024-09-17T22:41:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.848 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106896 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.252376+0000", 2024-09-17T22:41:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018048260999999999, 2024-09-17T22:41:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026156028000000001, 2024-09-17T22:41:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106896 2024-09-17T22:41:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252376+0000", 2024-09-17T22:41:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252376+0000", 2024-09-17T22:41:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252376+0000", 2024-09-17T22:41:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:41:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252378+0000", 2024-09-17T22:41:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9810000000000002e-06 2024-09-17T22:41:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252386+0000", 2024-09-17T22:41:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0800000000000006e-06 2024-09-17T22:41:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252392+0000", 2024-09-17T22:41:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0769999999999996e-06 2024-09-17T22:41:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.851 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106926 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.258188+0000", 2024-09-17T22:41:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.012236423999999999, 2024-09-17T22:41:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0204025, 2024-09-17T22:41:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106926 2024-09-17T22:41:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.852 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258188+0000", 2024-09-17T22:41:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258188+0000", 2024-09-17T22:41:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258188+0000", 2024-09-17T22:41:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.15e-07 2024-09-17T22:41:44.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258190+0000", 2024-09-17T22:41:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1380000000000001e-06 2024-09-17T22:41:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258196+0000", 2024-09-17T22:41:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9190000000000004e-06 2024-09-17T22:41:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258201+0000", 2024-09-17T22:41:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.741e-06 2024-09-17T22:41:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.854 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106958 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.266109+0000", 2024-09-17T22:41:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.004314638, 2024-09-17T22:41:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012535543, 2024-09-17T22:41:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106958 2024-09-17T22:41:44.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266109+0000", 2024-09-17T22:41:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266109+0000", 2024-09-17T22:41:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266110+0000", 2024-09-17T22:41:44.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3999999999999998e-07 2024-09-17T22:41:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266111+0000", 2024-09-17T22:41:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-09-17T22:41:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266114+0000", 2024-09-17T22:41:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.004e-06 2024-09-17T22:41:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266116+0000", 2024-09-17T22:41:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5790000000000002e-06 2024-09-17T22:41:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106990 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.270201+0000", 2024-09-17T22:41:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00022344800000000001, 2024-09-17T22:41:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0085026709999999998, 2024-09-17T22:41:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.858 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106990 2024-09-17T22:41:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270201+0000", 2024-09-17T22:41:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.270201+0000", 2024-09-17T22:41:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272076+0000", 2024-09-17T22:41:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001874978 2024-09-17T22:41:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272079+0000", 2024-09-17T22:41:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9750000000000001e-06 2024-09-17T22:41:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272090+0000", 2024-09-17T22:41:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0553999999999999e-05 2024-09-17T22:41:44.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272097+0000", 2024-09-17T22:41:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0190000000000001e-06 2024-09-17T22:41:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106833 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.242210+0000", 2024-09-17T22:41:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.028214080999999998, 2024-09-17T22:41:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036550544999999997, 2024-09-17T22:41:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.861 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106833 2024-09-17T22:41:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242210+0000", 2024-09-17T22:41:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242210+0000", 2024-09-17T22:41:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242210+0000", 2024-09-17T22:41:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-09-17T22:41:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242212+0000", 2024-09-17T22:41:44.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9470000000000002e-06 2024-09-17T22:41:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242221+0000", 2024-09-17T22:41:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9029999999999993e-06 2024-09-17T22:41:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242226+0000", 2024-09-17T22:41:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8400000000000002e-06 2024-09-17T22:41:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106865 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.248215+0000", 2024-09-17T22:41:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.022209258999999999, 2024-09-17T22:41:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030605034999999999, 2024-09-17T22:41:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106865 2024-09-17T22:41:44.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248215+0000", 2024-09-17T22:41:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248215+0000", 2024-09-17T22:41:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248215+0000", 2024-09-17T22:41:44.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.51e-07 2024-09-17T22:41:44.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248217+0000", 2024-09-17T22:41:44.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9709999999999998e-06 2024-09-17T22:41:44.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248224+0000", 2024-09-17T22:41:44.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6769999999999998e-06 2024-09-17T22:41:44.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248230+0000", 2024-09-17T22:41:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0020000000000003e-06 2024-09-17T22:41:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106897 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.252398+0000", 2024-09-17T22:41:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.018026022999999999, 2024-09-17T22:41:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026477482, 2024-09-17T22:41:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.868 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106897 2024-09-17T22:41:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252398+0000", 2024-09-17T22:41:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252398+0000", 2024-09-17T22:41:44.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252398+0000", 2024-09-17T22:41:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1e-07 2024-09-17T22:41:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252400+0000", 2024-09-17T22:41:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9769999999999999e-06 2024-09-17T22:41:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252412+0000", 2024-09-17T22:41:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1953999999999999e-05 2024-09-17T22:41:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252418+0000", 2024-09-17T22:41:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0120000000000002e-06 2024-09-17T22:41:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106927 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.871 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.258206+0000", 2024-09-17T22:41:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.012217591, 2024-09-17T22:41:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020726708, 2024-09-17T22:41:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106927 2024-09-17T22:41:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258206+0000", 2024-09-17T22:41:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258206+0000", 2024-09-17T22:41:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258207+0000", 2024-09-17T22:41:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7500000000000001e-07 2024-09-17T22:41:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258214+0000", 2024-09-17T22:41:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9160000000000004e-06 2024-09-17T22:41:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258220+0000", 2024-09-17T22:41:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0850000000000002e-06 2024-09-17T22:41:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258225+0000", 2024-09-17T22:41:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9529999999999998e-06 2024-09-17T22:41:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106959 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.266120+0000", 2024-09-17T22:41:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0043041909999999997, 2024-09-17T22:41:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012870220999999999, 2024-09-17T22:41:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106959 2024-09-17T22:41:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266120+0000", 2024-09-17T22:41:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266120+0000", 2024-09-17T22:41:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266120+0000", 2024-09-17T22:41:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2999999999999999e-07 2024-09-17T22:41:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266121+0000", 2024-09-17T22:41:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.24e-06 2024-09-17T22:41:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266124+0000", 2024-09-17T22:41:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9859999999999999e-06 2024-09-17T22:41:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266127+0000", 2024-09-17T22:41:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.683e-06 2024-09-17T22:41:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106991 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.272109+0000", 2024-09-17T22:41:44.878 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9983153, 2024-09-17T22:41:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0069478129999999997, 2024-09-17T22:41:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106991 2024-09-17T22:41:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272109+0000", 2024-09-17T22:41:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272109+0000", 2024-09-17T22:41:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272109+0000", 2024-09-17T22:41:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.75e-07 2024-09-17T22:41:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272112+0000", 2024-09-17T22:41:44.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.593e-06 2024-09-17T22:41:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272119+0000", 2024-09-17T22:41:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.815e-06 2024-09-17T22:41:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272124+0000", 2024-09-17T22:41:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.293e-06 2024-09-17T22:41:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106834 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.242233+0000", 2024-09-17T22:41:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.02819147, 2024-09-17T22:41:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036884330999999999, 2024-09-17T22:41:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106834 2024-09-17T22:41:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242233+0000", 2024-09-17T22:41:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242233+0000", 2024-09-17T22:41:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242233+0000", 2024-09-17T22:41:44.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9499999999999998e-07 2024-09-17T22:41:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242235+0000", 2024-09-17T22:41:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1509999999999998e-06 2024-09-17T22:41:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242242+0000", 2024-09-17T22:41:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9229999999999996e-06 2024-09-17T22:41:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242247+0000", 2024-09-17T22:41:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9729999999999997e-06 2024-09-17T22:41:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106866 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.248236+0000", 2024-09-17T22:41:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.022187872000000001, 2024-09-17T22:41:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030937289999999999, 2024-09-17T22:41:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.885 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106866 2024-09-17T22:41:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248236+0000", 2024-09-17T22:41:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248236+0000", 2024-09-17T22:41:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248236+0000", 2024-09-17T22:41:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.41e-07 2024-09-17T22:41:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248238+0000", 2024-09-17T22:41:44.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8369999999999999e-06 2024-09-17T22:41:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248245+0000", 2024-09-17T22:41:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0249999999999997e-06 2024-09-17T22:41:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248250+0000", 2024-09-17T22:41:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0250000000000002e-06 2024-09-17T22:41:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106898 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.252425+0000", 2024-09-17T22:41:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.017998510999999998, 2024-09-17T22:41:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026806478000000002, 2024-09-17T22:41:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106898 2024-09-17T22:41:44.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252425+0000", 2024-09-17T22:41:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252425+0000", 2024-09-17T22:41:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252426+0000", 2024-09-17T22:41:44.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:41:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252428+0000", 2024-09-17T22:41:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9520000000000001e-06 2024-09-17T22:41:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252436+0000", 2024-09-17T22:41:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3999999999999992e-06 2024-09-17T22:41:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252446+0000", 2024-09-17T22:41:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.482e-06 2024-09-17T22:41:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106928 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.258231+0000", 2024-09-17T22:41:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.012192962, 2024-09-17T22:41:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021056424000000001, 2024-09-17T22:41:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.892 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106928 2024-09-17T22:41:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258231+0000", 2024-09-17T22:41:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258231+0000", 2024-09-17T22:41:44.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258231+0000", 2024-09-17T22:41:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.27e-07 2024-09-17T22:41:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258233+0000", 2024-09-17T22:41:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9649999999999998e-06 2024-09-17T22:41:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258239+0000", 2024-09-17T22:41:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6890000000000001e-06 2024-09-17T22:41:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258244+0000", 2024-09-17T22:41:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7350000000000004e-06 2024-09-17T22:41:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106960 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.266130+0000", 2024-09-17T22:41:44.895 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0042941669999999998, 2024-09-17T22:41:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013216749999999999, 2024-09-17T22:41:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106960 2024-09-17T22:41:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266130+0000", 2024-09-17T22:41:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266130+0000", 2024-09-17T22:41:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266130+0000", 2024-09-17T22:41:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0599999999999999e-07 2024-09-17T22:41:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266131+0000", 2024-09-17T22:41:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.105e-06 2024-09-17T22:41:44.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266134+0000", 2024-09-17T22:41:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9569999999999998e-06 2024-09-17T22:41:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266137+0000", 2024-09-17T22:41:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5560000000000002e-06 2024-09-17T22:41:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.898 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106992 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.272132+0000", 2024-09-17T22:41:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9982924, 2024-09-17T22:41:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.007272021, 2024-09-17T22:41:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106992 2024-09-17T22:41:44.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272132+0000", 2024-09-17T22:41:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272132+0000", 2024-09-17T22:41:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272132+0000", 2024-09-17T22:41:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:41:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272134+0000", 2024-09-17T22:41:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3690000000000001e-06 2024-09-17T22:41:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272142+0000", 2024-09-17T22:41:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3010000000000001e-06 2024-09-17T22:41:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272145+0000", 2024-09-17T22:41:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.693e-06 2024-09-17T22:41:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.902 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106835 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.242253+0000", 2024-09-17T22:41:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.028170842000000001, 2024-09-17T22:41:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037209244000000002, 2024-09-17T22:41:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106835 2024-09-17T22:41:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242253+0000", 2024-09-17T22:41:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242253+0000", 2024-09-17T22:41:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242253+0000", 2024-09-17T22:41:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-09-17T22:41:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242255+0000", 2024-09-17T22:41:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.967e-06 2024-09-17T22:41:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242261+0000", 2024-09-17T22:41:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4240000000000001e-06 2024-09-17T22:41:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242266+0000", 2024-09-17T22:41:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6750000000000001e-06 2024-09-17T22:41:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106867 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.248256+0000", 2024-09-17T22:41:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.022168041999999999, 2024-09-17T22:41:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031263086000000002, 2024-09-17T22:41:44.906 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106867 2024-09-17T22:41:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248256+0000", 2024-09-17T22:41:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248256+0000", 2024-09-17T22:41:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248256+0000", 2024-09-17T22:41:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-09-17T22:41:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248262+0000", 2024-09-17T22:41:44.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.942e-06 2024-09-17T22:41:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248270+0000", 2024-09-17T22:41:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4279999999999999e-06 2024-09-17T22:41:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248275+0000", 2024-09-17T22:41:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6189999999999999e-06 2024-09-17T22:41:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.909 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106899 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.252453+0000", 2024-09-17T22:41:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.017970976999999999, 2024-09-17T22:41:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027125219999999998, 2024-09-17T22:41:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.910 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106899 2024-09-17T22:41:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252453+0000", 2024-09-17T22:41:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252453+0000", 2024-09-17T22:41:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252453+0000", 2024-09-17T22:41:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:41:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252456+0000", 2024-09-17T22:41:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099999999999999e-06 2024-09-17T22:41:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252462+0000", 2024-09-17T22:41:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5080000000000002e-06 2024-09-17T22:41:44.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252468+0000", 2024-09-17T22:41:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4550000000000003e-06 2024-09-17T22:41:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106929 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.258250+0000", 2024-09-17T22:41:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.012174312, 2024-09-17T22:41:44.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021388708999999999, 2024-09-17T22:41:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106929 2024-09-17T22:41:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258250+0000", 2024-09-17T22:41:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258250+0000", 2024-09-17T22:41:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258250+0000", 2024-09-17T22:41:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-09-17T22:41:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258252+0000", 2024-09-17T22:41:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.844e-06 2024-09-17T22:41:44.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258257+0000", 2024-09-17T22:41:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5470000000000004e-06 2024-09-17T22:41:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258262+0000", 2024-09-17T22:41:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8199999999999996e-06 2024-09-17T22:41:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106961 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.266139+0000", 2024-09-17T22:41:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0042846079999999996, 2024-09-17T22:41:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013565163, 2024-09-17T22:41:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106961 2024-09-17T22:41:44.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266139+0000", 2024-09-17T22:41:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266139+0000", 2024-09-17T22:41:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266140+0000", 2024-09-17T22:41:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.16e-07 2024-09-17T22:41:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266141+0000", 2024-09-17T22:41:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.15e-06 2024-09-17T22:41:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266144+0000", 2024-09-17T22:41:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.869e-06 2024-09-17T22:41:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266146+0000", 2024-09-17T22:41:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5119999999999998e-06 2024-09-17T22:41:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106993 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.272150+0000", 2024-09-17T22:41:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9982738, 2024-09-17T22:41:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076080310000000003, 2024-09-17T22:41:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106993 2024-09-17T22:41:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272150+0000", 2024-09-17T22:41:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272150+0000", 2024-09-17T22:41:44.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272150+0000", 2024-09-17T22:41:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3200000000000001e-07 2024-09-17T22:41:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272152+0000", 2024-09-17T22:41:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4500000000000001e-06 2024-09-17T22:41:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272155+0000", 2024-09-17T22:41:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.45e-06 2024-09-17T22:41:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272158+0000", 2024-09-17T22:41:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.627e-06 2024-09-17T22:41:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106836 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.242293+0000", 2024-09-17T22:41:44.923 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.028131008999999998, 2024-09-17T22:41:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037518762999999997, 2024-09-17T22:41:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106836 2024-09-17T22:41:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242293+0000", 2024-09-17T22:41:44.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242293+0000", 2024-09-17T22:41:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242293+0000", 2024-09-17T22:41:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1e-07 2024-09-17T22:41:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242295+0000", 2024-09-17T22:41:44.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9420000000000002e-06 2024-09-17T22:41:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242314+0000", 2024-09-17T22:41:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8919999999999998e-05 2024-09-17T22:41:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242317+0000", 2024-09-17T22:41:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2959999999999999e-06 2024-09-17T22:41:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106868 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.248282+0000", 2024-09-17T22:41:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.022141833999999999, 2024-09-17T22:41:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031578189999999999, 2024-09-17T22:41:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106868 2024-09-17T22:41:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248282+0000", 2024-09-17T22:41:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248282+0000", 2024-09-17T22:41:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248283+0000", 2024-09-17T22:41:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-07 2024-09-17T22:41:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248284+0000", 2024-09-17T22:41:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6899999999999999e-06 2024-09-17T22:41:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248291+0000", 2024-09-17T22:41:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0879999999999997e-06 2024-09-17T22:41:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248296+0000", 2024-09-17T22:41:44.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0509999999999996e-06 2024-09-17T22:41:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106900 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.252474+0000", 2024-09-17T22:41:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.017949722000000001, 2024-09-17T22:41:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027438105000000001, 2024-09-17T22:41:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.930 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106900 2024-09-17T22:41:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252474+0000", 2024-09-17T22:41:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252474+0000", 2024-09-17T22:41:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252475+0000", 2024-09-17T22:41:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3499999999999996e-07 2024-09-17T22:41:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252477+0000", 2024-09-17T22:41:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.181e-06 2024-09-17T22:41:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252483+0000", 2024-09-17T22:41:44.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4280000000000001e-06 2024-09-17T22:41:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252489+0000", 2024-09-17T22:41:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8370000000000002e-06 2024-09-17T22:41:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106930 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.258268+0000", 2024-09-17T22:41:44.933 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.012155914, 2024-09-17T22:41:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021691574000000002, 2024-09-17T22:41:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106930 2024-09-17T22:41:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258268+0000", 2024-09-17T22:41:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.258268+0000", 2024-09-17T22:41:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260049+0000", 2024-09-17T22:41:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017810530000000001 2024-09-17T22:41:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260053+0000", 2024-09-17T22:41:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0659999999999997e-06 2024-09-17T22:41:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260062+0000", 2024-09-17T22:41:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8990000000000007e-06 2024-09-17T22:41:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260067+0000", 2024-09-17T22:41:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0309999999999998e-06 2024-09-17T22:41:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.936 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106962 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.266152+0000", 2024-09-17T22:41:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0042723759999999996, 2024-09-17T22:41:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013864015, 2024-09-17T22:41:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.937 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106962 2024-09-17T22:41:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266152+0000", 2024-09-17T22:41:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266152+0000", 2024-09-17T22:41:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266152+0000", 2024-09-17T22:41:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3300000000000001e-07 2024-09-17T22:41:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266153+0000", 2024-09-17T22:41:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.217e-06 2024-09-17T22:41:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266156+0000", 2024-09-17T22:41:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0879999999999999e-06 2024-09-17T22:41:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266159+0000", 2024-09-17T22:41:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6759999999999999e-06 2024-09-17T22:41:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106994 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.272161+0000", 2024-09-17T22:41:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9982629, 2024-09-17T22:41:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079170969999999997, 2024-09-17T22:41:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.940 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106994 2024-09-17T22:41:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272161+0000", 2024-09-17T22:41:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272161+0000", 2024-09-17T22:41:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272161+0000", 2024-09-17T22:41:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1500000000000001e-07 2024-09-17T22:41:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272163+0000", 2024-09-17T22:41:44.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2759999999999999e-06 2024-09-17T22:41:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272166+0000", 2024-09-17T22:41:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.994e-06 2024-09-17T22:41:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272169+0000", 2024-09-17T22:41:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8130000000000002e-06 2024-09-17T22:41:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.943 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106837 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.242323+0000", 2024-09-17T22:41:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.028101279, 2024-09-17T22:41:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037812907999999999, 2024-09-17T22:41:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.944 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106837 2024-09-17T22:41:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242323+0000", 2024-09-17T22:41:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242323+0000", 2024-09-17T22:41:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242323+0000", 2024-09-17T22:41:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3099999999999999e-07 2024-09-17T22:41:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242325+0000", 2024-09-17T22:41:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-09-17T22:41:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242328+0000", 2024-09-17T22:41:44.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.123e-06 2024-09-17T22:41:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242330+0000", 2024-09-17T22:41:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6790000000000001e-06 2024-09-17T22:41:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106869 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.248302+0000", 2024-09-17T22:41:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.022122175000000001, 2024-09-17T22:41:44.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031891586999999999, 2024-09-17T22:41:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106869 2024-09-17T22:41:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248302+0000", 2024-09-17T22:41:44.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248302+0000", 2024-09-17T22:41:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248302+0000", 2024-09-17T22:41:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1899999999999998e-07 2024-09-17T22:41:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248304+0000", 2024-09-17T22:41:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8819999999999999e-06 2024-09-17T22:41:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248310+0000", 2024-09-17T22:41:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9150000000000001e-06 2024-09-17T22:41:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248316+0000", 2024-09-17T22:41:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7200000000000003e-06 2024-09-17T22:41:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.950 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106901 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.252495+0000", 2024-09-17T22:41:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.017928606999999999, 2024-09-17T22:41:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027752042000000001, 2024-09-17T22:41:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.951 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106901 2024-09-17T22:41:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252495+0000", 2024-09-17T22:41:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.252495+0000", 2024-09-17T22:41:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254062+0000", 2024-09-17T22:41:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001566675 2024-09-17T22:41:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254067+0000", 2024-09-17T22:41:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-09-17T22:41:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254075+0000", 2024-09-17T22:41:44.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4330000000000004e-06 2024-09-17T22:41:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254082+0000", 2024-09-17T22:41:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5769999999999999e-06 2024-09-17T22:41:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106931 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.260077+0000", 2024-09-17T22:41:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.010346655, 2024-09-17T22:41:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020230060000000001, 2024-09-17T22:41:44.954 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106931 2024-09-17T22:41:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260077+0000", 2024-09-17T22:41:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260077+0000", 2024-09-17T22:41:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260078+0000", 2024-09-17T22:41:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1300000000000001e-07 2024-09-17T22:41:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260079+0000", 2024-09-17T22:41:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5340000000000001e-06 2024-09-17T22:41:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260084+0000", 2024-09-17T22:41:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1030000000000001e-06 2024-09-17T22:41:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260087+0000", 2024-09-17T22:41:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.067e-06 2024-09-17T22:41:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106963 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.266165+0000", 2024-09-17T22:41:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0042588289999999996, 2024-09-17T22:41:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014202566, 2024-09-17T22:41:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106963 2024-09-17T22:41:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.958 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266165+0000", 2024-09-17T22:41:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266165+0000", 2024-09-17T22:41:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266165+0000", 2024-09-17T22:41:44.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2999999999999999e-07 2024-09-17T22:41:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266167+0000", 2024-09-17T22:41:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.209e-06 2024-09-17T22:41:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266170+0000", 2024-09-17T22:41:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2679999999999999e-06 2024-09-17T22:41:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266172+0000", 2024-09-17T22:41:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5849999999999998e-06 2024-09-17T22:41:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106995 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.272181+0000", 2024-09-17T22:41:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9982424, 2024-09-17T22:41:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0082609339999999993, 2024-09-17T22:41:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.961 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106995 2024-09-17T22:41:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272181+0000", 2024-09-17T22:41:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272181+0000", 2024-09-17T22:41:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272182+0000", 2024-09-17T22:41:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:41:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272183+0000", 2024-09-17T22:41:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.449e-06 2024-09-17T22:41:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272187+0000", 2024-09-17T22:41:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.58e-06 2024-09-17T22:41:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272190+0000", 2024-09-17T22:41:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.813e-06 2024-09-17T22:41:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.964 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106838 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.242333+0000", 2024-09-17T22:41:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.028090698000000001, 2024-09-17T22:41:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038169502000000001, 2024-09-17T22:41:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106838 2024-09-17T22:41:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242333+0000", 2024-09-17T22:41:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.242333+0000", 2024-09-17T22:41:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244050+0000", 2024-09-17T22:41:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017170670000000001 2024-09-17T22:41:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244055+0000", 2024-09-17T22:41:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4549999999999997e-06 2024-09-17T22:41:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244066+0000", 2024-09-17T22:41:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1080000000000001e-05 2024-09-17T22:41:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244072+0000", 2024-09-17T22:41:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.229e-06 2024-09-17T22:41:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.967 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106870 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.248327+0000", 2024-09-17T22:41:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.022096991, 2024-09-17T22:41:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032234974999999999, 2024-09-17T22:41:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106870 2024-09-17T22:41:44.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248327+0000", 2024-09-17T22:41:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248327+0000", 2024-09-17T22:41:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248327+0000", 2024-09-17T22:41:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7800000000000002e-07 2024-09-17T22:41:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248329+0000", 2024-09-17T22:41:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.888e-06 2024-09-17T22:41:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248336+0000", 2024-09-17T22:41:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3160000000000002e-06 2024-09-17T22:41:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248340+0000", 2024-09-17T22:41:44.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.899e-06 2024-09-17T22:41:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106902 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.254095+0000", 2024-09-17T22:41:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.016329291999999999, 2024-09-17T22:41:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026524280000000001, 2024-09-17T22:41:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.971 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106902 2024-09-17T22:41:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254095+0000", 2024-09-17T22:41:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254095+0000", 2024-09-17T22:41:44.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254095+0000", 2024-09-17T22:41:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6899999999999998e-07 2024-09-17T22:41:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254097+0000", 2024-09-17T22:41:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2919999999999998e-06 2024-09-17T22:41:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254103+0000", 2024-09-17T22:41:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7969999999999997e-06 2024-09-17T22:41:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254109+0000", 2024-09-17T22:41:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2830000000000001e-06 2024-09-17T22:41:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.974 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106932 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.260091+0000", 2024-09-17T22:41:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.010332653000000001, 2024-09-17T22:41:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020586534, 2024-09-17T22:41:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106932 2024-09-17T22:41:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260091+0000", 2024-09-17T22:41:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260091+0000", 2024-09-17T22:41:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260092+0000", 2024-09-17T22:41:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4400000000000001e-07 2024-09-17T22:41:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260093+0000", 2024-09-17T22:41:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3799999999999999e-06 2024-09-17T22:41:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260096+0000", 2024-09-17T22:41:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.174e-06 2024-09-17T22:41:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260100+0000", 2024-09-17T22:41:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4180000000000001e-06 2024-09-17T22:41:44.977 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106964 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.266176+0000", 2024-09-17T22:41:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0042483900000000003, 2024-09-17T22:41:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014557983, 2024-09-17T22:41:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.978 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106964 2024-09-17T22:41:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266176+0000", 2024-09-17T22:41:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266176+0000", 2024-09-17T22:41:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266176+0000", 2024-09-17T22:41:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-09-17T22:41:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266177+0000", 2024-09-17T22:41:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.074e-06 2024-09-17T22:41:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266181+0000", 2024-09-17T22:41:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8399999999999997e-06 2024-09-17T22:41:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266183+0000", 2024-09-17T22:41:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.678e-06 2024-09-17T22:41:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.981 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106996 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.272193+0000", 2024-09-17T22:41:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9982305, 2024-09-17T22:41:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0085993949999999993, 2024-09-17T22:41:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106996 2024-09-17T22:41:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272193+0000", 2024-09-17T22:41:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272193+0000", 2024-09-17T22:41:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272194+0000", 2024-09-17T22:41:44.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:41:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272195+0000", 2024-09-17T22:41:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2079999999999999e-06 2024-09-17T22:41:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272198+0000", 2024-09-17T22:41:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9100000000000001e-06 2024-09-17T22:41:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272200+0000", 2024-09-17T22:41:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.61e-06 2024-09-17T22:41:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106839 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.244084+0000", 2024-09-17T22:41:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.026339482000000001, 2024-09-17T22:41:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036768120000000001, 2024-09-17T22:41:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.985 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106839 2024-09-17T22:41:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244084+0000", 2024-09-17T22:41:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244084+0000", 2024-09-17T22:41:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244085+0000", 2024-09-17T22:41:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6199999999999999e-07 2024-09-17T22:41:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244087+0000", 2024-09-17T22:41:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9199999999999998e-06 2024-09-17T22:41:44.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244095+0000", 2024-09-17T22:41:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0339999999999997e-06 2024-09-17T22:41:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244098+0000", 2024-09-17T22:41:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5300000000000001e-06 2024-09-17T22:41:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106871 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.248347+0000", 2024-09-17T22:41:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.022076760000000001, 2024-09-17T22:41:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032564114999999998, 2024-09-17T22:41:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.989 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106871 2024-09-17T22:41:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248347+0000", 2024-09-17T22:41:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.248347+0000", 2024-09-17T22:41:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250065+0000", 2024-09-17T22:41:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001718048 2024-09-17T22:41:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250070+0000", 2024-09-17T22:41:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1200000000000001e-06 2024-09-17T22:41:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250083+0000", 2024-09-17T22:41:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2357999999999999e-05 2024-09-17T22:41:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250091+0000", 2024-09-17T22:41:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9990000000000008e-06 2024-09-17T22:41:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106903 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.254116+0000", 2024-09-17T22:41:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.016308300000000001, 2024-09-17T22:41:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026851719, 2024-09-17T22:41:44.992 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106903 2024-09-17T22:41:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254116+0000", 2024-09-17T22:41:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254116+0000", 2024-09-17T22:41:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254116+0000", 2024-09-17T22:41:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:41:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254122+0000", 2024-09-17T22:41:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0279999999999997e-06 2024-09-17T22:41:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254129+0000", 2024-09-17T22:41:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.832e-06 2024-09-17T22:41:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254134+0000", 2024-09-17T22:41:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9300000000000002e-06 2024-09-17T22:41:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.995 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106933 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.260103+0000", 2024-09-17T22:41:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.010321027999999999, 2024-09-17T22:41:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020922303999999999, 2024-09-17T22:41:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:44.996 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106933 2024-09-17T22:41:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260103+0000", 2024-09-17T22:41:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260103+0000", 2024-09-17T22:41:44.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260103+0000", 2024-09-17T22:41:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-09-17T22:41:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260104+0000", 2024-09-17T22:41:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.263e-06 2024-09-17T22:41:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:44.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260107+0000", 2024-09-17T22:41:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.976e-06 2024-09-17T22:41:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260110+0000", 2024-09-17T22:41:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6529999999999998e-06 2024-09-17T22:41:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106965 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:44.999 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.266187+0000", 2024-09-17T22:41:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.004237083, 2024-09-17T22:41:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014897726999999999, 2024-09-17T22:41:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106965 2024-09-17T22:41:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:45.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:45.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266187+0000", 2024-09-17T22:41:45.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:45.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.266187+0000", 2024-09-17T22:41:45.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:45.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268058+0000", 2024-09-17T22:41:45.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018708290000000001 2024-09-17T22:41:45.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:45.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268062+0000", 2024-09-17T22:41:45.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9439999999999998e-06 2024-09-17T22:41:45.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:45.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268073+0000", 2024-09-17T22:41:45.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1318000000000001e-05 2024-09-17T22:41:45.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:45.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268080+0000", 2024-09-17T22:41:45.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.311e-06 2024-09-17T22:41:45.003 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.003 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:45.003 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.003 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106997 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:45.003 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.272203+0000", 2024-09-17T22:41:45.003 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9982204, 2024-09-17T22:41:45.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089385360000000004, 2024-09-17T22:41:45.003 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:45.003 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:45.004 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:45.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:45.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:45.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:45.004 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106997 2024-09-17T22:41:45.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.004 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:45.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:45.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272203+0000", 2024-09-17T22:41:45.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:45.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272203+0000", 2024-09-17T22:41:45.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:45.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272204+0000", 2024-09-17T22:41:45.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.04e-07 2024-09-17T22:41:45.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:45.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272205+0000", 2024-09-17T22:41:45.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1260000000000001e-06 2024-09-17T22:41:45.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:45.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272208+0000", 2024-09-17T22:41:45.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8849999999999999e-06 2024-09-17T22:41:45.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:45.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272211+0000", 2024-09-17T22:41:45.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0070000000000002e-06 2024-09-17T22:41:45.006 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.006 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:45.006 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.006 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106840 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:45.006 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.244103+0000", 2024-09-17T22:41:45.007 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.026321006000000001, 2024-09-17T22:41:45.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037099339000000002, 2024-09-17T22:41:45.007 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:45.007 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:45.007 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:45.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:45.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:45.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:45.007 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106840 2024-09-17T22:41:45.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.007 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:45.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:45.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244103+0000", 2024-09-17T22:41:45.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:45.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244103+0000", 2024-09-17T22:41:45.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:45.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244103+0000", 2024-09-17T22:41:45.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7399999999999999e-07 2024-09-17T22:41:45.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:45.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244105+0000", 2024-09-17T22:41:45.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3319999999999999e-06 2024-09-17T22:41:45.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:45.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244108+0000", 2024-09-17T22:41:45.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2430000000000001e-06 2024-09-17T22:41:45.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:45.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244110+0000", 2024-09-17T22:41:45.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-17T22:41:45.009 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.009 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:45.009 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.010 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106872 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:45.010 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.250105+0000", 2024-09-17T22:41:45.010 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020319467000000001, 2024-09-17T22:41:45.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031153578000000001, 2024-09-17T22:41:45.010 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:45.010 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:45.010 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:45.010 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:45.010 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:45.010 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:45.010 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106872 2024-09-17T22:41:45.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.010 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:45.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:45.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250105+0000", 2024-09-17T22:41:45.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:45.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250105+0000", 2024-09-17T22:41:45.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:45.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250105+0000", 2024-09-17T22:41:45.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4600000000000005e-07 2024-09-17T22:41:45.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:45.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250107+0000", 2024-09-17T22:41:45.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.367e-06 2024-09-17T22:41:45.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:45.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250136+0000", 2024-09-17T22:41:45.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8481999999999999e-05 2024-09-17T22:41:45.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:45.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250142+0000", 2024-09-17T22:41:45.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9359999999999996e-06 2024-09-17T22:41:45.012 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.013 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:45.013 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.013 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106904 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:45.013 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.254141+0000", 2024-09-17T22:41:45.013 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.016283382999999998, 2024-09-17T22:41:45.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027176154000000001, 2024-09-17T22:41:45.013 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:45.013 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:45.013 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:45.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:45.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:45.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:45.013 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106904 2024-09-17T22:41:45.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.014 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:45.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:45.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254141+0000", 2024-09-17T22:41:45.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:45.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254141+0000", 2024-09-17T22:41:45.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:45.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254141+0000", 2024-09-17T22:41:45.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:41:45.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:45.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254143+0000", 2024-09-17T22:41:45.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1019999999999999e-06 2024-09-17T22:41:45.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:45.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254160+0000", 2024-09-17T22:41:45.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7323e-05 2024-09-17T22:41:45.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:45.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254166+0000", 2024-09-17T22:41:45.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6049999999999997e-06 2024-09-17T22:41:45.016 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.016 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:45.016 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.016 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106934 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:45.016 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.260113+0000", 2024-09-17T22:41:45.016 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.010310915, 2024-09-17T22:41:45.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021258224999999999, 2024-09-17T22:41:45.016 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:45.016 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:45.016 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:45.016 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:45.016 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:45.017 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:45.017 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106934 2024-09-17T22:41:45.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.017 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:45.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:45.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260113+0000", 2024-09-17T22:41:45.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:45.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260113+0000", 2024-09-17T22:41:45.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:45.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260113+0000", 2024-09-17T22:41:45.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3200000000000001e-07 2024-09-17T22:41:45.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:45.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260114+0000", 2024-09-17T22:41:45.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1069999999999999e-06 2024-09-17T22:41:45.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:45.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260118+0000", 2024-09-17T22:41:45.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3629999999999998e-06 2024-09-17T22:41:45.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:45.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260120+0000", 2024-09-17T22:41:45.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5749999999999999e-06 2024-09-17T22:41:45.019 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.019 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:45.019 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.019 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106966 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:45.019 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.268092+0000", 2024-09-17T22:41:45.019 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0023321610000000001, 2024-09-17T22:41:45.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013336361999999999, 2024-09-17T22:41:45.019 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:45.019 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:45.020 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:45.020 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:45.020 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:45.020 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:45.020 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106966 2024-09-17T22:41:45.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.020 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:45.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:45.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268092+0000", 2024-09-17T22:41:45.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:45.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268092+0000", 2024-09-17T22:41:45.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:45.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268092+0000", 2024-09-17T22:41:45.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8100000000000003e-07 2024-09-17T22:41:45.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:45.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268095+0000", 2024-09-17T22:41:45.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5189999999999999e-06 2024-09-17T22:41:45.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:45.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268102+0000", 2024-09-17T22:41:45.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9580000000000001e-06 2024-09-17T22:41:45.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:45.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268107+0000", 2024-09-17T22:41:45.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-09-17T22:41:45.022 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.022 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:45.022 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.023 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106998 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:45.023 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.272213+0000", 2024-09-17T22:41:45.023 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9982104, 2024-09-17T22:41:45.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0092725229999999995, 2024-09-17T22:41:45.023 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:45.023 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:45.023 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:45.023 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:45.023 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:45.023 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:45.023 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106998 2024-09-17T22:41:45.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.023 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:45.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:45.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272213+0000", 2024-09-17T22:41:45.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:45.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272213+0000", 2024-09-17T22:41:45.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:45.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272214+0000", 2024-09-17T22:41:45.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0400000000000002e-07 2024-09-17T22:41:45.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:45.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272218+0000", 2024-09-17T22:41:45.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9400000000000004e-06 2024-09-17T22:41:45.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:45.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272221+0000", 2024-09-17T22:41:45.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2310000000000001e-06 2024-09-17T22:41:45.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:45.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272223+0000", 2024-09-17T22:41:45.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5890000000000001e-06 2024-09-17T22:41:45.025 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.026 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:45.026 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.026 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106841 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:45.026 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.244115+0000", 2024-09-17T22:41:45.026 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.026309316999999999, 2024-09-17T22:41:45.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037433634, 2024-09-17T22:41:45.026 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:45.026 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:45.026 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:45.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:45.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:45.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:45.026 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106841 2024-09-17T22:41:45.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.027 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:45.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:45.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244115+0000", 2024-09-17T22:41:45.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:45.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244115+0000", 2024-09-17T22:41:45.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:45.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244115+0000", 2024-09-17T22:41:45.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2999999999999999e-07 2024-09-17T22:41:45.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:45.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244116+0000", 2024-09-17T22:41:45.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2869999999999999e-06 2024-09-17T22:41:45.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:45.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244119+0000", 2024-09-17T22:41:45.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.988e-06 2024-09-17T22:41:45.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:45.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.244122+0000", 2024-09-17T22:41:45.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7489999999999999e-06 2024-09-17T22:41:45.029 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.029 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:45.029 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.029 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106873 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:45.029 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.250150+0000", 2024-09-17T22:41:45.029 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.020273736000000001, 2024-09-17T22:41:45.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031456092999999997, 2024-09-17T22:41:45.029 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:45.029 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:45.029 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:45.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:45.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:45.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:45.030 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106873 2024-09-17T22:41:45.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.030 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:45.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:45.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250150+0000", 2024-09-17T22:41:45.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:45.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250150+0000", 2024-09-17T22:41:45.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:45.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250151+0000", 2024-09-17T22:41:45.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-17T22:41:45.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:45.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250153+0000", 2024-09-17T22:41:45.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4229999999999999e-06 2024-09-17T22:41:45.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:45.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250160+0000", 2024-09-17T22:41:45.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6660000000000002e-06 2024-09-17T22:41:45.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:45.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.250165+0000", 2024-09-17T22:41:45.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8799999999999999e-06 2024-09-17T22:41:45.032 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.032 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:45.032 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.033 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106905 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:45.033 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.254173+0000", 2024-09-17T22:41:45.033 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.016250611000000002, 2024-09-17T22:41:45.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027491285000000001, 2024-09-17T22:41:45.033 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:45.033 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:45.033 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:45.033 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:45.033 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:45.033 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:45.033 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106905 2024-09-17T22:41:45.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.034 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:45.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:45.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254173+0000", 2024-09-17T22:41:45.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:45.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254173+0000", 2024-09-17T22:41:45.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:45.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254174+0000", 2024-09-17T22:41:45.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4999999999999998e-07 2024-09-17T22:41:45.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:45.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254176+0000", 2024-09-17T22:41:45.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2910000000000002e-06 2024-09-17T22:41:45.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:45.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254183+0000", 2024-09-17T22:41:45.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.533e-06 2024-09-17T22:41:45.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:45.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.254187+0000", 2024-09-17T22:41:45.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5909999999999998e-06 2024-09-17T22:41:45.036 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.036 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:45.036 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.036 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106935 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:45.036 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.260123+0000", 2024-09-17T22:41:45.036 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.010300673999999999, 2024-09-17T22:41:45.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021596936000000001, 2024-09-17T22:41:45.036 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:45.036 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:45.036 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:45.036 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:45.037 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:45.037 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:45.037 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106935 2024-09-17T22:41:45.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.037 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:45.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:45.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260123+0000", 2024-09-17T22:41:45.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:45.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260123+0000", 2024-09-17T22:41:45.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:45.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260124+0000", 2024-09-17T22:41:45.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.35e-07 2024-09-17T22:41:45.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:45.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260125+0000", 2024-09-17T22:41:45.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.181e-06 2024-09-17T22:41:45.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:45.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260128+0000", 2024-09-17T22:41:45.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0939999999999999e-06 2024-09-17T22:41:45.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:45.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.260130+0000", 2024-09-17T22:41:45.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5900000000000002e-06 2024-09-17T22:41:45.039 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.039 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:45.039 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.039 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106967 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:45.039 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.268115+0000", 2024-09-17T22:41:45.039 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0023086420000000001, 2024-09-17T22:41:45.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013663263, 2024-09-17T22:41:45.040 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:45.040 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:45.040 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:45.040 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:45.040 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:45.040 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:45.040 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106967 2024-09-17T22:41:45.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.040 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:45.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:45.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268115+0000", 2024-09-17T22:41:45.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:45.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268115+0000", 2024-09-17T22:41:45.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:45.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268116+0000", 2024-09-17T22:41:45.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.82e-07 2024-09-17T22:41:45.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:45.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268118+0000", 2024-09-17T22:41:45.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.294e-06 2024-09-17T22:41:45.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:45.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268124+0000", 2024-09-17T22:41:45.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0390000000000001e-06 2024-09-17T22:41:45.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:45.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.268127+0000", 2024-09-17T22:41:45.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1370000000000002e-06 2024-09-17T22:41:45.042 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.042 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:45.042 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.043 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:106999 3.3 3.14ffd7e3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:45.043 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:44.272227+0000", 2024-09-17T22:41:45.043 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9981971, 2024-09-17T22:41:45.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0096070570000000004, 2024-09-17T22:41:45.043 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:45.043 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:45.043 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:45.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:45.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:45.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:45.043 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 106999 2024-09-17T22:41:45.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.043 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:45.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:45.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272227+0000", 2024-09-17T22:41:45.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:45.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272227+0000", 2024-09-17T22:41:45.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:45.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:45.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272227+0000", 2024-09-17T22:41:45.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-09-17T22:41:45.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:45.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272228+0000", 2024-09-17T22:41:45.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1519999999999999e-06 2024-09-17T22:41:45.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:45.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272232+0000", 2024-09-17T22:41:45.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4829999999999999e-06 2024-09-17T22:41:45.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:45.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:45.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:45.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:44.272235+0000", 2024-09-17T22:41:45.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.835e-06 2024-09-17T22:41:45.046 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.046 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:45.046 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.046 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:45.046 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:41:45.046 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 221 2024-09-17T22:41:45.046 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:45.046 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:41:45.048 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:45.045+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:45.048 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:41:45.064 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.6 to 1 (10000) 2024-09-17T22:41:45.084 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-17T22:41:45.120 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:41:45.147 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:45.145+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:45.232 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0: {} 2024-09-17T22:41:45.232 INFO:teuthology.orchestra.run.smithi115.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:41:45.232 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:45.233 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:45.242 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:41:45.253 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:41:45.254 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:41:45.254 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:45.251+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:45.283 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:41:45.283 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:41:45.313 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:41:45.314 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:41:45.336 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:41:45.336 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:41:45.350 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:45.347+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:45.353 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:41:45.353 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:41:45.378 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:41:45.378 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:41:45.389 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:45.390 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:45.401 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:41:45.401 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:41:45.403 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:41:45.457 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:45.454+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:45.508 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-17T22:41:45.552 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:45.549+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:45.572 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:45.572 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:45.584 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:41:45.653 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:45.650+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:45.692 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:45.692 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:45.701 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:41:45.714 INFO:teuthology.orchestra.run.smithi115.stdout:833 read 0x7c2e309 thru 0x7c300ba (0x1db2 bytes) 2024-09-17T22:41:45.727 INFO:teuthology.orchestra.run.smithi115.stdout:834 trunc from 0xd38849f to 0x24c14e0 2024-09-17T22:41:45.753 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:45.751+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:45.855 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:45.852+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:45.877 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:45.877 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:45.890 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:41:45.973 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:45.971+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:46.056 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:46.054+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:46.074 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.7 to 1 (10000) 2024-09-17T22:41:46.075 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is set 2024-09-17T22:41:46.093 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T22:41:46.093 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:41:46.099 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:46.099 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:46.112 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:41:46.157 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:46.155+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:46.258 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:46.255+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:46.335 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:46.335 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:46.343 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:41:46.389 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:46.386+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:46.459 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:46.456+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:46.531 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:46.531 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:46.546 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:41:46.593 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:46.587+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:46.629 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:41:46.630 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:41:46.647 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-17T22:41:46.312941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":376617436,"num_objects":777,"num_object_clones":190,"num_object_copies":1554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":777,"num_whiteouts":415,"num_read":114406,"num_read_kb":261237,"num_write":3295,"num_write_kb":54492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3017,"ondisk_log_size":3017,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263720,"kb_used_data":14200,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711000,"statfs":{"total":773094113280,"available":772824064000,"internally_reserved":0,"allocated":14540800,"data_stored":72470327,"data_compressed":1852135,"data_compressed_allocated":6762496,"data_compressed_original":68675880,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[27,3,0,17,30,60],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":98924650,"num_objects":66,"num_object_clones":13,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":34,"num_read":28839,"num_read_kb":71968,"num_write":214,"num_write_kb":18939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000564"},"pg_stats":[{"pgid":"3.1f","version":"42'78","reported_seq":1165,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.252339+0000","last_change":"2024-09-17T22:41:21.453392+0000","last_active":"2024-09-17T22:41:41.252339+0000","last_peered":"2024-09-17T22:41:41.252339+0000","last_clean":"2024-09-17T22:41:41.252339+0000","last_became_active":"2024-09-17T22:41:19.443658+0000","last_became_peered":"2024-09-17T22:41:19.443658+0000","last_unstale":"2024-09-17T22:41:41.252339+0000","last_undegraded":"2024-09-17T22:41:41.252339+0000","last_fullsized":"2024-09-17T22:41:41.252339+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:29.421387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047335399999999999,"stat_sum":{"num_bytes":1894330,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":1340,"num_read_kb":2199,"num_write":62,"num_write_kb":822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"42'59","reported_seq":5327,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.040779+0000","last_change":"2024-09-17T22:41:21.453587+0000","last_active":"2024-09-17T22:41:41.040779+0000","last_peered":"2024-09-17T22:41:41.040779+0000","last_clean":"2024-09-17T22:41:41.040779+0000","last_became_active":"2024-09-17T22:41:19.449357+0000","last_became_peered":"2024-09-17T22:41:19.449357+0000","last_unstale":"2024-09-17T22:41:41.040779+0000","last_undegraded":"2024-09-17T22:41:41.040779+0000","last_fullsized":"2024-09-17T22:41:41.040779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:25:36.188309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996099999999995,"stat_sum":{"num_bytes":9686090,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":13,"num_read":5248,"num_read_kb":8953,"num_write":47,"num_write_kb":411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"47'51","reported_seq":13427,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.139987+0000","last_change":"2024-09-17T22:41:21.446449+0000","last_active":"2024-09-17T22:41:45.139987+0000","last_peered":"2024-09-17T22:41:45.139987+0000","last_clean":"2024-09-17T22:41:45.139987+0000","last_became_active":"2024-09-17T22:41:19.447823+0000","last_became_peered":"2024-09-17T22:41:19.447823+0000","last_unstale":"2024-09-17T22:41:45.139987+0000","last_undegraded":"2024-09-17T22:41:45.139987+0000","last_fullsized":"2024-09-17T22:41:45.139987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:09:48.270353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000382498,"stat_sum":{"num_bytes":26956791,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":13328,"num_read_kb":39794,"num_write":43,"num_write_kb":3101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"42'52","reported_seq":698,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T22:41:42.077123+0000","last_change":"2024-09-17T22:41:21.446501+0000","last_active":"2024-09-17T22:41:42.077123+0000","last_peered":"2024-09-17T22:41:42.077123+0000","last_clean":"2024-09-17T22:41:42.077123+0000","last_became_active":"2024-09-17T22:41:19.452435+0000","last_became_peered":"2024-09-17T22:41:19.452435+0000","last_unstale":"2024-09-17T22:41:42.077123+0000","last_undegraded":"2024-09-17T22:41:42.077123+0000","last_fullsized":"2024-09-17T22:41:42.077123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:07:21.829392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001451078,"stat_sum":{"num_bytes":1420340,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":623,"num_read_kb":994,"num_write":45,"num_write_kb":365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"42'37","reported_seq":671,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.222218+0000","last_change":"2024-09-17T22:41:21.448196+0000","last_active":"2024-09-17T22:41:41.222218+0000","last_peered":"2024-09-17T22:41:41.222218+0000","last_clean":"2024-09-17T22:41:41.222218+0000","last_became_active":"2024-09-17T22:41:19.449830+0000","last_became_peered":"2024-09-17T22:41:19.449830+0000","last_unstale":"2024-09-17T22:41:41.222218+0000","last_undegraded":"2024-09-17T22:41:41.222218+0000","last_fullsized":"2024-09-17T22:41:41.222218+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:05.977260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055813,"stat_sum":{"num_bytes":3336192,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":592,"num_read_kb":2616,"num_write":29,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"42'78","reported_seq":1034,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T22:41:42.298701+0000","last_change":"2024-09-17T22:41:21.446945+0000","last_active":"2024-09-17T22:41:42.298701+0000","last_peered":"2024-09-17T22:41:42.298701+0000","last_clean":"2024-09-17T22:41:42.298701+0000","last_became_active":"2024-09-17T22:41:19.446541+0000","last_became_peered":"2024-09-17T22:41:19.446541+0000","last_unstale":"2024-09-17T22:41:42.298701+0000","last_undegraded":"2024-09-17T22:41:42.298701+0000","last_fullsized":"2024-09-17T22:41:42.298701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:36:20.024806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001630631,"stat_sum":{"num_bytes":4877750,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":12,"num_read":1261,"num_read_kb":5475,"num_write":71,"num_write_kb":2479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"42'58","reported_seq":5182,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.222199+0000","last_change":"2024-09-17T22:41:21.448965+0000","last_active":"2024-09-17T22:41:41.222199+0000","last_peered":"2024-09-17T22:41:41.222199+0000","last_clean":"2024-09-17T22:41:41.222199+0000","last_became_active":"2024-09-17T22:41:19.449940+0000","last_became_peered":"2024-09-17T22:41:19.449940+0000","last_unstale":"2024-09-17T22:41:41.222199+0000","last_undegraded":"2024-09-17T22:41:41.222199+0000","last_fullsized":"2024-09-17T22:41:41.222199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:04:43.297490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011385119999999999,"stat_sum":{"num_bytes":14797003,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":5160,"num_read_kb":10228,"num_write":54,"num_write_kb":407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"43'51","reported_seq":1244,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.252256+0000","last_change":"2024-09-17T22:41:21.453701+0000","last_active":"2024-09-17T22:41:41.252256+0000","last_peered":"2024-09-17T22:41:41.252256+0000","last_clean":"2024-09-17T22:41:41.252256+0000","last_became_active":"2024-09-17T22:41:19.446287+0000","last_became_peered":"2024-09-17T22:41:19.446287+0000","last_unstale":"2024-09-17T22:41:41.252256+0000","last_undegraded":"2024-09-17T22:41:41.252256+0000","last_fullsized":"2024-09-17T22:41:41.252256+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:02:36.857335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067184299999999998,"stat_sum":{"num_bytes":5853887,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":12,"num_read":1195,"num_read_kb":7316,"num_write":39,"num_write_kb":2262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"42'102","reported_seq":944,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.040858+0000","last_change":"2024-09-17T22:41:21.453344+0000","last_active":"2024-09-17T22:41:41.040858+0000","last_peered":"2024-09-17T22:41:41.040858+0000","last_clean":"2024-09-17T22:41:41.040858+0000","last_became_active":"2024-09-17T22:41:19.445923+0000","last_became_peered":"2024-09-17T22:41:19.445923+0000","last_unstale":"2024-09-17T22:41:41.040858+0000","last_undegraded":"2024-09-17T22:41:41.040858+0000","last_fullsized":"2024-09-17T22:41:41.040858+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:40:16.364316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043560199999999999,"stat_sum":{"num_bytes":5714444,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":14,"num_read":1082,"num_read_kb":5412,"num_write":132,"num_write_kb":2755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"44'106","reported_seq":13923,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.110283+0000","last_change":"2024-09-17T22:41:21.516305+0000","last_active":"2024-09-17T22:41:45.110283+0000","last_peered":"2024-09-17T22:41:45.110283+0000","last_clean":"2024-09-17T22:41:45.110283+0000","last_became_active":"2024-09-17T22:41:19.450587+0000","last_became_peered":"2024-09-17T22:41:19.450587+0000","last_unstale":"2024-09-17T22:41:45.110283+0000","last_undegraded":"2024-09-17T22:41:45.110283+0000","last_fullsized":"2024-09-17T22:41:45.110283+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:19:38.192774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000275164,"stat_sum":{"num_bytes":25124231,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":14,"num_read":14291,"num_read_kb":35504,"num_write":97,"num_write_kb":5076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"42'53","reported_seq":5408,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T22:41:42.047450+0000","last_change":"2024-09-17T22:41:21.454760+0000","last_active":"2024-09-17T22:41:42.047450+0000","last_peered":"2024-09-17T22:41:42.047450+0000","last_clean":"2024-09-17T22:41:42.047450+0000","last_became_active":"2024-09-17T22:41:19.449329+0000","last_became_peered":"2024-09-17T22:41:19.449329+0000","last_unstale":"2024-09-17T22:41:42.047450+0000","last_undegraded":"2024-09-17T22:41:42.047450+0000","last_fullsized":"2024-09-17T22:41:42.047450+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:49:10.136052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041055799999999999,"stat_sum":{"num_bytes":24482622,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":10,"num_read":5311,"num_read_kb":10056,"num_write":42,"num_write_kb":437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"47'51","reported_seq":2416,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.179358+0000","last_change":"2024-09-17T22:41:21.446210+0000","last_active":"2024-09-17T22:41:45.179358+0000","last_peered":"2024-09-17T22:41:45.179358+0000","last_clean":"2024-09-17T22:41:45.179358+0000","last_became_active":"2024-09-17T22:41:19.451631+0000","last_became_peered":"2024-09-17T22:41:19.451631+0000","last_unstale":"2024-09-17T22:41:45.179358+0000","last_undegraded":"2024-09-17T22:41:45.179358+0000","last_fullsized":"2024-09-17T22:41:45.179358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:57:37.622177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081193699999999999,"stat_sum":{"num_bytes":14365995,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":10,"num_read":2323,"num_read_kb":3280,"num_write":42,"num_write_kb":356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.061245+0000","last_change":"2024-09-17T22:41:11.397814+0000","last_active":"2024-09-17T22:41:45.061245+0000","last_peered":"2024-09-17T22:41:45.061245+0000","last_clean":"2024-09-17T22:41:45.061245+0000","last_became_active":"2024-09-17T22:41:09.506775+0000","last_became_peered":"2024-09-17T22:41:09.506775+0000","last_unstale":"2024-09-17T22:41:45.061245+0000","last_undegraded":"2024-09-17T22:41:45.061245+0000","last_fullsized":"2024-09-17T22:41:45.061245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:47:43.867775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"42'144","reported_seq":989,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.222267+0000","last_change":"2024-09-17T22:41:21.448698+0000","last_active":"2024-09-17T22:41:41.222267+0000","last_peered":"2024-09-17T22:41:41.222267+0000","last_clean":"2024-09-17T22:41:41.222267+0000","last_became_active":"2024-09-17T22:41:19.448966+0000","last_became_peered":"2024-09-17T22:41:19.448966+0000","last_unstale":"2024-09-17T22:41:41.222267+0000","last_undegraded":"2024-09-17T22:41:41.222267+0000","last_fullsized":"2024-09-17T22:41:41.222267+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:44.524786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089570000000000003,"stat_sum":{"num_bytes":4415630,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":13,"num_read":1205,"num_read_kb":3276,"num_write":190,"num_write_kb":713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":63,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.222362+0000","last_change":"2024-09-17T22:41:11.395783+0000","last_active":"2024-09-17T22:41:41.222362+0000","last_peered":"2024-09-17T22:41:41.222362+0000","last_clean":"2024-09-17T22:41:41.222362+0000","last_became_active":"2024-09-17T22:41:09.507056+0000","last_became_peered":"2024-09-17T22:41:09.507056+0000","last_unstale":"2024-09-17T22:41:41.222362+0000","last_undegraded":"2024-09-17T22:41:41.222362+0000","last_fullsized":"2024-09-17T22:41:41.222362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:13.438783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"42'152","reported_seq":669,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.222245+0000","last_change":"2024-09-17T22:41:21.448596+0000","last_active":"2024-09-17T22:41:41.222245+0000","last_peered":"2024-09-17T22:41:41.222245+0000","last_clean":"2024-09-17T22:41:41.222245+0000","last_became_active":"2024-09-17T22:41:19.451637+0000","last_became_peered":"2024-09-17T22:41:19.451637+0000","last_unstale":"2024-09-17T22:41:41.222245+0000","last_undegraded":"2024-09-17T22:41:41.222245+0000","last_fullsized":"2024-09-17T22:41:41.222245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:27:06.317680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085737499999999996,"stat_sum":{"num_bytes":13080273,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":15,"num_read":952,"num_read_kb":2502,"num_write":203,"num_write_kb":1591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.061245+0000","last_change":"2024-09-17T22:41:11.397753+0000","last_active":"2024-09-17T22:41:45.061245+0000","last_peered":"2024-09-17T22:41:45.061245+0000","last_clean":"2024-09-17T22:41:45.061245+0000","last_became_active":"2024-09-17T22:41:09.387651+0000","last_became_peered":"2024-09-17T22:41:09.387651+0000","last_unstale":"2024-09-17T22:41:45.061245+0000","last_undegraded":"2024-09-17T22:41:45.061245+0000","last_fullsized":"2024-09-17T22:41:45.061245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:14:30.915047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"39'52","reported_seq":454,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.222418+0000","last_change":"2024-09-17T22:41:21.466363+0000","last_active":"2024-09-17T22:41:41.222418+0000","last_peered":"2024-09-17T22:41:41.222418+0000","last_clean":"2024-09-17T22:41:41.222418+0000","last_became_active":"2024-09-17T22:41:19.444893+0000","last_became_peered":"2024-09-17T22:41:19.444893+0000","last_unstale":"2024-09-17T22:41:41.222418+0000","last_undegraded":"2024-09-17T22:41:41.222418+0000","last_fullsized":"2024-09-17T22:41:41.222418+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:01:17.856099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053617200000000002,"stat_sum":{"num_bytes":5475472,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":10,"num_read":354,"num_read_kb":1870,"num_write":49,"num_write_kb":884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":63,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.040778+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:41:41.040778+0000","last_peered":"2024-09-17T22:41:41.040778+0000","last_clean":"2024-09-17T22:41:41.040778+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:41:41.040778+0000","last_undegraded":"2024-09-17T22:41:41.040778+0000","last_fullsized":"2024-09-17T22:41:41.040778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"45'155","reported_seq":10182,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.110155+0000","last_change":"2024-09-17T22:41:21.516349+0000","last_active":"2024-09-17T22:41:45.110155+0000","last_peered":"2024-09-17T22:41:45.110155+0000","last_clean":"2024-09-17T22:41:45.110155+0000","last_became_active":"2024-09-17T22:41:19.451723+0000","last_became_peered":"2024-09-17T22:41:19.451723+0000","last_unstale":"2024-09-17T22:41:45.110155+0000","last_undegraded":"2024-09-17T22:41:45.110155+0000","last_fullsized":"2024-09-17T22:41:45.110155+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:04:49.129682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040033199999999999,"stat_sum":{"num_bytes":31174924,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":10463,"num_read_kb":17889,"num_write":219,"num_write_kb":2882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"44'133","reported_seq":2349,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T22:41:42.094286+0000","last_change":"2024-09-17T22:41:21.454999+0000","last_active":"2024-09-17T22:41:42.094286+0000","last_peered":"2024-09-17T22:41:42.094286+0000","last_clean":"2024-09-17T22:41:42.094286+0000","last_became_active":"2024-09-17T22:41:19.450477+0000","last_became_peered":"2024-09-17T22:41:19.450477+0000","last_unstale":"2024-09-17T22:41:42.094286+0000","last_undegraded":"2024-09-17T22:41:42.094286+0000","last_fullsized":"2024-09-17T22:41:42.094286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:00:27.747495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063898899999999996,"stat_sum":{"num_bytes":22311294,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":16,"num_read":3559,"num_read_kb":8460,"num_write":115,"num_write_kb":463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":73,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.061003+0000","last_change":"2024-09-17T22:41:11.397488+0000","last_active":"2024-09-17T22:41:45.061003+0000","last_peered":"2024-09-17T22:41:45.061003+0000","last_clean":"2024-09-17T22:41:45.061003+0000","last_became_active":"2024-09-17T22:41:09.651390+0000","last_became_peered":"2024-09-17T22:41:09.651390+0000","last_unstale":"2024-09-17T22:41:45.061003+0000","last_undegraded":"2024-09-17T22:41:45.061003+0000","last_fullsized":"2024-09-17T22:41:45.061003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:00.598921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"39'75","reported_seq":1751,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.254191+0000","last_change":"2024-09-17T22:41:21.453941+0000","last_active":"2024-09-17T22:41:41.254191+0000","last_peered":"2024-09-17T22:41:41.254191+0000","last_clean":"2024-09-17T22:41:41.254191+0000","last_became_active":"2024-09-17T22:41:19.444314+0000","last_became_peered":"2024-09-17T22:41:19.444314+0000","last_unstale":"2024-09-17T22:41:41.254191+0000","last_undegraded":"2024-09-17T22:41:41.254191+0000","last_fullsized":"2024-09-17T22:41:41.254191+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:02:49.032558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085427099999999998,"stat_sum":{"num_bytes":14740336,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":13,"num_read":1735,"num_read_kb":3824,"num_write":74,"num_write_kb":810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":63,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.034522+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:41:41.034522+0000","last_peered":"2024-09-17T22:41:41.034522+0000","last_clean":"2024-09-17T22:41:41.034522+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:41:41.034522+0000","last_undegraded":"2024-09-17T22:41:41.034522+0000","last_fullsized":"2024-09-17T22:41:41.034522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"42'78","reported_seq":5984,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.222352+0000","last_change":"2024-09-17T22:41:21.466094+0000","last_active":"2024-09-17T22:41:41.222352+0000","last_peered":"2024-09-17T22:41:41.222352+0000","last_clean":"2024-09-17T22:41:41.222352+0000","last_became_active":"2024-09-17T22:41:19.449791+0000","last_became_peered":"2024-09-17T22:41:19.449791+0000","last_unstale":"2024-09-17T22:41:41.222352+0000","last_undegraded":"2024-09-17T22:41:41.222352+0000","last_fullsized":"2024-09-17T22:41:41.222352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:52:36.557991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042775300000000001,"stat_sum":{"num_bytes":5558819,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":8,"num_read":6083,"num_read_kb":4202,"num_write":110,"num_write_kb":897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.060822+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:41:45.060822+0000","last_peered":"2024-09-17T22:41:45.060822+0000","last_clean":"2024-09-17T22:41:45.060822+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:41:45.060822+0000","last_undegraded":"2024-09-17T22:41:45.060822+0000","last_fullsized":"2024-09-17T22:41:45.060822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"42'73","reported_seq":1006,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.229472+0000","last_change":"2024-09-17T22:41:21.448409+0000","last_active":"2024-09-17T22:41:41.229472+0000","last_peered":"2024-09-17T22:41:41.229472+0000","last_clean":"2024-09-17T22:41:41.229472+0000","last_became_active":"2024-09-17T22:41:19.449279+0000","last_became_peered":"2024-09-17T22:41:19.449279+0000","last_unstale":"2024-09-17T22:41:41.229472+0000","last_undegraded":"2024-09-17T22:41:41.229472+0000","last_fullsized":"2024-09-17T22:41:41.229472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:47.262077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075217299999999997,"stat_sum":{"num_bytes":5903782,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":17,"num_read":911,"num_read_kb":650,"num_write":58,"num_write_kb":495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.061144+0000","last_change":"2024-09-17T22:41:11.398108+0000","last_active":"2024-09-17T22:41:45.061144+0000","last_peered":"2024-09-17T22:41:45.061144+0000","last_clean":"2024-09-17T22:41:45.061144+0000","last_became_active":"2024-09-17T22:41:09.388184+0000","last_became_peered":"2024-09-17T22:41:09.388184+0000","last_unstale":"2024-09-17T22:41:45.061144+0000","last_undegraded":"2024-09-17T22:41:45.061144+0000","last_fullsized":"2024-09-17T22:41:45.061144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:43.506329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":216,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T22:41:42.076994+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:41:42.076994+0000","last_peered":"2024-09-17T22:41:42.076994+0000","last_clean":"2024-09-17T22:41:42.076994+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:41:42.076994+0000","last_undegraded":"2024-09-17T22:41:42.076994+0000","last_fullsized":"2024-09-17T22:41:42.076994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"42'41","reported_seq":372,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.222175+0000","last_change":"2024-09-17T22:41:21.448156+0000","last_active":"2024-09-17T22:41:41.222175+0000","last_peered":"2024-09-17T22:41:41.222175+0000","last_clean":"2024-09-17T22:41:41.222175+0000","last_became_active":"2024-09-17T22:41:19.448956+0000","last_became_peered":"2024-09-17T22:41:19.448956+0000","last_unstale":"2024-09-17T22:41:41.222175+0000","last_undegraded":"2024-09-17T22:41:41.222175+0000","last_fullsized":"2024-09-17T22:41:41.222175+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:29:07.495009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055851200000000005,"stat_sum":{"num_bytes":2437136,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":12,"num_read":290,"num_read_kb":2144,"num_write":29,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"47'210","reported_seq":13095,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.267354+0000","last_change":"2024-09-17T22:41:21.445421+0000","last_active":"2024-09-17T22:41:45.267354+0000","last_peered":"2024-09-17T22:41:45.267354+0000","last_clean":"2024-09-17T22:41:45.267354+0000","last_became_active":"2024-09-17T22:41:19.449003+0000","last_became_peered":"2024-09-17T22:41:19.449003+0000","last_unstale":"2024-09-17T22:41:45.267354+0000","last_undegraded":"2024-09-17T22:41:45.267354+0000","last_fullsized":"2024-09-17T22:41:45.267354+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:16:20.907361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051465400000000002,"stat_sum":{"num_bytes":10247083,"num_objects":34,"num_object_clones":10,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":17,"num_read":13662,"num_read_kb":32375,"num_write":330,"num_write_kb":1549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"46'189","reported_seq":6684,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.060911+0000","last_change":"2024-09-17T22:41:21.445728+0000","last_active":"2024-09-17T22:41:45.060911+0000","last_peered":"2024-09-17T22:41:45.060911+0000","last_clean":"2024-09-17T22:41:45.060911+0000","last_became_active":"2024-09-17T22:41:19.448011+0000","last_became_peered":"2024-09-17T22:41:19.448011+0000","last_unstale":"2024-09-17T22:41:45.060911+0000","last_undegraded":"2024-09-17T22:41:45.060911+0000","last_fullsized":"2024-09-17T22:41:45.060911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:00:46.395080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068494200000000002,"stat_sum":{"num_bytes":19986595,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":18,"num_read":6878,"num_read_kb":15255,"num_write":224,"num_write_kb":1766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"42'41","reported_seq":578,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.034534+0000","last_change":"2024-09-17T22:41:21.462738+0000","last_active":"2024-09-17T22:41:41.034534+0000","last_peered":"2024-09-17T22:41:41.034534+0000","last_clean":"2024-09-17T22:41:41.034534+0000","last_became_active":"2024-09-17T22:41:19.447215+0000","last_became_peered":"2024-09-17T22:41:19.447215+0000","last_unstale":"2024-09-17T22:41:41.034534+0000","last_undegraded":"2024-09-17T22:41:41.034534+0000","last_fullsized":"2024-09-17T22:41:41.034534+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:20:59.739946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042703000000000001,"stat_sum":{"num_bytes":6535629,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":12,"num_read":495,"num_read_kb":2871,"num_write":31,"num_write_kb":370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"44'88","reported_seq":1707,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T22:41:42.318998+0000","last_change":"2024-09-17T22:41:21.454754+0000","last_active":"2024-09-17T22:41:42.318998+0000","last_peered":"2024-09-17T22:41:42.318998+0000","last_clean":"2024-09-17T22:41:42.318998+0000","last_became_active":"2024-09-17T22:41:19.448928+0000","last_became_peered":"2024-09-17T22:41:19.448928+0000","last_unstale":"2024-09-17T22:41:42.318998+0000","last_undegraded":"2024-09-17T22:41:42.318998+0000","last_fullsized":"2024-09-17T22:41:42.318998+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:55:39.904266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000395712,"stat_sum":{"num_bytes":47362905,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":10,"num_read":2137,"num_read_kb":10345,"num_write":79,"num_write_kb":10068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"47'56","reported_seq":858,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.238429+0000","last_change":"2024-09-17T22:41:21.446562+0000","last_active":"2024-09-17T22:41:45.238429+0000","last_peered":"2024-09-17T22:41:45.238429+0000","last_clean":"2024-09-17T22:41:45.238429+0000","last_became_active":"2024-09-17T22:41:19.447631+0000","last_became_peered":"2024-09-17T22:41:19.447631+0000","last_unstale":"2024-09-17T22:41:45.238429+0000","last_undegraded":"2024-09-17T22:41:45.238429+0000","last_fullsized":"2024-09-17T22:41:45.238429+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:15:34.708107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000402657,"stat_sum":{"num_bytes":1788122,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":15,"num_read":756,"num_read_kb":1144,"num_write":43,"num_write_kb":643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"47'116","reported_seq":1903,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.335350+0000","last_change":"2024-09-17T22:41:21.516505+0000","last_active":"2024-09-17T22:41:45.335350+0000","last_peered":"2024-09-17T22:41:45.335350+0000","last_clean":"2024-09-17T22:41:45.335350+0000","last_became_active":"2024-09-17T22:41:19.450839+0000","last_became_peered":"2024-09-17T22:41:19.450839+0000","last_unstale":"2024-09-17T22:41:45.335350+0000","last_undegraded":"2024-09-17T22:41:45.335350+0000","last_fullsized":"2024-09-17T22:41:45.335350+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.126384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038538199999999998,"stat_sum":{"num_bytes":16294946,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":21,"num_read":2380,"num_read_kb":2776,"num_write":80,"num_write_kb":4656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"42'70","reported_seq":916,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T22:41:42.047520+0000","last_change":"2024-09-17T22:41:21.454987+0000","last_active":"2024-09-17T22:41:42.047520+0000","last_peered":"2024-09-17T22:41:42.047520+0000","last_clean":"2024-09-17T22:41:42.047520+0000","last_became_active":"2024-09-17T22:41:19.451864+0000","last_became_peered":"2024-09-17T22:41:19.451864+0000","last_unstale":"2024-09-17T22:41:42.047520+0000","last_undegraded":"2024-09-17T22:41:42.047520+0000","last_fullsized":"2024-09-17T22:41:42.047520+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:57:10.852093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000602379,"stat_sum":{"num_bytes":2624457,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":20,"num_read":826,"num_read_kb":2780,"num_write":61,"num_write_kb":1318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"42'88","reported_seq":753,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T22:41:42.095247+0000","last_change":"2024-09-17T22:41:21.446576+0000","last_active":"2024-09-17T22:41:42.095247+0000","last_peered":"2024-09-17T22:41:42.095247+0000","last_clean":"2024-09-17T22:41:42.095247+0000","last_became_active":"2024-09-17T22:41:19.447419+0000","last_became_peered":"2024-09-17T22:41:19.447419+0000","last_unstale":"2024-09-17T22:41:42.095247+0000","last_undegraded":"2024-09-17T22:41:42.095247+0000","last_fullsized":"2024-09-17T22:41:42.095247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:04.300024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001865071,"stat_sum":{"num_bytes":2869096,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":10,"num_read":1282,"num_read_kb":3946,"num_write":75,"num_write_kb":1495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"47'99","reported_seq":1165,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.060885+0000","last_change":"2024-09-17T22:41:21.445226+0000","last_active":"2024-09-17T22:41:45.060885+0000","last_peered":"2024-09-17T22:41:45.060885+0000","last_clean":"2024-09-17T22:41:45.060885+0000","last_became_active":"2024-09-17T22:41:19.454151+0000","last_became_peered":"2024-09-17T22:41:19.454151+0000","last_unstale":"2024-09-17T22:41:45.060885+0000","last_undegraded":"2024-09-17T22:41:45.060885+0000","last_fullsized":"2024-09-17T22:41:45.060885+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:57:55.031784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050545600000000005,"stat_sum":{"num_bytes":9251081,"num_objects":37,"num_object_clones":14,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":21,"num_read":1158,"num_read_kb":1629,"num_write":100,"num_write_kb":866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"39'39","reported_seq":5251,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.222308+0000","last_change":"2024-09-17T22:41:21.466130+0000","last_active":"2024-09-17T22:41:41.222308+0000","last_peered":"2024-09-17T22:41:41.222308+0000","last_clean":"2024-09-17T22:41:41.222308+0000","last_became_active":"2024-09-17T22:41:19.446028+0000","last_became_peered":"2024-09-17T22:41:19.446028+0000","last_unstale":"2024-09-17T22:41:41.222308+0000","last_undegraded":"2024-09-17T22:41:41.222308+0000","last_fullsized":"2024-09-17T22:41:41.222308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:21:37.479162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048487299999999998,"stat_sum":{"num_bytes":4648898,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":7,"num_read":5170,"num_read_kb":6144,"num_write":32,"num_write_kb":464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"47'222","reported_seq":1669,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.136186+0000","last_change":"2024-09-17T22:41:21.446196+0000","last_active":"2024-09-17T22:41:45.136186+0000","last_peered":"2024-09-17T22:41:45.136186+0000","last_clean":"2024-09-17T22:41:45.136186+0000","last_became_active":"2024-09-17T22:41:19.451662+0000","last_became_peered":"2024-09-17T22:41:19.451662+0000","last_unstale":"2024-09-17T22:41:45.136186+0000","last_undegraded":"2024-09-17T22:41:45.136186+0000","last_fullsized":"2024-09-17T22:41:45.136186+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:54:04.082193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062146200000000003,"stat_sum":{"num_bytes":10810896,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":19,"num_read":2240,"num_read_kb":5228,"num_write":329,"num_write_kb":1378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":376027049,"num_objects":773,"num_object_clones":190,"num_object_copies":1546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":773,"num_whiteouts":415,"num_read":114290,"num_read_kb":261137,"num_write":3134,"num_write_kb":52074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":11710464,"data_stored":84725566,"data_compressed":2021070,"data_compressed_allocated":7794688,"data_compressed_original":82583245,"omap_allocated":0,"internal_metadata":0},"log_size":2897,"ondisk_log_size":2897,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542156,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32640,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339200,"statfs":{"total":96636764160,"available":96603340800,"internally_reserved":0,"allocated":1486848,"data_stored":7796082,"data_compressed":92465,"data_compressed_allocated":626688,"data_compressed_original":7372800,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542154,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1675264,"data_stored":7657953,"data_compressed":280840,"data_compressed_allocated":733184,"data_compressed_original":7147520,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1064960,"data_stored":3543745,"data_compressed":83618,"data_compressed_allocated":344064,"data_compressed_original":3208740,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1642496,"data_stored":8345602,"data_compressed":181420,"data_compressed_allocated":753664,"data_compressed_original":7916324,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32992,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338848,"statfs":{"total":96636764160,"available":96602980352,"internally_reserved":0,"allocated":1851392,"data_stored":11372676,"data_compressed":182781,"data_compressed_allocated":950272,"data_compressed_original":10945684,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1343488,"data_stored":3933951,"data_compressed":113372,"data_compressed_allocated":438272,"data_compressed_original":3502080,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33916,"kb_used_data":2728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337924,"statfs":{"total":96636764160,"available":96602034176,"internally_reserved":0,"allocated":2793472,"data_stored":14319660,"data_compressed":497141,"data_compressed_allocated":1486848,"data_compressed_original":13693772,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33812,"kb_used_data":2620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338028,"statfs":{"total":96636764160,"available":96602140672,"internally_reserved":0,"allocated":2682880,"data_stored":15500658,"data_compressed":420498,"data_compressed_allocated":1429504,"data_compressed_original":14888960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[26,3,0,17,30,60],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2158592,"data_stored":19865064,"data_compressed":405808,"data_compressed_allocated":1630208,"data_compressed_original":19531173,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":987136,"data_stored":7055940,"data_compressed":118161,"data_compressed_allocated":622592,"data_compressed_original":6848512,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2236416,"data_stored":16326240,"data_compressed":434537,"data_compressed_allocated":1503232,"data_compressed_original":15937536,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2371584,"data_stored":15669511,"data_compressed":506351,"data_compressed_allocated":1589248,"data_compressed_original":15266636,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":872448,"data_stored":3734757,"data_compressed":113372,"data_compressed_allocated":438272,"data_compressed_original":3502080,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1323008,"data_stored":10583114,"data_compressed":177803,"data_compressed_allocated":913408,"data_compressed_original":10372244,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1167360,"data_stored":8146389,"data_compressed":181420,"data_compressed_allocated":753664,"data_compressed_original":7916324,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":3344551,"data_compressed":83618,"data_compressed_allocated":344064,"data_compressed_original":3208740,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:41:46.649 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:41:46.689 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:46.686+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:46.761 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:46.759+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:46.764 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:46.764 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:46.779 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:41:46.864 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:46.862+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:46.963 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:46.960+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:46.995 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:46.995 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:47.009 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:41:47.064 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:47.061+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:47.165 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:47.162+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:47.207 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:41:47.207 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:41:47.221 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:47.221 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:47.225 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-17T22:41:46.312941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":376617436,"num_objects":777,"num_object_clones":190,"num_object_copies":1554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":777,"num_whiteouts":415,"num_read":114406,"num_read_kb":261237,"num_write":3295,"num_write_kb":54492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3017,"ondisk_log_size":3017,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263720,"kb_used_data":14200,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711000,"statfs":{"total":773094113280,"available":772824064000,"internally_reserved":0,"allocated":14540800,"data_stored":72470327,"data_compressed":1852135,"data_compressed_allocated":6762496,"data_compressed_original":68675880,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[27,3,0,17,30,60],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":98924650,"num_objects":66,"num_object_clones":13,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":34,"num_read":28839,"num_read_kb":71968,"num_write":214,"num_write_kb":18939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000564"},"pg_stats":[{"pgid":"3.1f","version":"42'78","reported_seq":1165,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.252339+0000","last_change":"2024-09-17T22:41:21.453392+0000","last_active":"2024-09-17T22:41:41.252339+0000","last_peered":"2024-09-17T22:41:41.252339+0000","last_clean":"2024-09-17T22:41:41.252339+0000","last_became_active":"2024-09-17T22:41:19.443658+0000","last_became_peered":"2024-09-17T22:41:19.443658+0000","last_unstale":"2024-09-17T22:41:41.252339+0000","last_undegraded":"2024-09-17T22:41:41.252339+0000","last_fullsized":"2024-09-17T22:41:41.252339+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:36:29.421387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047335399999999999,"stat_sum":{"num_bytes":1894330,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":1340,"num_read_kb":2199,"num_write":62,"num_write_kb":822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"42'59","reported_seq":5327,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.040779+0000","last_change":"2024-09-17T22:41:21.453587+0000","last_active":"2024-09-17T22:41:41.040779+0000","last_peered":"2024-09-17T22:41:41.040779+0000","last_clean":"2024-09-17T22:41:41.040779+0000","last_became_active":"2024-09-17T22:41:19.449357+0000","last_became_peered":"2024-09-17T22:41:19.449357+0000","last_unstale":"2024-09-17T22:41:41.040779+0000","last_undegraded":"2024-09-17T22:41:41.040779+0000","last_fullsized":"2024-09-17T22:41:41.040779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:25:36.188309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996099999999995,"stat_sum":{"num_bytes":9686090,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":13,"num_read":5248,"num_read_kb":8953,"num_write":47,"num_write_kb":411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"47'51","reported_seq":13427,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.139987+0000","last_change":"2024-09-17T22:41:21.446449+0000","last_active":"2024-09-17T22:41:45.139987+0000","last_peered":"2024-09-17T22:41:45.139987+0000","last_clean":"2024-09-17T22:41:45.139987+0000","last_became_active":"2024-09-17T22:41:19.447823+0000","last_became_peered":"2024-09-17T22:41:19.447823+0000","last_unstale":"2024-09-17T22:41:45.139987+0000","last_undegraded":"2024-09-17T22:41:45.139987+0000","last_fullsized":"2024-09-17T22:41:45.139987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:09:48.270353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000382498,"stat_sum":{"num_bytes":26956791,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":13328,"num_read_kb":39794,"num_write":43,"num_write_kb":3101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"42'52","reported_seq":698,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T22:41:42.077123+0000","last_change":"2024-09-17T22:41:21.446501+0000","last_active":"2024-09-17T22:41:42.077123+0000","last_peered":"2024-09-17T22:41:42.077123+0000","last_clean":"2024-09-17T22:41:42.077123+0000","last_became_active":"2024-09-17T22:41:19.452435+0000","last_became_peered":"2024-09-17T22:41:19.452435+0000","last_unstale":"2024-09-17T22:41:42.077123+0000","last_undegraded":"2024-09-17T22:41:42.077123+0000","last_fullsized":"2024-09-17T22:41:42.077123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:07:21.829392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001451078,"stat_sum":{"num_bytes":1420340,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":623,"num_read_kb":994,"num_write":45,"num_write_kb":365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"42'37","reported_seq":671,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.222218+0000","last_change":"2024-09-17T22:41:21.448196+0000","last_active":"2024-09-17T22:41:41.222218+0000","last_peered":"2024-09-17T22:41:41.222218+0000","last_clean":"2024-09-17T22:41:41.222218+0000","last_became_active":"2024-09-17T22:41:19.449830+0000","last_became_peered":"2024-09-17T22:41:19.449830+0000","last_unstale":"2024-09-17T22:41:41.222218+0000","last_undegraded":"2024-09-17T22:41:41.222218+0000","last_fullsized":"2024-09-17T22:41:41.222218+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:05.977260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055813,"stat_sum":{"num_bytes":3336192,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":592,"num_read_kb":2616,"num_write":29,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"42'78","reported_seq":1034,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T22:41:42.298701+0000","last_change":"2024-09-17T22:41:21.446945+0000","last_active":"2024-09-17T22:41:42.298701+0000","last_peered":"2024-09-17T22:41:42.298701+0000","last_clean":"2024-09-17T22:41:42.298701+0000","last_became_active":"2024-09-17T22:41:19.446541+0000","last_became_peered":"2024-09-17T22:41:19.446541+0000","last_unstale":"2024-09-17T22:41:42.298701+0000","last_undegraded":"2024-09-17T22:41:42.298701+0000","last_fullsized":"2024-09-17T22:41:42.298701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:36:20.024806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001630631,"stat_sum":{"num_bytes":4877750,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":12,"num_read":1261,"num_read_kb":5475,"num_write":71,"num_write_kb":2479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"42'58","reported_seq":5182,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.222199+0000","last_change":"2024-09-17T22:41:21.448965+0000","last_active":"2024-09-17T22:41:41.222199+0000","last_peered":"2024-09-17T22:41:41.222199+0000","last_clean":"2024-09-17T22:41:41.222199+0000","last_became_active":"2024-09-17T22:41:19.449940+0000","last_became_peered":"2024-09-17T22:41:19.449940+0000","last_unstale":"2024-09-17T22:41:41.222199+0000","last_undegraded":"2024-09-17T22:41:41.222199+0000","last_fullsized":"2024-09-17T22:41:41.222199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:04:43.297490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011385119999999999,"stat_sum":{"num_bytes":14797003,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":5160,"num_read_kb":10228,"num_write":54,"num_write_kb":407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"43'51","reported_seq":1244,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.252256+0000","last_change":"2024-09-17T22:41:21.453701+0000","last_active":"2024-09-17T22:41:41.252256+0000","last_peered":"2024-09-17T22:41:41.252256+0000","last_clean":"2024-09-17T22:41:41.252256+0000","last_became_active":"2024-09-17T22:41:19.446287+0000","last_became_peered":"2024-09-17T22:41:19.446287+0000","last_unstale":"2024-09-17T22:41:41.252256+0000","last_undegraded":"2024-09-17T22:41:41.252256+0000","last_fullsized":"2024-09-17T22:41:41.252256+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:02:36.857335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067184299999999998,"stat_sum":{"num_bytes":5853887,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":12,"num_read":1195,"num_read_kb":7316,"num_write":39,"num_write_kb":2262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"42'102","reported_seq":944,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.040858+0000","last_change":"2024-09-17T22:41:21.453344+0000","last_active":"2024-09-17T22:41:41.040858+0000","last_peered":"2024-09-17T22:41:41.040858+0000","last_clean":"2024-09-17T22:41:41.040858+0000","last_became_active":"2024-09-17T22:41:19.445923+0000","last_became_peered":"2024-09-17T22:41:19.445923+0000","last_unstale":"2024-09-17T22:41:41.040858+0000","last_undegraded":"2024-09-17T22:41:41.040858+0000","last_fullsized":"2024-09-17T22:41:41.040858+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:40:16.364316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043560199999999999,"stat_sum":{"num_bytes":5714444,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":14,"num_read":1082,"num_read_kb":5412,"num_write":132,"num_write_kb":2755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"44'106","reported_seq":13923,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.110283+0000","last_change":"2024-09-17T22:41:21.516305+0000","last_active":"2024-09-17T22:41:45.110283+0000","last_peered":"2024-09-17T22:41:45.110283+0000","last_clean":"2024-09-17T22:41:45.110283+0000","last_became_active":"2024-09-17T22:41:19.450587+0000","last_became_peered":"2024-09-17T22:41:19.450587+0000","last_unstale":"2024-09-17T22:41:45.110283+0000","last_undegraded":"2024-09-17T22:41:45.110283+0000","last_fullsized":"2024-09-17T22:41:45.110283+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:19:38.192774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000275164,"stat_sum":{"num_bytes":25124231,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":14,"num_read":14291,"num_read_kb":35504,"num_write":97,"num_write_kb":5076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"42'53","reported_seq":5408,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T22:41:42.047450+0000","last_change":"2024-09-17T22:41:21.454760+0000","last_active":"2024-09-17T22:41:42.047450+0000","last_peered":"2024-09-17T22:41:42.047450+0000","last_clean":"2024-09-17T22:41:42.047450+0000","last_became_active":"2024-09-17T22:41:19.449329+0000","last_became_peered":"2024-09-17T22:41:19.449329+0000","last_unstale":"2024-09-17T22:41:42.047450+0000","last_undegraded":"2024-09-17T22:41:42.047450+0000","last_fullsized":"2024-09-17T22:41:42.047450+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:49:10.136052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041055799999999999,"stat_sum":{"num_bytes":24482622,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":10,"num_read":5311,"num_read_kb":10056,"num_write":42,"num_write_kb":437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"47'51","reported_seq":2416,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.179358+0000","last_change":"2024-09-17T22:41:21.446210+0000","last_active":"2024-09-17T22:41:45.179358+0000","last_peered":"2024-09-17T22:41:45.179358+0000","last_clean":"2024-09-17T22:41:45.179358+0000","last_became_active":"2024-09-17T22:41:19.451631+0000","last_became_peered":"2024-09-17T22:41:19.451631+0000","last_unstale":"2024-09-17T22:41:45.179358+0000","last_undegraded":"2024-09-17T22:41:45.179358+0000","last_fullsized":"2024-09-17T22:41:45.179358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:57:37.622177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081193699999999999,"stat_sum":{"num_bytes":14365995,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":10,"num_read":2323,"num_read_kb":3280,"num_write":42,"num_write_kb":356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.061245+0000","last_change":"2024-09-17T22:41:11.397814+0000","last_active":"2024-09-17T22:41:45.061245+0000","last_peered":"2024-09-17T22:41:45.061245+0000","last_clean":"2024-09-17T22:41:45.061245+0000","last_became_active":"2024-09-17T22:41:09.506775+0000","last_became_peered":"2024-09-17T22:41:09.506775+0000","last_unstale":"2024-09-17T22:41:45.061245+0000","last_undegraded":"2024-09-17T22:41:45.061245+0000","last_fullsized":"2024-09-17T22:41:45.061245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:47:43.867775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"42'144","reported_seq":989,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.222267+0000","last_change":"2024-09-17T22:41:21.448698+0000","last_active":"2024-09-17T22:41:41.222267+0000","last_peered":"2024-09-17T22:41:41.222267+0000","last_clean":"2024-09-17T22:41:41.222267+0000","last_became_active":"2024-09-17T22:41:19.448966+0000","last_became_peered":"2024-09-17T22:41:19.448966+0000","last_unstale":"2024-09-17T22:41:41.222267+0000","last_undegraded":"2024-09-17T22:41:41.222267+0000","last_fullsized":"2024-09-17T22:41:41.222267+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:51:44.524786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089570000000000003,"stat_sum":{"num_bytes":4415630,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":13,"num_read":1205,"num_read_kb":3276,"num_write":190,"num_write_kb":713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":63,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.222362+0000","last_change":"2024-09-17T22:41:11.395783+0000","last_active":"2024-09-17T22:41:41.222362+0000","last_peered":"2024-09-17T22:41:41.222362+0000","last_clean":"2024-09-17T22:41:41.222362+0000","last_became_active":"2024-09-17T22:41:09.507056+0000","last_became_peered":"2024-09-17T22:41:09.507056+0000","last_unstale":"2024-09-17T22:41:41.222362+0000","last_undegraded":"2024-09-17T22:41:41.222362+0000","last_fullsized":"2024-09-17T22:41:41.222362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:13.438783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"42'152","reported_seq":669,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.222245+0000","last_change":"2024-09-17T22:41:21.448596+0000","last_active":"2024-09-17T22:41:41.222245+0000","last_peered":"2024-09-17T22:41:41.222245+0000","last_clean":"2024-09-17T22:41:41.222245+0000","last_became_active":"2024-09-17T22:41:19.451637+0000","last_became_peered":"2024-09-17T22:41:19.451637+0000","last_unstale":"2024-09-17T22:41:41.222245+0000","last_undegraded":"2024-09-17T22:41:41.222245+0000","last_fullsized":"2024-09-17T22:41:41.222245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:27:06.317680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085737499999999996,"stat_sum":{"num_bytes":13080273,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":15,"num_read":952,"num_read_kb":2502,"num_write":203,"num_write_kb":1591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.061245+0000","last_change":"2024-09-17T22:41:11.397753+0000","last_active":"2024-09-17T22:41:45.061245+0000","last_peered":"2024-09-17T22:41:45.061245+0000","last_clean":"2024-09-17T22:41:45.061245+0000","last_became_active":"2024-09-17T22:41:09.387651+0000","last_became_peered":"2024-09-17T22:41:09.387651+0000","last_unstale":"2024-09-17T22:41:45.061245+0000","last_undegraded":"2024-09-17T22:41:45.061245+0000","last_fullsized":"2024-09-17T22:41:45.061245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:14:30.915047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"39'52","reported_seq":454,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.222418+0000","last_change":"2024-09-17T22:41:21.466363+0000","last_active":"2024-09-17T22:41:41.222418+0000","last_peered":"2024-09-17T22:41:41.222418+0000","last_clean":"2024-09-17T22:41:41.222418+0000","last_became_active":"2024-09-17T22:41:19.444893+0000","last_became_peered":"2024-09-17T22:41:19.444893+0000","last_unstale":"2024-09-17T22:41:41.222418+0000","last_undegraded":"2024-09-17T22:41:41.222418+0000","last_fullsized":"2024-09-17T22:41:41.222418+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:01:17.856099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053617200000000002,"stat_sum":{"num_bytes":5475472,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":10,"num_read":354,"num_read_kb":1870,"num_write":49,"num_write_kb":884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":63,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.040778+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:41:41.040778+0000","last_peered":"2024-09-17T22:41:41.040778+0000","last_clean":"2024-09-17T22:41:41.040778+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:41:41.040778+0000","last_undegraded":"2024-09-17T22:41:41.040778+0000","last_fullsized":"2024-09-17T22:41:41.040778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"45'155","reported_seq":10182,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.110155+0000","last_change":"2024-09-17T22:41:21.516349+0000","last_active":"2024-09-17T22:41:45.110155+0000","last_peered":"2024-09-17T22:41:45.110155+0000","last_clean":"2024-09-17T22:41:45.110155+0000","last_became_active":"2024-09-17T22:41:19.451723+0000","last_became_peered":"2024-09-17T22:41:19.451723+0000","last_unstale":"2024-09-17T22:41:45.110155+0000","last_undegraded":"2024-09-17T22:41:45.110155+0000","last_fullsized":"2024-09-17T22:41:45.110155+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:04:49.129682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040033199999999999,"stat_sum":{"num_bytes":31174924,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":10463,"num_read_kb":17889,"num_write":219,"num_write_kb":2882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"44'133","reported_seq":2349,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T22:41:42.094286+0000","last_change":"2024-09-17T22:41:21.454999+0000","last_active":"2024-09-17T22:41:42.094286+0000","last_peered":"2024-09-17T22:41:42.094286+0000","last_clean":"2024-09-17T22:41:42.094286+0000","last_became_active":"2024-09-17T22:41:19.450477+0000","last_became_peered":"2024-09-17T22:41:19.450477+0000","last_unstale":"2024-09-17T22:41:42.094286+0000","last_undegraded":"2024-09-17T22:41:42.094286+0000","last_fullsized":"2024-09-17T22:41:42.094286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:00:27.747495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063898899999999996,"stat_sum":{"num_bytes":22311294,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":16,"num_read":3559,"num_read_kb":8460,"num_write":115,"num_write_kb":463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":73,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.061003+0000","last_change":"2024-09-17T22:41:11.397488+0000","last_active":"2024-09-17T22:41:45.061003+0000","last_peered":"2024-09-17T22:41:45.061003+0000","last_clean":"2024-09-17T22:41:45.061003+0000","last_became_active":"2024-09-17T22:41:09.651390+0000","last_became_peered":"2024-09-17T22:41:09.651390+0000","last_unstale":"2024-09-17T22:41:45.061003+0000","last_undegraded":"2024-09-17T22:41:45.061003+0000","last_fullsized":"2024-09-17T22:41:45.061003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:00.598921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"39'75","reported_seq":1751,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.254191+0000","last_change":"2024-09-17T22:41:21.453941+0000","last_active":"2024-09-17T22:41:41.254191+0000","last_peered":"2024-09-17T22:41:41.254191+0000","last_clean":"2024-09-17T22:41:41.254191+0000","last_became_active":"2024-09-17T22:41:19.444314+0000","last_became_peered":"2024-09-17T22:41:19.444314+0000","last_unstale":"2024-09-17T22:41:41.254191+0000","last_undegraded":"2024-09-17T22:41:41.254191+0000","last_fullsized":"2024-09-17T22:41:41.254191+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:02:49.032558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085427099999999998,"stat_sum":{"num_bytes":14740336,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":13,"num_read":1735,"num_read_kb":3824,"num_write":74,"num_write_kb":810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":63,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.034522+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:41:41.034522+0000","last_peered":"2024-09-17T22:41:41.034522+0000","last_clean":"2024-09-17T22:41:41.034522+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:41:41.034522+0000","last_undegraded":"2024-09-17T22:41:41.034522+0000","last_fullsized":"2024-09-17T22:41:41.034522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"42'78","reported_seq":5984,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.222352+0000","last_change":"2024-09-17T22:41:21.466094+0000","last_active":"2024-09-17T22:41:41.222352+0000","last_peered":"2024-09-17T22:41:41.222352+0000","last_clean":"2024-09-17T22:41:41.222352+0000","last_became_active":"2024-09-17T22:41:19.449791+0000","last_became_peered":"2024-09-17T22:41:19.449791+0000","last_unstale":"2024-09-17T22:41:41.222352+0000","last_undegraded":"2024-09-17T22:41:41.222352+0000","last_fullsized":"2024-09-17T22:41:41.222352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:52:36.557991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042775300000000001,"stat_sum":{"num_bytes":5558819,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":8,"num_read":6083,"num_read_kb":4202,"num_write":110,"num_write_kb":897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.060822+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:41:45.060822+0000","last_peered":"2024-09-17T22:41:45.060822+0000","last_clean":"2024-09-17T22:41:45.060822+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:41:45.060822+0000","last_undegraded":"2024-09-17T22:41:45.060822+0000","last_fullsized":"2024-09-17T22:41:45.060822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"42'73","reported_seq":1006,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.229472+0000","last_change":"2024-09-17T22:41:21.448409+0000","last_active":"2024-09-17T22:41:41.229472+0000","last_peered":"2024-09-17T22:41:41.229472+0000","last_clean":"2024-09-17T22:41:41.229472+0000","last_became_active":"2024-09-17T22:41:19.449279+0000","last_became_peered":"2024-09-17T22:41:19.449279+0000","last_unstale":"2024-09-17T22:41:41.229472+0000","last_undegraded":"2024-09-17T22:41:41.229472+0000","last_fullsized":"2024-09-17T22:41:41.229472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:47.262077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075217299999999997,"stat_sum":{"num_bytes":5903782,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":17,"num_read":911,"num_read_kb":650,"num_write":58,"num_write_kb":495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":72,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.061144+0000","last_change":"2024-09-17T22:41:11.398108+0000","last_active":"2024-09-17T22:41:45.061144+0000","last_peered":"2024-09-17T22:41:45.061144+0000","last_clean":"2024-09-17T22:41:45.061144+0000","last_became_active":"2024-09-17T22:41:09.388184+0000","last_became_peered":"2024-09-17T22:41:09.388184+0000","last_unstale":"2024-09-17T22:41:45.061144+0000","last_undegraded":"2024-09-17T22:41:45.061144+0000","last_fullsized":"2024-09-17T22:41:45.061144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:43.506329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":216,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T22:41:42.076994+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:41:42.076994+0000","last_peered":"2024-09-17T22:41:42.076994+0000","last_clean":"2024-09-17T22:41:42.076994+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:41:42.076994+0000","last_undegraded":"2024-09-17T22:41:42.076994+0000","last_fullsized":"2024-09-17T22:41:42.076994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"42'41","reported_seq":372,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.222175+0000","last_change":"2024-09-17T22:41:21.448156+0000","last_active":"2024-09-17T22:41:41.222175+0000","last_peered":"2024-09-17T22:41:41.222175+0000","last_clean":"2024-09-17T22:41:41.222175+0000","last_became_active":"2024-09-17T22:41:19.448956+0000","last_became_peered":"2024-09-17T22:41:19.448956+0000","last_unstale":"2024-09-17T22:41:41.222175+0000","last_undegraded":"2024-09-17T22:41:41.222175+0000","last_fullsized":"2024-09-17T22:41:41.222175+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:29:07.495009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055851200000000005,"stat_sum":{"num_bytes":2437136,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":12,"num_read":290,"num_read_kb":2144,"num_write":29,"num_write_kb":156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"47'210","reported_seq":13095,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.267354+0000","last_change":"2024-09-17T22:41:21.445421+0000","last_active":"2024-09-17T22:41:45.267354+0000","last_peered":"2024-09-17T22:41:45.267354+0000","last_clean":"2024-09-17T22:41:45.267354+0000","last_became_active":"2024-09-17T22:41:19.449003+0000","last_became_peered":"2024-09-17T22:41:19.449003+0000","last_unstale":"2024-09-17T22:41:45.267354+0000","last_undegraded":"2024-09-17T22:41:45.267354+0000","last_fullsized":"2024-09-17T22:41:45.267354+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:16:20.907361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051465400000000002,"stat_sum":{"num_bytes":10247083,"num_objects":34,"num_object_clones":10,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":17,"num_read":13662,"num_read_kb":32375,"num_write":330,"num_write_kb":1549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"46'189","reported_seq":6684,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.060911+0000","last_change":"2024-09-17T22:41:21.445728+0000","last_active":"2024-09-17T22:41:45.060911+0000","last_peered":"2024-09-17T22:41:45.060911+0000","last_clean":"2024-09-17T22:41:45.060911+0000","last_became_active":"2024-09-17T22:41:19.448011+0000","last_became_peered":"2024-09-17T22:41:19.448011+0000","last_unstale":"2024-09-17T22:41:45.060911+0000","last_undegraded":"2024-09-17T22:41:45.060911+0000","last_fullsized":"2024-09-17T22:41:45.060911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:00:46.395080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068494200000000002,"stat_sum":{"num_bytes":19986595,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":18,"num_read":6878,"num_read_kb":15255,"num_write":224,"num_write_kb":1766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"42'41","reported_seq":578,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.034534+0000","last_change":"2024-09-17T22:41:21.462738+0000","last_active":"2024-09-17T22:41:41.034534+0000","last_peered":"2024-09-17T22:41:41.034534+0000","last_clean":"2024-09-17T22:41:41.034534+0000","last_became_active":"2024-09-17T22:41:19.447215+0000","last_became_peered":"2024-09-17T22:41:19.447215+0000","last_unstale":"2024-09-17T22:41:41.034534+0000","last_undegraded":"2024-09-17T22:41:41.034534+0000","last_fullsized":"2024-09-17T22:41:41.034534+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:20:59.739946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042703000000000001,"stat_sum":{"num_bytes":6535629,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":12,"num_read":495,"num_read_kb":2871,"num_write":31,"num_write_kb":370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"44'88","reported_seq":1707,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T22:41:42.318998+0000","last_change":"2024-09-17T22:41:21.454754+0000","last_active":"2024-09-17T22:41:42.318998+0000","last_peered":"2024-09-17T22:41:42.318998+0000","last_clean":"2024-09-17T22:41:42.318998+0000","last_became_active":"2024-09-17T22:41:19.448928+0000","last_became_peered":"2024-09-17T22:41:19.448928+0000","last_unstale":"2024-09-17T22:41:42.318998+0000","last_undegraded":"2024-09-17T22:41:42.318998+0000","last_fullsized":"2024-09-17T22:41:42.318998+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:55:39.904266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000395712,"stat_sum":{"num_bytes":47362905,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":10,"num_read":2137,"num_read_kb":10345,"num_write":79,"num_write_kb":10068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"47'56","reported_seq":858,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.238429+0000","last_change":"2024-09-17T22:41:21.446562+0000","last_active":"2024-09-17T22:41:45.238429+0000","last_peered":"2024-09-17T22:41:45.238429+0000","last_clean":"2024-09-17T22:41:45.238429+0000","last_became_active":"2024-09-17T22:41:19.447631+0000","last_became_peered":"2024-09-17T22:41:19.447631+0000","last_unstale":"2024-09-17T22:41:45.238429+0000","last_undegraded":"2024-09-17T22:41:45.238429+0000","last_fullsized":"2024-09-17T22:41:45.238429+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:15:34.708107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000402657,"stat_sum":{"num_bytes":1788122,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":15,"num_read":756,"num_read_kb":1144,"num_write":43,"num_write_kb":643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"47'116","reported_seq":1903,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.335350+0000","last_change":"2024-09-17T22:41:21.516505+0000","last_active":"2024-09-17T22:41:45.335350+0000","last_peered":"2024-09-17T22:41:45.335350+0000","last_clean":"2024-09-17T22:41:45.335350+0000","last_became_active":"2024-09-17T22:41:19.450839+0000","last_became_peered":"2024-09-17T22:41:19.450839+0000","last_unstale":"2024-09-17T22:41:45.335350+0000","last_undegraded":"2024-09-17T22:41:45.335350+0000","last_fullsized":"2024-09-17T22:41:45.335350+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:16:18.126384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038538199999999998,"stat_sum":{"num_bytes":16294946,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":21,"num_read":2380,"num_read_kb":2776,"num_write":80,"num_write_kb":4656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"42'70","reported_seq":916,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T22:41:42.047520+0000","last_change":"2024-09-17T22:41:21.454987+0000","last_active":"2024-09-17T22:41:42.047520+0000","last_peered":"2024-09-17T22:41:42.047520+0000","last_clean":"2024-09-17T22:41:42.047520+0000","last_became_active":"2024-09-17T22:41:19.451864+0000","last_became_peered":"2024-09-17T22:41:19.451864+0000","last_unstale":"2024-09-17T22:41:42.047520+0000","last_undegraded":"2024-09-17T22:41:42.047520+0000","last_fullsized":"2024-09-17T22:41:42.047520+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:57:10.852093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000602379,"stat_sum":{"num_bytes":2624457,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":20,"num_read":826,"num_read_kb":2780,"num_write":61,"num_write_kb":1318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"42'88","reported_seq":753,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-17T22:41:42.095247+0000","last_change":"2024-09-17T22:41:21.446576+0000","last_active":"2024-09-17T22:41:42.095247+0000","last_peered":"2024-09-17T22:41:42.095247+0000","last_clean":"2024-09-17T22:41:42.095247+0000","last_became_active":"2024-09-17T22:41:19.447419+0000","last_became_peered":"2024-09-17T22:41:19.447419+0000","last_unstale":"2024-09-17T22:41:42.095247+0000","last_undegraded":"2024-09-17T22:41:42.095247+0000","last_fullsized":"2024-09-17T22:41:42.095247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:07:04.300024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001865071,"stat_sum":{"num_bytes":2869096,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":10,"num_read":1282,"num_read_kb":3946,"num_write":75,"num_write_kb":1495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"47'99","reported_seq":1165,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.060885+0000","last_change":"2024-09-17T22:41:21.445226+0000","last_active":"2024-09-17T22:41:45.060885+0000","last_peered":"2024-09-17T22:41:45.060885+0000","last_clean":"2024-09-17T22:41:45.060885+0000","last_became_active":"2024-09-17T22:41:19.454151+0000","last_became_peered":"2024-09-17T22:41:19.454151+0000","last_unstale":"2024-09-17T22:41:45.060885+0000","last_undegraded":"2024-09-17T22:41:45.060885+0000","last_fullsized":"2024-09-17T22:41:45.060885+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:57:55.031784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050545600000000005,"stat_sum":{"num_bytes":9251081,"num_objects":37,"num_object_clones":14,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":21,"num_read":1158,"num_read_kb":1629,"num_write":100,"num_write_kb":866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"39'39","reported_seq":5251,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-17T22:41:41.222308+0000","last_change":"2024-09-17T22:41:21.466130+0000","last_active":"2024-09-17T22:41:41.222308+0000","last_peered":"2024-09-17T22:41:41.222308+0000","last_clean":"2024-09-17T22:41:41.222308+0000","last_became_active":"2024-09-17T22:41:19.446028+0000","last_became_peered":"2024-09-17T22:41:19.446028+0000","last_unstale":"2024-09-17T22:41:41.222308+0000","last_undegraded":"2024-09-17T22:41:41.222308+0000","last_fullsized":"2024-09-17T22:41:41.222308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:21:37.479162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048487299999999998,"stat_sum":{"num_bytes":4648898,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":7,"num_read":5170,"num_read_kb":6144,"num_write":32,"num_write_kb":464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"47'222","reported_seq":1669,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T22:41:45.136186+0000","last_change":"2024-09-17T22:41:21.446196+0000","last_active":"2024-09-17T22:41:45.136186+0000","last_peered":"2024-09-17T22:41:45.136186+0000","last_clean":"2024-09-17T22:41:45.136186+0000","last_became_active":"2024-09-17T22:41:19.451662+0000","last_became_peered":"2024-09-17T22:41:19.451662+0000","last_unstale":"2024-09-17T22:41:45.136186+0000","last_undegraded":"2024-09-17T22:41:45.136186+0000","last_fullsized":"2024-09-17T22:41:45.136186+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:54:04.082193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062146200000000003,"stat_sum":{"num_bytes":10810896,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":19,"num_read":2240,"num_read_kb":5228,"num_write":329,"num_write_kb":1378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":376027049,"num_objects":773,"num_object_clones":190,"num_object_copies":1546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":773,"num_whiteouts":415,"num_read":114290,"num_read_kb":261137,"num_write":3134,"num_write_kb":52074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":11710464,"data_stored":84725566,"data_compressed":2021070,"data_compressed_allocated":7794688,"data_compressed_original":82583245,"omap_allocated":0,"internal_metadata":0},"log_size":2897,"ondisk_log_size":2897,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542156,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32640,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339200,"statfs":{"total":96636764160,"available":96603340800,"internally_reserved":0,"allocated":1486848,"data_stored":7796082,"data_compressed":92465,"data_compressed_allocated":626688,"data_compressed_original":7372800,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542154,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1675264,"data_stored":7657953,"data_compressed":280840,"data_compressed_allocated":733184,"data_compressed_original":7147520,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1064960,"data_stored":3543745,"data_compressed":83618,"data_compressed_allocated":344064,"data_compressed_original":3208740,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1642496,"data_stored":8345602,"data_compressed":181420,"data_compressed_allocated":753664,"data_compressed_original":7916324,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32992,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338848,"statfs":{"total":96636764160,"available":96602980352,"internally_reserved":0,"allocated":1851392,"data_stored":11372676,"data_compressed":182781,"data_compressed_allocated":950272,"data_compressed_original":10945684,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1343488,"data_stored":3933951,"data_compressed":113372,"data_compressed_allocated":438272,"data_compressed_original":3502080,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33916,"kb_used_data":2728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337924,"statfs":{"total":96636764160,"available":96602034176,"internally_reserved":0,"allocated":2793472,"data_stored":14319660,"data_compressed":497141,"data_compressed_allocated":1486848,"data_compressed_original":13693772,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33812,"kb_used_data":2620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338028,"statfs":{"total":96636764160,"available":96602140672,"internally_reserved":0,"allocated":2682880,"data_stored":15500658,"data_compressed":420498,"data_compressed_allocated":1429504,"data_compressed_original":14888960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[26,3,0,17,30,60],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2158592,"data_stored":19865064,"data_compressed":405808,"data_compressed_allocated":1630208,"data_compressed_original":19531173,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":987136,"data_stored":7055940,"data_compressed":118161,"data_compressed_allocated":622592,"data_compressed_original":6848512,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2236416,"data_stored":16326240,"data_compressed":434537,"data_compressed_allocated":1503232,"data_compressed_original":15937536,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2371584,"data_stored":15669511,"data_compressed":506351,"data_compressed_allocated":1589248,"data_compressed_original":15266636,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":872448,"data_stored":3734757,"data_compressed":113372,"data_compressed_allocated":438272,"data_compressed_original":3502080,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1323008,"data_stored":10583114,"data_compressed":177803,"data_compressed_allocated":913408,"data_compressed_original":10372244,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1167360,"data_stored":8146389,"data_compressed":181420,"data_compressed_allocated":753664,"data_compressed_original":7916324,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":3344551,"data_compressed":83618,"data_compressed_allocated":344064,"data_compressed_original":3208740,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:41:47.227 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T22:41:47.228 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:41:47.228 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-17T22:41:47.228 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T22:41:47.228 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T22:41:47.228 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-17T22:41:47.233 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:41:47.277 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:47.275+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:47.332 INFO:teuthology.orchestra.run.smithi115.stdout:835 punch from 0x4701f7 to 0x474ad8, (0x48e1 bytes) 2024-09-17T22:41:47.344 INFO:teuthology.orchestra.run.smithi115.stdout:836 trunc from 0x24c14e0 to 0xc5c20b4 2024-09-17T22:41:47.344 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:47.344 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:47.349 INFO:teuthology.orchestra.run.smithi115.stdout:837 writesame 0x7d5282b thru 0x7d5a2a8 data_size 0x1(0x7a7e bytes) 2024-09-17T22:41:47.353 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:41:47.365 INFO:teuthology.orchestra.run.smithi115.stdout:838 read 0x3100af2 thru 0x310418c (0x369b bytes) 2024-09-17T22:41:47.365 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:47.375 INFO:teuthology.orchestra.run.smithi115.stdout:842 trunc from 0xc5c20b4 to 0x452b380 2024-09-17T22:41:47.395 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:47.393+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:47.396 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:47.396 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T22:41:47.396 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:47.397 INFO:teuthology.orchestra.run.smithi115.stdout:843 writesame 0xa870f6b thru 0xa87c42e data_size 0x2e(0xb4c4 bytes) 2024-09-17T22:41:47.397 INFO:teuthology.orchestra.run.smithi115.stdout:844 read 0x6bf5ed9 thru 0x6bfa1f8 (0x4320 bytes) 2024-09-17T22:41:47.406 INFO:teuthology.orchestra.run.smithi115.stdout:845 writesame 0xce2331b thru 0xce2bacb data_size 0x1(0x87b1 bytes) 2024-09-17T22:41:47.415 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:41:47.420 INFO:teuthology.orchestra.run.smithi115.stdout:846 trunc from 0xce2bacc to 0xbbcb9b 2024-09-17T22:41:47.430 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:47.430 INFO:teuthology.orchestra.run.smithi115.stdout:851 read 0x3390bf thru 0x344e66 (0xbda8 bytes) 2024-09-17T22:41:47.450 INFO:teuthology.orchestra.run.smithi115.stdout:853 write 0xc43fd41 thru 0xc449ee6 (0xa1a6 bytes) 2024-09-17T22:41:47.454 INFO:teuthology.orchestra.run.smithi115.stdout:855 punch from 0x6b23f9d to 0x6b2f1ab, (0xb20e bytes) 2024-09-17T22:41:47.454 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:47.454 INFO:teuthology.orchestra.run.smithi115.stdout:859 punch from 0xa78e52c to 0xa799fd3, (0xbaa7 bytes) 2024-09-17T22:41:47.454 INFO:teuthology.orchestra.run.smithi115.stdout:862 read 0xfbc3a6 thru 0xfcab80 (0xe7db bytes) 2024-09-17T22:41:47.455 INFO:teuthology.orchestra.run.smithi115.stdout:863 punch from 0x9af1670 to 0x9b006c9, (0xf059 bytes) 2024-09-17T22:41:47.455 INFO:teuthology.orchestra.run.smithi115.stdout:867 read 0xa55e05b thru 0xa56d467 (0xf40d bytes) 2024-09-17T22:41:47.455 INFO:teuthology.orchestra.run.smithi115.stdout:868 trunc from 0xc449ee7 to 0xbdb2812 2024-09-17T22:41:47.466 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:47.464+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:47.466 INFO:teuthology.orchestra.run.smithi115.stdout:871 writesame 0xbf73ceb thru 0xbf80076 data_size 0xa(0xc38c bytes) 2024-09-17T22:41:47.474 INFO:teuthology.orchestra.run.smithi115.stdout:873 write 0xe416ba5 thru 0xe423d5e (0xd1ba bytes) 2024-09-17T22:41:47.478 INFO:teuthology.orchestra.run.smithi115.stdout:874 writesame 0x185c42c thru 0x1866cbf data_size 0x2(0xa894 bytes) 2024-09-17T22:41:47.485 INFO:teuthology.orchestra.run.smithi115.stdout:876 punch from 0x80c41cc to 0x80ca07b, (0x5eaf bytes) 2024-09-17T22:41:47.485 INFO:teuthology.orchestra.run.smithi115.stdout:877 write 0xc73e2b7 thru 0xc74d16f (0xeeb9 bytes) 2024-09-17T22:41:47.489 INFO:teuthology.orchestra.run.smithi115.stdout:878 write 0x2a8bc52 thru 0x2a911df (0x558e bytes) 2024-09-17T22:41:47.491 INFO:teuthology.orchestra.run.smithi115.stdout:879 read 0x31b1ad3 thru 0x31c1a05 (0xff33 bytes) 2024-09-17T22:41:47.491 INFO:teuthology.orchestra.run.smithi115.stdout:880 write 0xc9bd4ee thru 0xc9c704d (0x9b60 bytes) 2024-09-17T22:41:47.501 INFO:teuthology.orchestra.run.smithi115.stdout:882 write 0xe4575e5 thru 0xe45c400 (0x4e1c bytes) 2024-09-17T22:41:47.503 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:47.503 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:47.504 INFO:teuthology.orchestra.run.smithi115.stdout:884 writesame 0xa4dec82 thru 0xa4e145d data_size 0x2(0x27dc bytes) 2024-09-17T22:41:47.512 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:47.512 INFO:teuthology.orchestra.run.smithi115.stdout:889 read 0x3a35168 thru 0x3a3d100 (0x7f99 bytes) 2024-09-17T22:41:47.512 INFO:teuthology.orchestra.run.smithi115.stdout:890 read 0x57365e2 thru 0x5743699 (0xd0b8 bytes) 2024-09-17T22:41:47.512 INFO:teuthology.orchestra.run.smithi115.stdout:truncating to largest ever: 0xe73cd35 2024-09-17T22:41:47.512 INFO:teuthology.orchestra.run.smithi115.stdout:892 trunc from 0xe45c401 to 0xe73cd35 2024-09-17T22:41:47.513 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:41:47.517 INFO:teuthology.orchestra.run.smithi115.stdout:893 write 0x9fb4c01 thru 0x9fc3a29 (0xee29 bytes) 2024-09-17T22:41:47.520 INFO:teuthology.orchestra.run.smithi115.stdout:894 writesame 0x90bbcba thru 0x90c14e3 data_size 0x7a(0x582a bytes) 2024-09-17T22:41:47.532 INFO:teuthology.orchestra.run.smithi115.stdout:896 trunc from 0xe73cd35 to 0x70b0647 2024-09-17T22:41:47.547 INFO:teuthology.orchestra.run.smithi115.stdout:898 punch from 0x6856125 to 0x68634b4, (0xd38f bytes) 2024-09-17T22:41:47.551 INFO:teuthology.orchestra.run.smithi115.stdout:899 trunc from 0x70b0647 to 0x3e72832 2024-09-17T22:41:47.558 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0: {} 2024-09-17T22:41:47.558 INFO:teuthology.orchestra.run.smithi115.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:41:47.560 INFO:teuthology.orchestra.run.smithi115.stdout:900 write 0x2266a0e thru 0x226ba86 (0x5079 bytes) 2024-09-17T22:41:47.567 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:47.565+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:47.576 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:41:47.576 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:41:47.580 INFO:teuthology.orchestra.run.smithi115.stdout:901 writesame 0xec4a723 thru 0xec5133d data_size 0x1b(0x6c1b bytes) 2024-09-17T22:41:47.580 INFO:teuthology.orchestra.run.smithi115.stdout:902 read 0x796bdf3 thru 0x797add9 (0xefe7 bytes) 2024-09-17T22:41:47.581 INFO:teuthology.orchestra.run.smithi115.stdout:903 read 0x4a229f2 thru 0x4a2408f (0x169e bytes) 2024-09-17T22:41:47.581 INFO:teuthology.orchestra.run.smithi115.stdout:904 read 0xd6c70ec thru 0xd6d0400 (0x9315 bytes) 2024-09-17T22:41:47.581 INFO:teuthology.orchestra.run.smithi115.stdout:905 write 0x5c21692 thru 0x5c243f8 (0x2d67 bytes) 2024-09-17T22:41:47.583 INFO:teuthology.orchestra.run.smithi115.stdout:906 punch from 0x6ecb856 to 0x6edace7, (0xf491 bytes) 2024-09-17T22:41:47.588 INFO:teuthology.orchestra.run.smithi115.stdout:907 trunc from 0xec5133e to 0xa681806 2024-09-17T22:41:47.593 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:41:47.593 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:41:47.594 INFO:teuthology.orchestra.run.smithi115.stdout:908 read 0x1cfa3fe thru 0x1cfaab5 (0x6b8 bytes) 2024-09-17T22:41:47.594 INFO:teuthology.orchestra.run.smithi115.stdout:910 writesame 0x124f305 thru 0x1254cd7 data_size 0x2d(0x59d3 bytes) 2024-09-17T22:41:47.596 INFO:teuthology.orchestra.run.smithi115.stdout:911 writesame 0x439d345 thru 0x43a59ec data_size 0x3e(0x86a8 bytes) 2024-09-17T22:41:47.599 INFO:teuthology.orchestra.run.smithi115.stdout:914 writesame 0x24ae859 thru 0x24b98e2 data_size 0x2(0xb08a bytes) 2024-09-17T22:41:47.610 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:41:47.610 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:41:47.611 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:47.611 INFO:teuthology.orchestra.run.smithi115.stdout:916 punch from 0x96b0caa to 0x96b3916, (0x2c6c bytes) 2024-09-17T22:41:47.611 INFO:teuthology.orchestra.run.smithi115.stdout:918 read 0x5a0b762 thru 0x5a1a30b (0xebaa bytes) 2024-09-17T22:41:47.611 INFO:teuthology.orchestra.run.smithi115.stdout:920 writesame 0x658542e thru 0x6588a99 data_size 0x12(0x366c bytes) 2024-09-17T22:41:47.613 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:47.613 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:47.613 INFO:teuthology.orchestra.run.smithi115.stdout:923 writesame 0x4db187e thru 0x4db8b28 data_size 0x39(0x72ab bytes) 2024-09-17T22:41:47.616 INFO:teuthology.orchestra.run.smithi115.stdout:924 read 0x107ada thru 0x10aa6e (0x2f95 bytes) 2024-09-17T22:41:47.617 INFO:teuthology.orchestra.run.smithi115.stdout:925 read 0xa8bdfd thru 0xa95404 (0x9608 bytes) 2024-09-17T22:41:47.626 INFO:teuthology.orchestra.run.smithi115.stdout:926 writesame 0xcec4718 thru 0xced0300 data_size 0xf(0xbbe9 bytes) 2024-09-17T22:41:47.626 INFO:teuthology.orchestra.run.smithi115.stdout:930 write 0x52af354 thru 0x52b43d2 (0x507f bytes) 2024-09-17T22:41:47.628 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:41:47.628 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:41:47.635 INFO:teuthology.orchestra.run.smithi115.stdout:931 write 0xe107b07 thru 0xe117427 (0xf921 bytes) 2024-09-17T22:41:47.644 INFO:teuthology.orchestra.run.smithi115.stdout:932 trunc from 0xe117428 to 0x8ddbb44 2024-09-17T22:41:47.646 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:41:47.646 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:41:47.653 INFO:teuthology.orchestra.run.smithi115.stdout:933 write 0x2e86f0 thru 0x2f08fd (0x820e bytes) 2024-09-17T22:41:47.655 INFO:teuthology.orchestra.run.smithi115.stdout:934 read 0x51fda82 thru 0x520b3b1 (0xd930 bytes) 2024-09-17T22:41:47.658 INFO:teuthology.orchestra.run.smithi115.stdout:935 trunc from 0x8ddbb44 to 0xbbf73d6 2024-09-17T22:41:47.663 INFO:teuthology.orchestra.run.smithi115.stdout:936 writesame 0xba22626 thru 0xba22d6d data_size 0x4(0x748 bytes) 2024-09-17T22:41:47.665 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:41:47.665 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:41:47.666 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:47.667 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:47.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:47.667 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:47.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.667 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113242 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 7974912~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.667 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.971137+0000", 2024-09-17T22:41:47.667 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.6912108589999999, 2024-09-17T22:41:47.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038092916999999997, 2024-09-17T22:41:47.667 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.667 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.667 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.668 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.668 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.668 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.668 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113242 2024-09-17T22:41:47.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.668 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.971137+0000", 2024-09-17T22:41:47.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.971137+0000", 2024-09-17T22:41:47.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.972843+0000", 2024-09-17T22:41:47.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001706579 2024-09-17T22:41:47.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.972846+0000", 2024-09-17T22:41:47.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1269999999999999e-06 2024-09-17T22:41:47.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.972853+0000", 2024-09-17T22:41:47.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4409999999999998e-06 2024-09-17T22:41:47.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.972857+0000", 2024-09-17T22:41:47.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1910000000000002e-06 2024-09-17T22:41:47.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.009080+0000", 2024-09-17T22:41:47.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036223334000000003 2024-09-17T22:41:47.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.009147+0000", 2024-09-17T22:41:47.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6617000000000003e-05 2024-09-17T22:41:47.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.009230+0000", 2024-09-17T22:41:47.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2627999999999996e-05 2024-09-17T22:41:47.671 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.671 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.671 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.671 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113256 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 8060928~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.671 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.973140+0000", 2024-09-17T22:41:47.672 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.6892078719999999, 2024-09-17T22:41:47.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038100811999999998, 2024-09-17T22:41:47.672 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.672 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.672 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.672 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.672 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.672 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.672 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113256 2024-09-17T22:41:47.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.672 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973140+0000", 2024-09-17T22:41:47.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973140+0000", 2024-09-17T22:41:47.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973140+0000", 2024-09-17T22:41:47.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1500000000000001e-07 2024-09-17T22:41:47.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973141+0000", 2024-09-17T22:41:47.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0160000000000001e-06 2024-09-17T22:41:47.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973145+0000", 2024-09-17T22:41:47.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7979999999999999e-06 2024-09-17T22:41:47.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973152+0000", 2024-09-17T22:41:47.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3719999999999999e-06 2024-09-17T22:41:47.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.011161+0000", 2024-09-17T22:41:47.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038008620999999999 2024-09-17T22:41:47.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.011195+0000", 2024-09-17T22:41:47.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4830999999999999e-05 2024-09-17T22:41:47.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.011240+0000", 2024-09-17T22:41:47.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4959000000000001e-05 2024-09-17T22:41:47.676 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.676 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.676 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.676 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113257 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 8067072~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.676 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.973155+0000", 2024-09-17T22:41:47.676 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.6891922360000002, 2024-09-17T22:41:47.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038267190999999999, 2024-09-17T22:41:47.676 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.676 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.676 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.677 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113257 2024-09-17T22:41:47.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.677 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973155+0000", 2024-09-17T22:41:47.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973155+0000", 2024-09-17T22:41:47.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973155+0000", 2024-09-17T22:41:47.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.28e-07 2024-09-17T22:41:47.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973160+0000", 2024-09-17T22:41:47.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7450000000000003e-06 2024-09-17T22:41:47.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973164+0000", 2024-09-17T22:41:47.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3620000000000002e-06 2024-09-17T22:41:47.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973166+0000", 2024-09-17T22:41:47.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4109999999999999e-06 2024-09-17T22:41:47.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.011266+0000", 2024-09-17T22:41:47.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.03810036 2024-09-17T22:41:47.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.011331+0000", 2024-09-17T22:41:47.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4523000000000005e-05 2024-09-17T22:41:47.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.011422+0000", 2024-09-17T22:41:47.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.1562000000000002e-05 2024-09-17T22:41:47.680 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.680 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.680 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.680 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113258 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 8073216~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.681 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.973169+0000", 2024-09-17T22:41:47.681 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.6891783469999999, 2024-09-17T22:41:47.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038357256999999999, 2024-09-17T22:41:47.681 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.681 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.681 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.681 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.681 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.681 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.681 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113258 2024-09-17T22:41:47.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.681 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973169+0000", 2024-09-17T22:41:47.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973169+0000", 2024-09-17T22:41:47.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973169+0000", 2024-09-17T22:41:47.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2399999999999999e-07 2024-09-17T22:41:47.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973170+0000", 2024-09-17T22:41:47.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0130000000000001e-06 2024-09-17T22:41:47.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973174+0000", 2024-09-17T22:41:47.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7280000000000001e-06 2024-09-17T22:41:47.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973177+0000", 2024-09-17T22:41:47.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7329999999999999e-06 2024-09-17T22:41:47.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.011448+0000", 2024-09-17T22:41:47.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038271634999999998 2024-09-17T22:41:47.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.011482+0000", 2024-09-17T22:41:47.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3880000000000001e-05 2024-09-17T22:41:47.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.011526+0000", 2024-09-17T22:41:47.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4044e-05 2024-09-17T22:41:47.684 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.685 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.685 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.685 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113260 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 8079360~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.685 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.973180+0000", 2024-09-17T22:41:47.685 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.6891677469999999, 2024-09-17T22:41:47.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038489950000000002, 2024-09-17T22:41:47.685 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.685 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.685 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.685 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.685 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.686 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.686 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113260 2024-09-17T22:41:47.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.686 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973180+0000", 2024-09-17T22:41:47.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973180+0000", 2024-09-17T22:41:47.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973206+0000", 2024-09-17T22:41:47.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6665999999999998e-05 2024-09-17T22:41:47.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973209+0000", 2024-09-17T22:41:47.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.447e-06 2024-09-17T22:41:47.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973215+0000", 2024-09-17T22:41:47.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4119999999999998e-06 2024-09-17T22:41:47.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973220+0000", 2024-09-17T22:41:47.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1000000000000003e-06 2024-09-17T22:41:47.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.011555+0000", 2024-09-17T22:41:47.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038334745000000003 2024-09-17T22:41:47.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.011603+0000", 2024-09-17T22:41:47.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7917999999999998e-05 2024-09-17T22:41:47.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.011670+0000", 2024-09-17T22:41:47.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6662000000000007e-05 2024-09-17T22:41:47.689 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.689 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.689 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.689 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113261 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 8085504~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.690 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.973232+0000", 2024-09-17T22:41:47.690 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.6891150980000003, 2024-09-17T22:41:47.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038542165000000003, 2024-09-17T22:41:47.690 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.690 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.690 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.690 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113261 2024-09-17T22:41:47.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.690 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973232+0000", 2024-09-17T22:41:47.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.973232+0000", 2024-09-17T22:41:47.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.974844+0000", 2024-09-17T22:41:47.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001611225 2024-09-17T22:41:47.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.974853+0000", 2024-09-17T22:41:47.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7370000000000001e-06 2024-09-17T22:41:47.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.974861+0000", 2024-09-17T22:41:47.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3540000000000002e-06 2024-09-17T22:41:47.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.974865+0000", 2024-09-17T22:41:47.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1470000000000003e-06 2024-09-17T22:41:47.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.011693+0000", 2024-09-17T22:41:47.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.036827886999999997 2024-09-17T22:41:47.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.011728+0000", 2024-09-17T22:41:47.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4926999999999999e-05 2024-09-17T22:41:47.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.011775+0000", 2024-09-17T22:41:47.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6888000000000002e-05 2024-09-17T22:41:47.694 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.694 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.694 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.694 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113274 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 8165376~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.694 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.975151+0000", 2024-09-17T22:41:47.694 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.687196304, 2024-09-17T22:41:47.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038140827000000002, 2024-09-17T22:41:47.694 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.694 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.694 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.695 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.695 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.695 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113274 2024-09-17T22:41:47.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.695 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975151+0000", 2024-09-17T22:41:47.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975151+0000", 2024-09-17T22:41:47.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975307+0000", 2024-09-17T22:41:47.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00015541700000000001 2024-09-17T22:41:47.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975310+0000", 2024-09-17T22:41:47.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.444e-06 2024-09-17T22:41:47.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975318+0000", 2024-09-17T22:41:47.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7530000000000001e-06 2024-09-17T22:41:47.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975323+0000", 2024-09-17T22:41:47.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.22e-06 2024-09-17T22:41:47.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.013185+0000", 2024-09-17T22:41:47.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037862241999999997 2024-09-17T22:41:47.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.013220+0000", 2024-09-17T22:41:47.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4666000000000003e-05 2024-09-17T22:41:47.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.013292+0000", 2024-09-17T22:41:47.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2084999999999998e-05 2024-09-17T22:41:47.698 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.698 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.698 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.699 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113276 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 8177664~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.699 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.975351+0000", 2024-09-17T22:41:47.699 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.686996835, 2024-09-17T22:41:47.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038136468, 2024-09-17T22:41:47.699 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.699 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.699 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.699 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113276 2024-09-17T22:41:47.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.699 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975351+0000", 2024-09-17T22:41:47.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975351+0000", 2024-09-17T22:41:47.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975462+0000", 2024-09-17T22:41:47.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011131799999999999 2024-09-17T22:41:47.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975465+0000", 2024-09-17T22:41:47.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.061e-06 2024-09-17T22:41:47.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975472+0000", 2024-09-17T22:41:47.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4829999999999996e-06 2024-09-17T22:41:47.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975477+0000", 2024-09-17T22:41:47.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5249999999999996e-06 2024-09-17T22:41:47.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.013406+0000", 2024-09-17T22:41:47.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037929364 2024-09-17T22:41:47.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.013441+0000", 2024-09-17T22:41:47.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4836999999999999e-05 2024-09-17T22:41:47.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.013487+0000", 2024-09-17T22:41:47.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5880000000000001e-05 2024-09-17T22:41:47.703 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.703 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.703 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.703 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113277 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 8183808~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.703 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.975488+0000", 2024-09-17T22:41:47.703 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.6868596199999999, 2024-09-17T22:41:47.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038096107999999997, 2024-09-17T22:41:47.703 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.703 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.704 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.704 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.704 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.704 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.704 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113277 2024-09-17T22:41:47.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.704 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975488+0000", 2024-09-17T22:41:47.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975488+0000", 2024-09-17T22:41:47.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975645+0000", 2024-09-17T22:41:47.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000157312 2024-09-17T22:41:47.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975648+0000", 2024-09-17T22:41:47.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5890000000000001e-06 2024-09-17T22:41:47.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975653+0000", 2024-09-17T22:41:47.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8609999999999997e-06 2024-09-17T22:41:47.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975657+0000", 2024-09-17T22:41:47.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2490000000000003e-06 2024-09-17T22:41:47.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.013507+0000", 2024-09-17T22:41:47.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037849687 2024-09-17T22:41:47.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.013540+0000", 2024-09-17T22:41:47.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3312000000000002e-05 2024-09-17T22:41:47.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.013584+0000", 2024-09-17T22:41:47.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4097999999999997e-05 2024-09-17T22:41:47.707 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.707 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.707 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.707 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113279 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 8196096~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.707 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.975709+0000", 2024-09-17T22:41:47.707 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.686638072, 2024-09-17T22:41:47.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038104405000000001, 2024-09-17T22:41:47.708 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.708 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.708 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.708 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113279 2024-09-17T22:41:47.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.708 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975709+0000", 2024-09-17T22:41:47.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975709+0000", 2024-09-17T22:41:47.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975829+0000", 2024-09-17T22:41:47.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011959999999999999 2024-09-17T22:41:47.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975832+0000", 2024-09-17T22:41:47.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1269999999999999e-06 2024-09-17T22:41:47.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975845+0000", 2024-09-17T22:41:47.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.238e-05 2024-09-17T22:41:47.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975849+0000", 2024-09-17T22:41:47.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4220000000000002e-06 2024-09-17T22:41:47.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.013735+0000", 2024-09-17T22:41:47.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037886033999999999 2024-09-17T22:41:47.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.013770+0000", 2024-09-17T22:41:47.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4737999999999998e-05 2024-09-17T22:41:47.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.013814+0000", 2024-09-17T22:41:47.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4104000000000003e-05 2024-09-17T22:41:47.711 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.711 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.711 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.712 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113280 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 8202240~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.712 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.975857+0000", 2024-09-17T22:41:47.712 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.6864906149999999, 2024-09-17T22:41:47.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038131380999999999, 2024-09-17T22:41:47.712 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.712 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.712 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.712 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.712 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.712 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.712 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113280 2024-09-17T22:41:47.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.713 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975857+0000", 2024-09-17T22:41:47.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975857+0000", 2024-09-17T22:41:47.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975991+0000", 2024-09-17T22:41:47.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013454100000000001 2024-09-17T22:41:47.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.975995+0000", 2024-09-17T22:41:47.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2739999999999999e-06 2024-09-17T22:41:47.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.976001+0000", 2024-09-17T22:41:47.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4060000000000002e-06 2024-09-17T22:41:47.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.976008+0000", 2024-09-17T22:41:47.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8560000000000001e-06 2024-09-17T22:41:47.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.013834+0000", 2024-09-17T22:41:47.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037825833000000003 2024-09-17T22:41:47.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.013902+0000", 2024-09-17T22:41:47.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8447000000000004e-05 2024-09-17T22:41:47.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.013988+0000", 2024-09-17T22:41:47.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6024000000000002e-05 2024-09-17T22:41:47.716 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.716 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.716 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.716 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113281 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 8208384~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.716 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.976017+0000", 2024-09-17T22:41:47.716 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.6863303260000002, 2024-09-17T22:41:47.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038064803000000001, 2024-09-17T22:41:47.716 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.717 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.717 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.717 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.717 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.717 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.717 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113281 2024-09-17T22:41:47.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.717 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.976017+0000", 2024-09-17T22:41:47.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.976017+0000", 2024-09-17T22:41:47.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.976183+0000", 2024-09-17T22:41:47.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00016631299999999999 2024-09-17T22:41:47.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.976186+0000", 2024-09-17T22:41:47.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4940000000000002e-06 2024-09-17T22:41:47.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.976191+0000", 2024-09-17T22:41:47.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3040000000000004e-06 2024-09-17T22:41:47.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.976195+0000", 2024-09-17T22:41:47.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6660000000000001e-06 2024-09-17T22:41:47.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.014002+0000", 2024-09-17T22:41:47.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037806944000000002 2024-09-17T22:41:47.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.014036+0000", 2024-09-17T22:41:47.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4063000000000002e-05 2024-09-17T22:41:47.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.014082+0000", 2024-09-17T22:41:47.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6019000000000001e-05 2024-09-17T22:41:47.720 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.720 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.720 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.720 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113299 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 8318976~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.721 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.978463+0000", 2024-09-17T22:41:47.721 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.6838840149999998, 2024-09-17T22:41:47.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038078267999999998, 2024-09-17T22:41:47.721 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.721 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.721 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.721 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.721 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.721 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.721 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113299 2024-09-17T22:41:47.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.721 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.978463+0000", 2024-09-17T22:41:47.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.978463+0000", 2024-09-17T22:41:47.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.978648+0000", 2024-09-17T22:41:47.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00018484099999999999 2024-09-17T22:41:47.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.978651+0000", 2024-09-17T22:41:47.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.559e-06 2024-09-17T22:41:47.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.978656+0000", 2024-09-17T22:41:47.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6720000000000003e-06 2024-09-17T22:41:47.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.978660+0000", 2024-09-17T22:41:47.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2479999999999998e-06 2024-09-17T22:41:47.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.016382+0000", 2024-09-17T22:41:47.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037722050999999999 2024-09-17T22:41:47.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.016454+0000", 2024-09-17T22:41:47.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2568e-05 2024-09-17T22:41:47.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.016542+0000", 2024-09-17T22:41:47.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7329000000000004e-05 2024-09-17T22:41:47.725 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.725 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.725 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.725 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113303 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 8343552~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.725 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.979059+0000", 2024-09-17T22:41:47.725 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.6832888590000001, 2024-09-17T22:41:47.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038086364999999997, 2024-09-17T22:41:47.725 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.725 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.725 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.725 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.726 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.726 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.726 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113303 2024-09-17T22:41:47.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.726 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979059+0000", 2024-09-17T22:41:47.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979059+0000", 2024-09-17T22:41:47.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979244+0000", 2024-09-17T22:41:47.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000185818 2024-09-17T22:41:47.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979247+0000", 2024-09-17T22:41:47.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.491e-06 2024-09-17T22:41:47.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979252+0000", 2024-09-17T22:41:47.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.558e-06 2024-09-17T22:41:47.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979256+0000", 2024-09-17T22:41:47.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.827e-06 2024-09-17T22:41:47.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.016995+0000", 2024-09-17T22:41:47.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037738870000000001 2024-09-17T22:41:47.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.017063+0000", 2024-09-17T22:41:47.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7411999999999998e-05 2024-09-17T22:41:47.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.017145+0000", 2024-09-17T22:41:47.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2389e-05 2024-09-17T22:41:47.729 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.729 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.729 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.729 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113305 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 8355840~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.729 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.979353+0000", 2024-09-17T22:41:47.730 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.6829942149999999, 2024-09-17T22:41:47.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038110179000000001, 2024-09-17T22:41:47.730 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.730 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.730 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.730 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.730 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.730 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.730 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113305 2024-09-17T22:41:47.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.730 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979353+0000", 2024-09-17T22:41:47.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979353+0000", 2024-09-17T22:41:47.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979508+0000", 2024-09-17T22:41:47.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00015506600000000001 2024-09-17T22:41:47.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979516+0000", 2024-09-17T22:41:47.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6599999999999995e-06 2024-09-17T22:41:47.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979521+0000", 2024-09-17T22:41:47.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.04e-06 2024-09-17T22:41:47.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979525+0000", 2024-09-17T22:41:47.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5990000000000002e-06 2024-09-17T22:41:47.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.017306+0000", 2024-09-17T22:41:47.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037781666999999998 2024-09-17T22:41:47.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.017375+0000", 2024-09-17T22:41:47.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8967000000000006e-05 2024-09-17T22:41:47.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.017463+0000", 2024-09-17T22:41:47.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8179999999999999e-05 2024-09-17T22:41:47.733 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.734 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.734 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.734 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113306 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 8361984~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.734 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.979534+0000", 2024-09-17T22:41:47.734 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.6828139769999999, 2024-09-17T22:41:47.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038053277000000003, 2024-09-17T22:41:47.734 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.734 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.734 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.734 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.734 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.734 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.734 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113306 2024-09-17T22:41:47.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.735 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979534+0000", 2024-09-17T22:41:47.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979534+0000", 2024-09-17T22:41:47.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979534+0000", 2024-09-17T22:41:47.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.06e-07 2024-09-17T22:41:47.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979542+0000", 2024-09-17T22:41:47.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.622e-06 2024-09-17T22:41:47.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979550+0000", 2024-09-17T22:41:47.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9859999999999994e-06 2024-09-17T22:41:47.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979555+0000", 2024-09-17T22:41:47.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9570000000000001e-06 2024-09-17T22:41:47.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.017492+0000", 2024-09-17T22:41:47.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037937815999999999 2024-09-17T22:41:47.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.017538+0000", 2024-09-17T22:41:47.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5587000000000002e-05 2024-09-17T22:41:47.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.017587+0000", 2024-09-17T22:41:47.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8702999999999998e-05 2024-09-17T22:41:47.738 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.738 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.738 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.738 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113307 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 8368128~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.738 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.979563+0000", 2024-09-17T22:41:47.738 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.6827847399999998, 2024-09-17T22:41:47.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038208837000000002, 2024-09-17T22:41:47.738 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.738 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.738 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.739 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.739 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.739 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.739 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113307 2024-09-17T22:41:47.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.739 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979563+0000", 2024-09-17T22:41:47.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979563+0000", 2024-09-17T22:41:47.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979717+0000", 2024-09-17T22:41:47.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00015433800000000001 2024-09-17T22:41:47.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979720+0000", 2024-09-17T22:41:47.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6110000000000001e-06 2024-09-17T22:41:47.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979724+0000", 2024-09-17T22:41:47.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6519999999999997e-06 2024-09-17T22:41:47.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979728+0000", 2024-09-17T22:41:47.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8260000000000003e-06 2024-09-17T22:41:47.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.017618+0000", 2024-09-17T22:41:47.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037889421999999999 2024-09-17T22:41:47.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.017685+0000", 2024-09-17T22:41:47.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7565999999999997e-05 2024-09-17T22:41:47.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.017772+0000", 2024-09-17T22:41:47.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6421999999999994e-05 2024-09-17T22:41:47.742 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.742 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.742 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.742 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113308 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 8374272~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.742 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.979742+0000", 2024-09-17T22:41:47.743 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.6826052540000003, 2024-09-17T22:41:47.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038142294, 2024-09-17T22:41:47.743 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.743 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.743 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.743 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.743 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.743 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.743 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113308 2024-09-17T22:41:47.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.743 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979742+0000", 2024-09-17T22:41:47.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979742+0000", 2024-09-17T22:41:47.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979874+0000", 2024-09-17T22:41:47.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013163899999999999 2024-09-17T22:41:47.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979878+0000", 2024-09-17T22:41:47.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6320000000000001e-06 2024-09-17T22:41:47.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979886+0000", 2024-09-17T22:41:47.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5089999999999993e-06 2024-09-17T22:41:47.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979892+0000", 2024-09-17T22:41:47.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8889999999999999e-06 2024-09-17T22:41:47.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.017789+0000", 2024-09-17T22:41:47.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037896738999999999 2024-09-17T22:41:47.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.017831+0000", 2024-09-17T22:41:47.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1897999999999998e-05 2024-09-17T22:41:47.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.017885+0000", 2024-09-17T22:41:47.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3987999999999999e-05 2024-09-17T22:41:47.746 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.746 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.747 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.747 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113309 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 8380416~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.747 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.979902+0000", 2024-09-17T22:41:47.747 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.6824454740000001, 2024-09-17T22:41:47.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038163704999999999, 2024-09-17T22:41:47.747 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.747 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.747 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.747 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.747 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.747 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.747 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113309 2024-09-17T22:41:47.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.748 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979902+0000", 2024-09-17T22:41:47.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.979902+0000", 2024-09-17T22:41:47.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.980006+0000", 2024-09-17T22:41:47.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010439500000000001 2024-09-17T22:41:47.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.980009+0000", 2024-09-17T22:41:47.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3939999999999999e-06 2024-09-17T22:41:47.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.980013+0000", 2024-09-17T22:41:47.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5900000000000001e-06 2024-09-17T22:41:47.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.980017+0000", 2024-09-17T22:41:47.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4929999999999998e-06 2024-09-17T22:41:47.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.017914+0000", 2024-09-17T22:41:47.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037897426999999997 2024-09-17T22:41:47.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.017982+0000", 2024-09-17T22:41:47.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7917999999999996e-05 2024-09-17T22:41:47.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.018066+0000", 2024-09-17T22:41:47.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3487999999999998e-05 2024-09-17T22:41:47.751 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.751 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.751 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.751 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:113310 3.1b 3:debbac8f:::rbd_data.115f14f2d1d5.0000000000000006:e [read 8386560~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-17T22:41:47.751 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:44.980023+0000", 2024-09-17T22:41:47.751 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.682324071, 2024-09-17T22:41:47.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038155284999999997, 2024-09-17T22:41:47.752 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.752 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.752 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:47.752 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.752 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.752 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.752 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 113310 2024-09-17T22:41:47.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.752 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.980023+0000", 2024-09-17T22:41:47.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.980023+0000", 2024-09-17T22:41:47.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.980145+0000", 2024-09-17T22:41:47.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012194 2024-09-17T22:41:47.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.980148+0000", 2024-09-17T22:41:47.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8679999999999999e-06 2024-09-17T22:41:47.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.980155+0000", 2024-09-17T22:41:47.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5429999999999999e-06 2024-09-17T22:41:47.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:44.980160+0000", 2024-09-17T22:41:47.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4999999999999999e-06 2024-09-17T22:41:47.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.018085+0000", 2024-09-17T22:41:47.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.037924642000000001 2024-09-17T22:41:47.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.018126+0000", 2024-09-17T22:41:47.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1452999999999998e-05 2024-09-17T22:41:47.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:47.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:45.018179+0000", 2024-09-17T22:41:47.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2339000000000002e-05 2024-09-17T22:41:47.755 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.755 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.755 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.755 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.755 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.755 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:47.757 INFO:teuthology.orchestra.run.smithi115.stdout:938 write 0xd7b1d92 thru 0xd7b9575 (0x77e4 bytes) 2024-09-17T22:41:47.757 INFO:teuthology.orchestra.run.smithi115.stdout:939 read 0x36d40c9 thru 0x36d99e5 (0x591d bytes) 2024-09-17T22:41:47.757 INFO:teuthology.orchestra.run.smithi115.stdout:940 read 0xa4b33b2 thru 0xa4bac22 (0x7871 bytes) 2024-09-17T22:41:47.757 INFO:teuthology.orchestra.run.smithi115.stdout:941 read 0x7b1518d thru 0x7b1b5c1 (0x6435 bytes) 2024-09-17T22:41:47.757 INFO:teuthology.orchestra.run.smithi115.stdout:942 read 0x68be3fc thru 0x68c714c (0x8d51 bytes) 2024-09-17T22:41:47.758 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:41:47.758 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:41:47.759 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:47.754+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:47.759 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:41:47.759 INFO:teuthology.orchestra.run.smithi115.stdout:943 trunc from 0xd7b9576 to 0x50af403 2024-09-17T22:41:47.760 INFO:teuthology.orchestra.run.smithi115.stdout:944 read 0x36c09d2 thru 0x36c990e (0x8f3d bytes) 2024-09-17T22:41:47.761 INFO:teuthology.orchestra.run.smithi115.stdout:945 trunc from 0x50af403 to 0x2ce6b4a 2024-09-17T22:41:47.761 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:47.761 INFO:teuthology.orchestra.run.smithi115.stdout:947 trunc from 0x2ce6b4a to 0xc673587 2024-09-17T22:41:47.761 INFO:teuthology.orchestra.run.smithi115.stdout:948 trunc from 0xc673587 to 0x53447bb 2024-09-17T22:41:47.761 INFO:teuthology.orchestra.run.smithi115.stdout:949 punch from 0x1798d51 to 0x179f9f1, (0x6ca0 bytes) 2024-09-17T22:41:47.761 INFO:teuthology.orchestra.run.smithi115.stdout:950 punch from 0x34e5c16 to 0x34e94fb, (0x38e5 bytes) 2024-09-17T22:41:47.761 INFO:teuthology.orchestra.run.smithi115.stdout:951 write 0x3f6d341 thru 0x3f7142d (0x40ed bytes) 2024-09-17T22:41:47.767 INFO:teuthology.orchestra.run.smithi115.stdout:952 write 0xd65abe8 thru 0xd66a2c4 (0xf6dd bytes) 2024-09-17T22:41:47.771 INFO:teuthology.orchestra.run.smithi115.stdout:954 write 0x1576b48 thru 0x158418b (0xd644 bytes) 2024-09-17T22:41:47.772 INFO:teuthology.orchestra.run.smithi115.stdout:955 read 0x9b22e74 thru 0x9b31fb3 (0xf140 bytes) 2024-09-17T22:41:47.785 INFO:teuthology.orchestra.run.smithi115.stdout:956 trunc from 0xd66a2c5 to 0x2b0cdd7 2024-09-17T22:41:47.797 INFO:teuthology.orchestra.run.smithi115.stdout:truncating to largest ever: 0xec42916 2024-09-17T22:41:47.811 INFO:teuthology.orchestra.run.smithi115.stdout:957 trunc from 0x2b0cdd7 to 0xec42916 2024-09-17T22:41:47.817 INFO:teuthology.orchestra.run.smithi115.stdout:959 read 0x4c3db5c thru 0x4c475a4 (0x9a49 bytes) 2024-09-17T22:41:47.817 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:47.817 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:47.818 INFO:teuthology.orchestra.run.smithi115.stdout:963 read 0x597d63a thru 0x5984c55 (0x761c bytes) 2024-09-17T22:41:47.818 INFO:teuthology.orchestra.run.smithi115.stdout:964 writesame 0xe97f211 thru 0xe97fdfa data_size 0x32(0xbea bytes) 2024-09-17T22:41:47.823 INFO:teuthology.orchestra.run.smithi115.stdout:965 trunc from 0xec42916 to 0xc328569 2024-09-17T22:41:47.832 INFO:teuthology.orchestra.run.smithi115.stdout:966 trunc from 0xc328569 to 0xbd87105 2024-09-17T22:41:47.837 INFO:teuthology.orchestra.run.smithi115.stdout:968 write 0x554af7e thru 0x554d401 (0x2484 bytes) 2024-09-17T22:41:47.850 INFO:teuthology.orchestra.run.smithi115.stdout:969 trunc from 0xbd87105 to 0x3546efd 2024-09-17T22:41:47.858 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:47.855+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:47.861 INFO:teuthology.orchestra.run.smithi115.stdout:970 read 0x2a127b0 thru 0x2a1747c (0x4ccd bytes) 2024-09-17T22:41:47.861 INFO:teuthology.orchestra.run.smithi115.stdout:973 punch from 0xff588e to 0x1002d74, (0xd4e6 bytes) 2024-09-17T22:41:47.879 INFO:teuthology.orchestra.run.smithi115.stdout:974 write 0xe3568e8 thru 0xe359015 (0x272e bytes) 2024-09-17T22:41:47.882 INFO:teuthology.orchestra.run.smithi115.stdout:975 writesame 0x87a0190 thru 0x87a1733 data_size 0xa(0x15a4 bytes) 2024-09-17T22:41:47.885 INFO:teuthology.orchestra.run.smithi115.stdout:976 read 0x567f701 thru 0x568af9e (0xb89e bytes) 2024-09-17T22:41:47.901 INFO:teuthology.orchestra.run.smithi115.stdout:980 trunc from 0xe359016 to 0x2d8f61 2024-09-17T22:41:47.906 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:47.906 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:47.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.907 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:143893 3.1e 3:7f749196:::rbd_data.115fed663bc2.0000000000000005:head [stat out=16b,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T22:41:47.907 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:47.904281+0000", 2024-09-17T22:41:47.907 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.00057826399999999999, 2024-09-17T22:41:47.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00060348300000000004, 2024-09-17T22:41:47.907 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.907 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.907 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "waiting for sub ops", 2024-09-17T22:41:47.907 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:47.907 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:47.907 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:47.907 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 143893 2024-09-17T22:41:47.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.908 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:47.904281+0000", 2024-09-17T22:41:47.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:47.904281+0000", 2024-09-17T22:41:47.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:47.904282+0000", 2024-09-17T22:41:47.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8999999999999995e-07 2024-09-17T22:41:47.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:47.904287+0000", 2024-09-17T22:41:47.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1449999999999999e-06 2024-09-17T22:41:47.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:47.904301+0000", 2024-09-17T22:41:47.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3488e-05 2024-09-17T22:41:47.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:47.904309+0000", 2024-09-17T22:41:47.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9789999999999993e-06 2024-09-17T22:41:47.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:47.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:47.904387+0000", 2024-09-17T22:41:47.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8405999999999994e-05 2024-09-17T22:41:47.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:47.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:47.904464+0000", 2024-09-17T22:41:47.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6358000000000003e-05 2024-09-17T22:41:47.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 2", 2024-09-17T22:41:47.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:47.904584+0000", 2024-09-17T22:41:47.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000119976 2024-09-17T22:41:47.911 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.911 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.911 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.911 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4447.0:143891 3.1a e48/23)", 2024-09-17T22:41:47.911 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:47.904900+0000", 2024-09-17T22:41:47.911 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4294967295.9999595, 2024-09-17T22:41:47.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9604000000000001e-05, 2024-09-17T22:41:47.911 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:47.912 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:47.912 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:47.912 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:47.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:47.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:47.904900+0000", 2024-09-17T22:41:47.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:47.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:47.904900+0000", 2024-09-17T22:41:47.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:47.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:47.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:47.904901+0000", 2024-09-17T22:41:47.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7499999999999999e-07 2024-09-17T22:41:47.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:47.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:47.904906+0000", 2024-09-17T22:41:47.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4389999999999999e-06 2024-09-17T22:41:47.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:47.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:47.904916+0000", 2024-09-17T22:41:47.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0057999999999999e-05 2024-09-17T22:41:47.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:47.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:47.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:47.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:47.904923+0000", 2024-09-17T22:41:47.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5889999999999999e-06 2024-09-17T22:41:47.914 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.914 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:47.914 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.914 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:47.914 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T22:41:47.914 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 2 2024-09-17T22:41:47.914 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:47.915 INFO:teuthology.orchestra.run.smithi115.stdout:982 punch from 0x152f12 to 0x161009, (0xe0f7 bytes) 2024-09-17T22:41:47.917 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:41:47.932 INFO:teuthology.orchestra.run.smithi115.stdout:984 write 0xb32099c thru 0xb3220cf (0x1734 bytes) 2024-09-17T22:41:47.940 INFO:teuthology.orchestra.run.smithi115.stdout:985 write 0xc40a430 thru 0xc40ef99 (0x4b6a bytes) 2024-09-17T22:41:47.943 INFO:teuthology.orchestra.run.smithi115.stdout:986 write 0x98d91d5 thru 0x98e4cd4 (0xbb00 bytes) 2024-09-17T22:41:47.951 INFO:teuthology.orchestra.run.smithi115.stdout:987 write 0x9e96469 thru 0x9e9d1d9 (0x6d71 bytes) 2024-09-17T22:41:47.953 INFO:teuthology.orchestra.run.smithi115.stdout:988 read 0xa1ddad3 thru 0xa1e3e77 (0x63a5 bytes) 2024-09-17T22:41:47.954 INFO:teuthology.orchestra.run.smithi115.stdout:989 read 0x5b5fb07 thru 0x5b6259c (0x2a96 bytes) 2024-09-17T22:41:47.954 INFO:teuthology.orchestra.run.smithi115.stdout:990 punch from 0xb5eeefb to 0xb5f4bdb, (0x5ce0 bytes) 2024-09-17T22:41:47.954 INFO:teuthology.orchestra.run.smithi115.stdout:992 write 0xb5bb0f thru 0xb5cbc1 (0x10b3 bytes) 2024-09-17T22:41:47.954 INFO:teuthology.orchestra.run.smithi115.stdout:993 write 0x7476efe thru 0x7478c37 (0x1d3a bytes) 2024-09-17T22:41:47.959 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:47.956+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:47.959 INFO:teuthology.orchestra.run.smithi115.stdout:994 writesame 0x14dc089 thru 0x14de07c data_size 0xa(0x1ff4 bytes) 2024-09-17T22:41:47.960 INFO:teuthology.orchestra.run.smithi115.stdout:995 write 0xbdd5079 thru 0xbde0410 (0xb398 bytes) 2024-09-17T22:41:47.966 INFO:teuthology.orchestra.run.smithi115.stdout:997 punch from 0x3b27710 to 0x3b2d76a, (0x605a bytes) 2024-09-17T22:41:47.966 INFO:teuthology.orchestra.run.smithi115.stdout:998 write 0x4e336be thru 0x4e39e44 (0x6787 bytes) 2024-09-17T22:41:47.969 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:47.969 INFO:teuthology.orchestra.run.smithi115.stdout:1000 write 0x11c9bac thru 0x11cce66 (0x32bb bytes) 2024-09-17T22:41:47.971 INFO:teuthology.orchestra.run.smithi115.stdout:1002 read 0x6afc146 thru 0x6afc3b7 (0x272 bytes) 2024-09-17T22:41:47.971 INFO:teuthology.orchestra.run.smithi115.stdout:1003 read 0x3002ce1 thru 0x3007f73 (0x5293 bytes) 2024-09-17T22:41:47.971 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:47.971 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:47.971 INFO:teuthology.orchestra.run.smithi115.stdout:1006 write 0x6713f11 thru 0x6716181 (0x2271 bytes) 2024-09-17T22:41:47.976 INFO:teuthology.orchestra.run.smithi115.stdout:1008 trunc from 0xc40ef9a to 0xa5a2acf 2024-09-17T22:41:47.988 INFO:teuthology.orchestra.run.smithi115.stdout:1011 read 0x36a46ef thru 0x36af4e8 (0xadfa bytes) 2024-09-17T22:41:47.989 INFO:teuthology.orchestra.run.smithi115.stdout:1013 trunc from 0xa5a2acf to 0xa4643a5 2024-09-17T22:41:47.997 INFO:teuthology.orchestra.run.smithi115.stdout:1014 trunc from 0xa4643a5 to 0xe9dc48b 2024-09-17T22:41:48.003 INFO:teuthology.orchestra.run.smithi115.stdout:1015 punch from 0x9cb57fb to 0x9cc4f20, (0xf725 bytes) 2024-09-17T22:41:48.006 INFO:teuthology.orchestra.run.smithi115.stdout:1017 trunc from 0xe9dc48b to 0xbdc4545 2024-09-17T22:41:48.011 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:48.019 INFO:teuthology.orchestra.run.smithi115.stdout:1019 writesame 0xdeaf752 thru 0xdeba9b6 data_size 0xd(0xb265 bytes) 2024-09-17T22:41:48.019 INFO:teuthology.orchestra.run.smithi115.stdout:1020 write 0x3d31e1c thru 0x3d3d374 (0xb559 bytes) 2024-09-17T22:41:48.022 INFO:teuthology.orchestra.run.smithi115.stdout:1021 write 0x80beca2 thru 0x80ccb3e (0xde9d bytes) 2024-09-17T22:41:48.024 INFO:teuthology.orchestra.run.smithi115.stdout:1022 write 0x1308f3f thru 0x1310d28 (0x7dea bytes) 2024-09-17T22:41:48.026 INFO:teuthology.orchestra.run.smithi115.stdout:1023 punch from 0x98d5b7b to 0x98e2bdb, (0xd060 bytes) 2024-09-17T22:41:48.026 INFO:teuthology.orchestra.run.smithi115.stdout:1025 read 0x61be3ef thru 0x61be9d7 (0x5e9 bytes) 2024-09-17T22:41:48.026 INFO:teuthology.orchestra.run.smithi115.stdout:1026 read 0x41d248e thru 0x41d847e (0x5ff1 bytes) 2024-09-17T22:41:48.026 INFO:teuthology.orchestra.run.smithi115.stdout:1028 punch from 0x35ee002 to 0x35fc3c9, (0xe3c7 bytes) 2024-09-17T22:41:48.036 INFO:teuthology.orchestra.run.smithi115.stdout:1029 trunc from 0xdeba9b7 to 0x5468775 2024-09-17T22:41:48.049 INFO:teuthology.orchestra.run.smithi115.stdout:1030 trunc from 0x5468775 to 0x4fad799 2024-09-17T22:41:48.060 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:48.057+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:48.067 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:48.068 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:48.068 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:48.068 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:48.068 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:48.072 INFO:teuthology.orchestra.run.smithi115.stdout:1031 write 0xd4e82f0 thru 0xd4f2efb (0xac0c bytes) 2024-09-17T22:41:48.075 INFO:teuthology.orchestra.run.smithi115.stdout:1032 read 0x15a1afb thru 0x15aa825 (0x8d2b bytes) 2024-09-17T22:41:48.075 INFO:teuthology.orchestra.run.smithi115.stdout:1033 write 0x102354a thru 0x1024fd3 (0x1a8a bytes) 2024-09-17T22:41:48.077 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:41:48.078 INFO:teuthology.orchestra.run.smithi115.stdout:1034 writesame 0x3174ba6 thru 0x3181585 data_size 0x28(0xc9e0 bytes) 2024-09-17T22:41:48.078 INFO:teuthology.orchestra.run.smithi115.stdout:1036 read 0x6e4fde4 thru 0x6e53841 (0x3a5e bytes) 2024-09-17T22:41:48.079 INFO:teuthology.orchestra.run.smithi115.stdout:1038 write 0xe5fe7a thru 0xe6efed (0xf174 bytes) 2024-09-17T22:41:48.080 INFO:teuthology.orchestra.run.smithi115.stdout:1040 writesame 0xa067206 thru 0xa072ae9 data_size 0x2(0xb8e4 bytes) 2024-09-17T22:41:48.091 INFO:teuthology.orchestra.run.smithi115.stdout:1041 write 0x278d1d1 thru 0x279910f (0xbf3f bytes) 2024-09-17T22:41:48.093 INFO:teuthology.orchestra.run.smithi115.stdout:1042 write 0x1e5b3bd thru 0x1e5b739 (0x37d bytes) 2024-09-17T22:41:48.094 INFO:teuthology.orchestra.run.smithi115.stdout:1043 read 0x52e0d33 thru 0x52e1e35 (0x1103 bytes) 2024-09-17T22:41:48.094 INFO:teuthology.orchestra.run.smithi115.stdout:1044 read 0xc63e491 thru 0xc6408e1 (0x2451 bytes) 2024-09-17T22:41:48.094 INFO:teuthology.orchestra.run.smithi115.stdout:1046 write 0xbe2a10b thru 0xbe36276 (0xc16c bytes) 2024-09-17T22:41:48.096 INFO:teuthology.orchestra.run.smithi115.stdout:1047 writesame 0x17e2aa4 thru 0x17eacbe data_size 0x1(0x821b bytes) 2024-09-17T22:41:48.097 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:41:48.111 INFO:teuthology.orchestra.run.smithi115.stdout:1050 writesame 0x3bb4719 thru 0x3bbf726 data_size 0x5(0xb00e bytes) 2024-09-17T22:41:48.119 INFO:teuthology.orchestra.run.smithi115.stdout:1051 trunc from 0xd4f2efc to 0xbcfa3af 2024-09-17T22:41:48.125 INFO:teuthology.orchestra.run.smithi115.stdout:1052 read 0x1c9c3e thru 0x1d55a8 (0xb96b bytes) 2024-09-17T22:41:48.126 INFO:teuthology.orchestra.run.smithi115.stdout:1053 write 0xaea6fa3 thru 0xaea8a58 (0x1ab6 bytes) 2024-09-17T22:41:48.129 INFO:teuthology.orchestra.run.smithi115.stdout:1054 punch from 0x22832b to 0x229266, (0xf3b bytes) 2024-09-17T22:41:48.129 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:48.129 INFO:teuthology.orchestra.run.smithi115.stdout:1056 write 0x5e0cb11 thru 0x5e1bdd6 (0xf2c6 bytes) 2024-09-17T22:41:48.133 INFO:teuthology.orchestra.run.smithi115.stdout:1057 writesame 0x2fad864 thru 0x2fb1847 data_size 0x74(0x3fe4 bytes) 2024-09-17T22:41:48.141 INFO:teuthology.orchestra.run.smithi115.stdout:1059 trunc from 0xbcfa3af to 0x653dd49 2024-09-17T22:41:48.161 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:48.158+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:48.161 INFO:teuthology.orchestra.run.smithi115.stdout:1060 trunc from 0x653dd49 to 0xdd4ba93 2024-09-17T22:41:48.166 INFO:teuthology.orchestra.run.smithi115.stdout:1062 write 0x364c83a thru 0x365691c (0xa0e3 bytes) 2024-09-17T22:41:48.168 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:48.168 INFO:teuthology.orchestra.run.smithi115.stdout:1067 punch from 0x6815180 to 0x6815b8c, (0xa0c bytes) 2024-09-17T22:41:48.168 INFO:teuthology.orchestra.run.smithi115.stdout:1068 read 0x39e63bc thru 0x39ec675 (0x62ba bytes) 2024-09-17T22:41:48.168 INFO:teuthology.orchestra.run.smithi115.stdout:1069 write 0xabb2f10 thru 0xabc1a6b (0xeb5c bytes) 2024-09-17T22:41:48.172 INFO:teuthology.orchestra.run.smithi115.stdout:1070 read 0x8f0fbcb thru 0x8f106c7 (0xafd bytes) 2024-09-17T22:41:48.172 INFO:teuthology.orchestra.run.smithi115.stdout:1071 writesame 0x5d4bfc2 thru 0x5d4e85c data_size 0x4d(0x289b bytes) 2024-09-17T22:41:48.175 INFO:teuthology.orchestra.run.smithi115.stdout:1072 writesame 0xab121b2 thru 0xab1bdfd data_size 0xe(0x9c4c bytes) 2024-09-17T22:41:48.179 INFO:teuthology.orchestra.run.smithi115.stdout:1073 writesame 0xdd04db thru 0xdd3917 data_size 0x1(0x343d bytes) 2024-09-17T22:41:48.184 INFO:teuthology.orchestra.run.smithi115.stdout:1074 write 0x58c2e91 thru 0x58d1fbf (0xf12f bytes) 2024-09-17T22:41:48.186 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:48.186 INFO:teuthology.orchestra.run.smithi115.stdout:1077 writesame 0x1c09e82 thru 0x1c184dd data_size 0x2e(0xe65c bytes) 2024-09-17T22:41:48.187 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:48.187 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:48.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:48.188 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:48.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.188 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72639 3.0 3:04644e28:::rbd_data.115fe9ac8535.0000000000000005:9 [read 4050944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.188 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.725903+0000", 2024-09-17T22:41:48.188 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4585270030000004, 2024-09-17T22:41:48.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023334941000000001, 2024-09-17T22:41:48.188 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.188 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.188 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.188 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.188 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.188 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.188 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72639 2024-09-17T22:41:48.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.189 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725903+0000", 2024-09-17T22:41:48.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725903+0000", 2024-09-17T22:41:48.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725904+0000", 2024-09-17T22:41:48.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0499999999999999e-07 2024-09-17T22:41:48.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725906+0000", 2024-09-17T22:41:48.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1430000000000001e-06 2024-09-17T22:41:48.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725912+0000", 2024-09-17T22:41:48.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8860000000000001e-06 2024-09-17T22:41:48.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725916+0000", 2024-09-17T22:41:48.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4229999999999998e-06 2024-09-17T22:41:48.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749094+0000", 2024-09-17T22:41:48.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023177729000000001 2024-09-17T22:41:48.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749167+0000", 2024-09-17T22:41:48.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2665999999999998e-05 2024-09-17T22:41:48.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749238+0000", 2024-09-17T22:41:48.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1688999999999998e-05 2024-09-17T22:41:48.192 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.192 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.192 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.192 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72640 3.18 3:1997a52f:::rbd_data.115fe9ac8535.0000000000000006:9 [read 4034560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.192 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.725936+0000", 2024-09-17T22:41:48.193 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4584942749999996, 2024-09-17T22:41:48.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023281237999999999, 2024-09-17T22:41:48.193 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.193 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.193 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.193 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.193 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.193 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.193 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72640 2024-09-17T22:41:48.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.193 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725936+0000", 2024-09-17T22:41:48.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725936+0000", 2024-09-17T22:41:48.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725937+0000", 2024-09-17T22:41:48.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1500000000000005e-07 2024-09-17T22:41:48.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725939+0000", 2024-09-17T22:41:48.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1110000000000002e-06 2024-09-17T22:41:48.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725945+0000", 2024-09-17T22:41:48.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4810000000000003e-06 2024-09-17T22:41:48.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725950+0000", 2024-09-17T22:41:48.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T22:41:48.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749083+0000", 2024-09-17T22:41:48.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023133236000000001 2024-09-17T22:41:48.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749153+0000", 2024-09-17T22:41:48.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0253000000000007e-05 2024-09-17T22:41:48.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749217+0000", 2024-09-17T22:41:48.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4084000000000005e-05 2024-09-17T22:41:48.196 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.196 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.197 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.197 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72641 3.0 3:0299d250:::rbd_data.115fe9ac8535.0000000000000000:9 [read 3502080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.197 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.725958+0000", 2024-09-17T22:41:48.197 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4584727490000002, 2024-09-17T22:41:48.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023452997999999999, 2024-09-17T22:41:48.197 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.197 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.197 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.197 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.197 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.197 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.197 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72641 2024-09-17T22:41:48.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.198 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725958+0000", 2024-09-17T22:41:48.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725958+0000", 2024-09-17T22:41:48.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725958+0000", 2024-09-17T22:41:48.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7300000000000001e-07 2024-09-17T22:41:48.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725960+0000", 2024-09-17T22:41:48.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6050000000000001e-06 2024-09-17T22:41:48.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725967+0000", 2024-09-17T22:41:48.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6720000000000004e-06 2024-09-17T22:41:48.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.200 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.200 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725972+0000", 2024-09-17T22:41:48.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3819999999999997e-06 2024-09-17T22:41:48.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.200 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.200 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749276+0000", 2024-09-17T22:41:48.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023304248 2024-09-17T22:41:48.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.200 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.200 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749344+0000", 2024-09-17T22:41:48.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7564999999999994e-05 2024-09-17T22:41:48.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.201 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749411+0000", 2024-09-17T22:41:48.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7052999999999997e-05 2024-09-17T22:41:48.201 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.201 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.201 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.201 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72643 3.18 3:1997a52f:::rbd_data.115fe9ac8535.0000000000000006:9 [read 3256320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.201 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.725978+0000", 2024-09-17T22:41:48.201 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.45845235, 2024-09-17T22:41:48.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023430645999999999, 2024-09-17T22:41:48.201 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.202 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.202 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.202 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.202 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.202 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.202 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72643 2024-09-17T22:41:48.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.202 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725978+0000", 2024-09-17T22:41:48.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725978+0000", 2024-09-17T22:41:48.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725979+0000", 2024-09-17T22:41:48.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:41:48.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725981+0000", 2024-09-17T22:41:48.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.294e-06 2024-09-17T22:41:48.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725989+0000", 2024-09-17T22:41:48.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7319999999999998e-06 2024-09-17T22:41:48.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.725993+0000", 2024-09-17T22:41:48.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6750000000000001e-06 2024-09-17T22:41:48.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.205 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749267+0000", 2024-09-17T22:41:48.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023273540999999998 2024-09-17T22:41:48.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.205 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.205 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749340+0000", 2024-09-17T22:41:48.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2799999999999994e-05 2024-09-17T22:41:48.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.205 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.205 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749409+0000", 2024-09-17T22:41:48.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9194000000000002e-05 2024-09-17T22:41:48.205 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.205 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.206 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.206 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72647 3.0 3:04644e28:::rbd_data.115fe9ac8535.0000000000000005:9 [read 3481600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.206 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726034+0000", 2024-09-17T22:41:48.206 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4583963850000004, 2024-09-17T22:41:48.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023543489000000001, 2024-09-17T22:41:48.206 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.206 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.206 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.206 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.206 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.206 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.206 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72647 2024-09-17T22:41:48.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.207 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726034+0000", 2024-09-17T22:41:48.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726034+0000", 2024-09-17T22:41:48.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726034+0000", 2024-09-17T22:41:48.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:41:48.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726037+0000", 2024-09-17T22:41:48.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.092e-06 2024-09-17T22:41:48.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726043+0000", 2024-09-17T22:41:48.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4969999999999998e-06 2024-09-17T22:41:48.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726048+0000", 2024-09-17T22:41:48.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7129999999999996e-06 2024-09-17T22:41:48.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749436+0000", 2024-09-17T22:41:48.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023388234000000001 2024-09-17T22:41:48.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749510+0000", 2024-09-17T22:41:48.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3930000000000005e-05 2024-09-17T22:41:48.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749578+0000", 2024-09-17T22:41:48.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7606000000000003e-05 2024-09-17T22:41:48.210 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.210 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.210 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.210 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72648 3.18 3:1997a52f:::rbd_data.115fe9ac8535.0000000000000006:9 [read 3526656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.210 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726055+0000", 2024-09-17T22:41:48.210 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4583755660000008, 2024-09-17T22:41:48.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023537293000000001, 2024-09-17T22:41:48.210 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.210 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.210 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.210 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.211 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.211 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.211 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72648 2024-09-17T22:41:48.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.211 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726055+0000", 2024-09-17T22:41:48.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726055+0000", 2024-09-17T22:41:48.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726055+0000", 2024-09-17T22:41:48.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:41:48.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726057+0000", 2024-09-17T22:41:48.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0480000000000001e-06 2024-09-17T22:41:48.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726064+0000", 2024-09-17T22:41:48.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1609999999999997e-06 2024-09-17T22:41:48.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726069+0000", 2024-09-17T22:41:48.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8879999999999996e-06 2024-09-17T22:41:48.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749451+0000", 2024-09-17T22:41:48.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023381246000000001 2024-09-17T22:41:48.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749527+0000", 2024-09-17T22:41:48.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6094999999999996e-05 2024-09-17T22:41:48.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749592+0000", 2024-09-17T22:41:48.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5475000000000006e-05 2024-09-17T22:41:48.214 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.214 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.214 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.215 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72649 3.0 3:0299d250:::rbd_data.115fe9ac8535.0000000000000000:9 [read 3772416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.215 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726098+0000", 2024-09-17T22:41:48.215 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4583326809999999, 2024-09-17T22:41:48.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023657582, 2024-09-17T22:41:48.215 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.215 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.215 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.215 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.215 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.215 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.215 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72649 2024-09-17T22:41:48.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.216 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726098+0000", 2024-09-17T22:41:48.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726098+0000", 2024-09-17T22:41:48.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726098+0000", 2024-09-17T22:41:48.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0900000000000001e-07 2024-09-17T22:41:48.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726103+0000", 2024-09-17T22:41:48.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0600000000000001e-06 2024-09-17T22:41:48.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726111+0000", 2024-09-17T22:41:48.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1810000000000001e-06 2024-09-17T22:41:48.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726117+0000", 2024-09-17T22:41:48.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6030000000000001e-06 2024-09-17T22:41:48.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749603+0000", 2024-09-17T22:41:48.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023485552999999999 2024-09-17T22:41:48.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749678+0000", 2024-09-17T22:41:48.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5446999999999998e-05 2024-09-17T22:41:48.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749755+0000", 2024-09-17T22:41:48.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7028999999999997e-05 2024-09-17T22:41:48.219 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.219 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.219 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.219 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72654 3.0 3:04644e28:::rbd_data.115fe9ac8535.0000000000000005:9 [read 3751936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.219 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726150+0000", 2024-09-17T22:41:48.219 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4582801639999996, 2024-09-17T22:41:48.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023793538999999999, 2024-09-17T22:41:48.219 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.219 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.219 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.219 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.220 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.220 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.220 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72654 2024-09-17T22:41:48.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.220 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726150+0000", 2024-09-17T22:41:48.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726150+0000", 2024-09-17T22:41:48.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726151+0000", 2024-09-17T22:41:48.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.89e-07 2024-09-17T22:41:48.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726153+0000", 2024-09-17T22:41:48.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0040000000000002e-06 2024-09-17T22:41:48.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726159+0000", 2024-09-17T22:41:48.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9000000000000003e-06 2024-09-17T22:41:48.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726163+0000", 2024-09-17T22:41:48.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7559999999999999e-06 2024-09-17T22:41:48.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749793+0000", 2024-09-17T22:41:48.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023630003 2024-09-17T22:41:48.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749869+0000", 2024-09-17T22:41:48.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5401000000000005e-05 2024-09-17T22:41:48.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749944+0000", 2024-09-17T22:41:48.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5086000000000006e-05 2024-09-17T22:41:48.223 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.223 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.223 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.223 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72655 3.18 3:1997a52f:::rbd_data.115fe9ac8535.0000000000000006:9 [read 3735552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.223 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726170+0000", 2024-09-17T22:41:48.224 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4582608300000004, 2024-09-17T22:41:48.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023772537, 2024-09-17T22:41:48.224 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.224 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.224 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.224 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.224 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.224 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.224 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72655 2024-09-17T22:41:48.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.224 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726170+0000", 2024-09-17T22:41:48.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726170+0000", 2024-09-17T22:41:48.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726170+0000", 2024-09-17T22:41:48.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.77e-07 2024-09-17T22:41:48.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726180+0000", 2024-09-17T22:41:48.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8199999999999992e-06 2024-09-17T22:41:48.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726186+0000", 2024-09-17T22:41:48.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3740000000000003e-06 2024-09-17T22:41:48.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726191+0000", 2024-09-17T22:41:48.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.259e-06 2024-09-17T22:41:48.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749785+0000", 2024-09-17T22:41:48.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023593139999999999 2024-09-17T22:41:48.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749867+0000", 2024-09-17T22:41:48.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2699000000000002e-05 2024-09-17T22:41:48.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749942+0000", 2024-09-17T22:41:48.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4868000000000002e-05 2024-09-17T22:41:48.227 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.227 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.227 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.228 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72656 3.18 3:1997a52f:::rbd_data.115fe9ac8535.0000000000000006:9 [read 3796992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.228 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726199+0000", 2024-09-17T22:41:48.228 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.45823182, 2024-09-17T22:41:48.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023907336000000001, 2024-09-17T22:41:48.228 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.228 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.228 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.228 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.228 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.228 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.228 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72656 2024-09-17T22:41:48.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.229 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726199+0000", 2024-09-17T22:41:48.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726199+0000", 2024-09-17T22:41:48.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726199+0000", 2024-09-17T22:41:48.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3399999999999999e-07 2024-09-17T22:41:48.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726201+0000", 2024-09-17T22:41:48.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0109999999999999e-06 2024-09-17T22:41:48.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726207+0000", 2024-09-17T22:41:48.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5249999999999996e-06 2024-09-17T22:41:48.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726212+0000", 2024-09-17T22:41:48.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8910000000000003e-06 2024-09-17T22:41:48.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749969+0000", 2024-09-17T22:41:48.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023757256000000001 2024-09-17T22:41:48.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750039+0000", 2024-09-17T22:41:48.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9925999999999994e-05 2024-09-17T22:41:48.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750106+0000", 2024-09-17T22:41:48.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7193000000000005e-05 2024-09-17T22:41:48.232 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.232 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.232 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.232 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72657 3.0 3:0299d250:::rbd_data.115fe9ac8535.0000000000000000:9 [read 4042752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.232 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726218+0000", 2024-09-17T22:41:48.232 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4582127220000007, 2024-09-17T22:41:48.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023899917999999999, 2024-09-17T22:41:48.232 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.232 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.232 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.233 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.233 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.233 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.233 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72657 2024-09-17T22:41:48.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.233 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726218+0000", 2024-09-17T22:41:48.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726218+0000", 2024-09-17T22:41:48.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726218+0000", 2024-09-17T22:41:48.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7099999999999997e-07 2024-09-17T22:41:48.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726220+0000", 2024-09-17T22:41:48.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.95e-06 2024-09-17T22:41:48.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726226+0000", 2024-09-17T22:41:48.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8069999999999996e-06 2024-09-17T22:41:48.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726230+0000", 2024-09-17T22:41:48.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.544e-06 2024-09-17T22:41:48.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.749983+0000", 2024-09-17T22:41:48.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023752932000000001 2024-09-17T22:41:48.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750051+0000", 2024-09-17T22:41:48.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8052000000000005e-05 2024-09-17T22:41:48.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750118+0000", 2024-09-17T22:41:48.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6261999999999997e-05 2024-09-17T22:41:48.236 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.236 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.236 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.236 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72662 3.0 3:04644e28:::rbd_data.115fe9ac8535.0000000000000005:9 [read 4022272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.236 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726261+0000", 2024-09-17T22:41:48.236 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4581696139999991, 2024-09-17T22:41:48.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02403953, 2024-09-17T22:41:48.237 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.237 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.237 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.237 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.237 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.237 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.237 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72662 2024-09-17T22:41:48.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.237 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726261+0000", 2024-09-17T22:41:48.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726261+0000", 2024-09-17T22:41:48.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726261+0000", 2024-09-17T22:41:48.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2e-07 2024-09-17T22:41:48.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726263+0000", 2024-09-17T22:41:48.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9199999999999998e-06 2024-09-17T22:41:48.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726269+0000", 2024-09-17T22:41:48.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.767e-06 2024-09-17T22:41:48.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726274+0000", 2024-09-17T22:41:48.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6770000000000003e-06 2024-09-17T22:41:48.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750158+0000", 2024-09-17T22:41:48.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023883824000000001 2024-09-17T22:41:48.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750228+0000", 2024-09-17T22:41:48.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0813000000000001e-05 2024-09-17T22:41:48.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750300+0000", 2024-09-17T22:41:48.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2008999999999994e-05 2024-09-17T22:41:48.240 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.240 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.240 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.240 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72663 3.18 3:1997a52f:::rbd_data.115fe9ac8535.0000000000000006:9 [read 4005888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.240 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726280+0000", 2024-09-17T22:41:48.240 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4581504850000009, 2024-09-17T22:41:48.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023998477000000001, 2024-09-17T22:41:48.241 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.241 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.241 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.241 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.241 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.241 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.241 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72663 2024-09-17T22:41:48.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.241 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726280+0000", 2024-09-17T22:41:48.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726280+0000", 2024-09-17T22:41:48.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726281+0000", 2024-09-17T22:41:48.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4099999999999999e-07 2024-09-17T22:41:48.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726283+0000", 2024-09-17T22:41:48.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1409999999999999e-06 2024-09-17T22:41:48.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726289+0000", 2024-09-17T22:41:48.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8830000000000003e-06 2024-09-17T22:41:48.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726301+0000", 2024-09-17T22:41:48.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2618e-05 2024-09-17T22:41:48.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750147+0000", 2024-09-17T22:41:48.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023846201000000001 2024-09-17T22:41:48.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750213+0000", 2024-09-17T22:41:48.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5724999999999998e-05 2024-09-17T22:41:48.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750278+0000", 2024-09-17T22:41:48.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5368000000000002e-05 2024-09-17T22:41:48.244 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.244 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.244 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.244 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72664 3.18 3:1997a52f:::rbd_data.115fe9ac8535.0000000000000006:9 [read 3215360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.245 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726307+0000", 2024-09-17T22:41:48.245 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4581232879999995, 2024-09-17T22:41:48.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024121613, 2024-09-17T22:41:48.245 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.245 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.245 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.245 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.245 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.245 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.245 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72664 2024-09-17T22:41:48.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.245 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726307+0000", 2024-09-17T22:41:48.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726307+0000", 2024-09-17T22:41:48.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726308+0000", 2024-09-17T22:41:48.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4700000000000002e-07 2024-09-17T22:41:48.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726315+0000", 2024-09-17T22:41:48.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4279999999999999e-06 2024-09-17T22:41:48.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726322+0000", 2024-09-17T22:41:48.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.601e-06 2024-09-17T22:41:48.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726326+0000", 2024-09-17T22:41:48.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8509999999999998e-06 2024-09-17T22:41:48.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750301+0000", 2024-09-17T22:41:48.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023974691999999999 2024-09-17T22:41:48.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750366+0000", 2024-09-17T22:41:48.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4863000000000006e-05 2024-09-17T22:41:48.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.248 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.248 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750429+0000", 2024-09-17T22:41:48.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2731000000000006e-05 2024-09-17T22:41:48.248 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.248 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.248 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.249 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72666 3.0 3:0299d250:::rbd_data.115fe9ac8535.0000000000000000:9 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.249 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726332+0000", 2024-09-17T22:41:48.249 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4580980209999996, 2024-09-17T22:41:48.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024140720000000001, 2024-09-17T22:41:48.249 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.249 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.249 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.249 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.249 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.249 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.249 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72666 2024-09-17T22:41:48.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.249 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726332+0000", 2024-09-17T22:41:48.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726332+0000", 2024-09-17T22:41:48.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726333+0000", 2024-09-17T22:41:48.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3300000000000003e-07 2024-09-17T22:41:48.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726335+0000", 2024-09-17T22:41:48.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.767e-06 2024-09-17T22:41:48.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726340+0000", 2024-09-17T22:41:48.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5489999999999997e-06 2024-09-17T22:41:48.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726345+0000", 2024-09-17T22:41:48.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.34e-06 2024-09-17T22:41:48.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750326+0000", 2024-09-17T22:41:48.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023981201000000001 2024-09-17T22:41:48.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750395+0000", 2024-09-17T22:41:48.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9615000000000003e-05 2024-09-17T22:41:48.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750473+0000", 2024-09-17T22:41:48.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7815e-05 2024-09-17T22:41:48.253 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.253 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.253 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.253 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72671 3.18 3:1997a52f:::rbd_data.115fe9ac8535.0000000000000006:9 [read 3485696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.253 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726372+0000", 2024-09-17T22:41:48.253 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4580588829999996, 2024-09-17T22:41:48.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024212465999999998, 2024-09-17T22:41:48.253 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.253 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.253 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.253 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.254 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.254 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.254 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72671 2024-09-17T22:41:48.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.254 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726372+0000", 2024-09-17T22:41:48.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726372+0000", 2024-09-17T22:41:48.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726372+0000", 2024-09-17T22:41:48.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.27e-07 2024-09-17T22:41:48.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726374+0000", 2024-09-17T22:41:48.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.708e-06 2024-09-17T22:41:48.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726380+0000", 2024-09-17T22:41:48.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0850000000000002e-06 2024-09-17T22:41:48.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726384+0000", 2024-09-17T22:41:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3640000000000001e-06 2024-09-17T22:41:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750452+0000", 2024-09-17T22:41:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024067708 2024-09-17T22:41:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750519+0000", 2024-09-17T22:41:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7248000000000004e-05 2024-09-17T22:41:48.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.257 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.257 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750584+0000", 2024-09-17T22:41:48.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5025999999999997e-05 2024-09-17T22:41:48.257 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.257 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.257 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.257 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72672 3.0 3:0299d250:::rbd_data.115fe9ac8535.0000000000000000:9 [read 3792896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.257 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726390+0000", 2024-09-17T22:41:48.257 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4580408229999993, 2024-09-17T22:41:48.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024256847000000002, 2024-09-17T22:41:48.257 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.257 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.257 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.258 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.258 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.258 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.258 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72672 2024-09-17T22:41:48.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.258 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726390+0000", 2024-09-17T22:41:48.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726390+0000", 2024-09-17T22:41:48.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726390+0000", 2024-09-17T22:41:48.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8500000000000002e-07 2024-09-17T22:41:48.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726392+0000", 2024-09-17T22:41:48.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9209999999999999e-06 2024-09-17T22:41:48.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726399+0000", 2024-09-17T22:41:48.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5910000000000001e-06 2024-09-17T22:41:48.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726403+0000", 2024-09-17T22:41:48.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.741e-06 2024-09-17T22:41:48.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750500+0000", 2024-09-17T22:41:48.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024096362999999999 2024-09-17T22:41:48.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750569+0000", 2024-09-17T22:41:48.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9243000000000001e-05 2024-09-17T22:41:48.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750646+0000", 2024-09-17T22:41:48.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7602999999999995e-05 2024-09-17T22:41:48.261 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.261 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.261 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.261 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72677 3.0 3:04644e28:::rbd_data.115fe9ac8535.0000000000000005:9 [read 3502080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.261 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726433+0000", 2024-09-17T22:41:48.261 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4579974270000005, 2024-09-17T22:41:48.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024382343000000001, 2024-09-17T22:41:48.261 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.262 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.262 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.262 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.262 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.262 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.262 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72677 2024-09-17T22:41:48.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.262 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726433+0000", 2024-09-17T22:41:48.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726433+0000", 2024-09-17T22:41:48.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726433+0000", 2024-09-17T22:41:48.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:41:48.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.263 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726435+0000", 2024-09-17T22:41:48.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.866e-06 2024-09-17T22:41:48.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726441+0000", 2024-09-17T22:41:48.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4840000000000003e-06 2024-09-17T22:41:48.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726451+0000", 2024-09-17T22:41:48.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0101999999999999e-05 2024-09-17T22:41:48.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750681+0000", 2024-09-17T22:41:48.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024230054000000001 2024-09-17T22:41:48.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750748+0000", 2024-09-17T22:41:48.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7269999999999998e-05 2024-09-17T22:41:48.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750815+0000", 2024-09-17T22:41:48.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7156000000000006e-05 2024-09-17T22:41:48.265 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.265 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.265 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.265 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72678 3.0 3:04644e28:::rbd_data.115fe9ac8535.0000000000000005:9 [read 3772416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.265 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726458+0000", 2024-09-17T22:41:48.265 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4579723090000005, 2024-09-17T22:41:48.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024566917000000001, 2024-09-17T22:41:48.266 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.266 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.266 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.266 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.266 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.266 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.266 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72678 2024-09-17T22:41:48.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.266 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726458+0000", 2024-09-17T22:41:48.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726458+0000", 2024-09-17T22:41:48.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726459+0000", 2024-09-17T22:41:48.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9100000000000004e-07 2024-09-17T22:41:48.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726461+0000", 2024-09-17T22:41:48.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0829999999999998e-06 2024-09-17T22:41:48.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726468+0000", 2024-09-17T22:41:48.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0169999999999999e-06 2024-09-17T22:41:48.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726472+0000", 2024-09-17T22:41:48.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6789999999999996e-06 2024-09-17T22:41:48.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750861+0000", 2024-09-17T22:41:48.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024388183000000001 2024-09-17T22:41:48.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.750942+0000", 2024-09-17T22:41:48.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0970999999999994e-05 2024-09-17T22:41:48.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.751025+0000", 2024-09-17T22:41:48.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3492999999999995e-05 2024-09-17T22:41:48.269 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.269 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.269 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.269 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:72679 3.0 3:0299d250:::rbd_data.115fe9ac8535.0000000000000000:9 [read 4001792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-17T22:41:48.270 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:38.726487+0000", 2024-09-17T22:41:48.270 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4579429909999995, 2024-09-17T22:41:48.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024710454999999999, 2024-09-17T22:41:48.270 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.270 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.270 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.270 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.270 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.270 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.270 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 72679 2024-09-17T22:41:48.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.270 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726487+0000", 2024-09-17T22:41:48.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726487+0000", 2024-09-17T22:41:48.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726488+0000", 2024-09-17T22:41:48.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2399999999999998e-07 2024-09-17T22:41:48.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726490+0000", 2024-09-17T22:41:48.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9819999999999998e-06 2024-09-17T22:41:48.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726496+0000", 2024-09-17T22:41:48.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8730000000000004e-06 2024-09-17T22:41:48.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.726501+0000", 2024-09-17T22:41:48.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7620000000000003e-06 2024-09-17T22:41:48.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.751063+0000", 2024-09-17T22:41:48.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024562143000000002 2024-09-17T22:41:48.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.751135+0000", 2024-09-17T22:41:48.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2385999999999994e-05 2024-09-17T22:41:48.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:38.751198+0000", 2024-09-17T22:41:48.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2785000000000003e-05 2024-09-17T22:41:48.273 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.273 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.273 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.273 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.273 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.274 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:48.275 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:41:48.276 INFO:teuthology.orchestra.run.smithi115.stdout:1078 write 0x6062a4b thru 0x6062dde (0x394 bytes) 2024-09-17T22:41:48.276 INFO:teuthology.orchestra.run.smithi115.stdout:1080 read 0xa63a31 thru 0xa6cc09 (0x91d9 bytes) 2024-09-17T22:41:48.276 INFO:teuthology.orchestra.run.smithi115.stdout:1081 read 0x37df78 thru 0x381b1f (0x3ba8 bytes) 2024-09-17T22:41:48.276 INFO:teuthology.orchestra.run.smithi115.stdout:1082 writesame 0xf5b448 thru 0xf67158 data_size 0x1(0xbd11 bytes) 2024-09-17T22:41:48.276 INFO:teuthology.orchestra.run.smithi115.stdout:1084 read 0x3598e07 thru 0x35a3866 (0xaa60 bytes) 2024-09-17T22:41:48.276 INFO:teuthology.orchestra.run.smithi115.stdout:1085 read 0x6ab0aaa thru 0x6ab4379 (0x38d0 bytes) 2024-09-17T22:41:48.276 INFO:teuthology.orchestra.run.smithi115.stdout:1086 write 0x3e1c601 thru 0x3e22186 (0x5b86 bytes) 2024-09-17T22:41:48.276 INFO:teuthology.orchestra.run.smithi115.stdout:1087 clone 14 order 18 su 32768 sc 7 2024-09-17T22:41:48.277 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:48.272+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:48.376 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:48.373+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:48.379 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:48.380 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:41:48.380 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:48.388 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:41:48.477 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:48.474+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:48.532 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:48.533 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:48.533 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:48.533 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:48.533 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:48.541 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:41:48.585 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:48.583+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:48.646 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:48.646 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:48.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:48.647 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:48.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.647 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38083 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.647 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.763873+0000", 2024-09-17T22:41:48.647 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.879534844999998, 2024-09-17T22:41:48.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023344747999999999, 2024-09-17T22:41:48.647 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.647 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.647 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.647 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.647 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.647 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.647 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38083 2024-09-17T22:41:48.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.648 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763873+0000", 2024-09-17T22:41:48.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763873+0000", 2024-09-17T22:41:48.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763874+0000", 2024-09-17T22:41:48.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0899999999999997e-07 2024-09-17T22:41:48.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763875+0000", 2024-09-17T22:41:48.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8530000000000001e-06 2024-09-17T22:41:48.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763881+0000", 2024-09-17T22:41:48.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4879999999999998e-06 2024-09-17T22:41:48.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763886+0000", 2024-09-17T22:41:48.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.587e-06 2024-09-17T22:41:48.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787057+0000", 2024-09-17T22:41:48.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023170611000000001 2024-09-17T22:41:48.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787127+0000", 2024-09-17T22:41:48.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0259000000000006e-05 2024-09-17T22:41:48.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787218+0000", 2024-09-17T22:41:48.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0641000000000002e-05 2024-09-17T22:41:48.651 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.651 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.651 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.651 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38105 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.651 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.763938+0000", 2024-09-17T22:41:48.651 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.879470038000001, 2024-09-17T22:41:48.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023471315999999999, 2024-09-17T22:41:48.651 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.651 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.651 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.651 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.652 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.652 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.652 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38105 2024-09-17T22:41:48.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.652 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763938+0000", 2024-09-17T22:41:48.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763938+0000", 2024-09-17T22:41:48.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763939+0000", 2024-09-17T22:41:48.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:41:48.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763941+0000", 2024-09-17T22:41:48.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9489999999999999e-06 2024-09-17T22:41:48.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763946+0000", 2024-09-17T22:41:48.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9139999999999996e-06 2024-09-17T22:41:48.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763951+0000", 2024-09-17T22:41:48.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6589999999999998e-06 2024-09-17T22:41:48.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787249+0000", 2024-09-17T22:41:48.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023297989000000002 2024-09-17T22:41:48.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787319+0000", 2024-09-17T22:41:48.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9836e-05 2024-09-17T22:41:48.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787409+0000", 2024-09-17T22:41:48.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0457000000000005e-05 2024-09-17T22:41:48.655 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.655 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.655 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.655 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38107 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.656 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:48.656 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.879451248999999, 2024-09-17T22:41:48.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023669550000000001, 2024-09-17T22:41:48.656 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.656 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.656 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.656 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.656 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.656 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.656 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38107 2024-09-17T22:41:48.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.656 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:48.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:48.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:48.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:41:48.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763959+0000", 2024-09-17T22:41:48.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.849e-06 2024-09-17T22:41:48.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763965+0000", 2024-09-17T22:41:48.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6110000000000001e-06 2024-09-17T22:41:48.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763969+0000", 2024-09-17T22:41:48.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-06 2024-09-17T22:41:48.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787451+0000", 2024-09-17T22:41:48.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023482281000000001 2024-09-17T22:41:48.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787516+0000", 2024-09-17T22:41:48.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4930000000000003e-05 2024-09-17T22:41:48.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787626+0000", 2024-09-17T22:41:48.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011000700000000001 2024-09-17T22:41:48.659 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.659 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.659 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.660 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38141 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.660 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764408+0000", 2024-09-17T22:41:48.660 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.879000016999999, 2024-09-17T22:41:48.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023422268, 2024-09-17T22:41:48.660 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.660 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.660 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.660 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.660 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.660 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.660 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38141 2024-09-17T22:41:48.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.660 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764408+0000", 2024-09-17T22:41:48.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764408+0000", 2024-09-17T22:41:48.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764409+0000", 2024-09-17T22:41:48.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:41:48.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764412+0000", 2024-09-17T22:41:48.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4470000000000002e-06 2024-09-17T22:41:48.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764421+0000", 2024-09-17T22:41:48.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7299999999999994e-06 2024-09-17T22:41:48.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764427+0000", 2024-09-17T22:41:48.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.392e-06 2024-09-17T22:41:48.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787669+0000", 2024-09-17T22:41:48.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023241385 2024-09-17T22:41:48.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787743+0000", 2024-09-17T22:41:48.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4260999999999999e-05 2024-09-17T22:41:48.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787830+0000", 2024-09-17T22:41:48.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7398999999999994e-05 2024-09-17T22:41:48.664 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.664 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.664 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.664 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38143 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.664 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:48.664 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.878970088999999, 2024-09-17T22:41:48.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023565691, 2024-09-17T22:41:48.664 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.664 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.664 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.664 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.665 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38143 2024-09-17T22:41:48.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.665 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:48.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:48.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:48.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6199999999999998e-07 2024-09-17T22:41:48.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764441+0000", 2024-09-17T22:41:48.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1179999999999999e-06 2024-09-17T22:41:48.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764447+0000", 2024-09-17T22:41:48.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1940000000000003e-06 2024-09-17T22:41:48.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764452+0000", 2024-09-17T22:41:48.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9150000000000004e-06 2024-09-17T22:41:48.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787881+0000", 2024-09-17T22:41:48.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023429141000000001 2024-09-17T22:41:48.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787947+0000", 2024-09-17T22:41:48.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5902999999999996e-05 2024-09-17T22:41:48.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788004+0000", 2024-09-17T22:41:48.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6957999999999999e-05 2024-09-17T22:41:48.668 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.668 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.668 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.668 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38150 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.668 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:48.668 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.878950087, 2024-09-17T22:41:48.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023741399, 2024-09-17T22:41:48.668 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.668 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.669 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.669 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.669 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.669 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.669 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38150 2024-09-17T22:41:48.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.669 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:48.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:48.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:48.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8299999999999998e-07 2024-09-17T22:41:48.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764460+0000", 2024-09-17T22:41:48.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.855e-06 2024-09-17T22:41:48.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764466+0000", 2024-09-17T22:41:48.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8200000000000002e-06 2024-09-17T22:41:48.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764471+0000", 2024-09-17T22:41:48.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8319999999999996e-06 2024-09-17T22:41:48.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788032+0000", 2024-09-17T22:41:48.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023561480999999999 2024-09-17T22:41:48.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788106+0000", 2024-09-17T22:41:48.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3406999999999997e-05 2024-09-17T22:41:48.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788199+0000", 2024-09-17T22:41:48.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3621000000000004e-05 2024-09-17T22:41:48.672 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.672 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.672 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.673 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38152 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 83968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.673 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:48.673 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.87893137, 2024-09-17T22:41:48.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023859726000000001, 2024-09-17T22:41:48.673 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.673 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.673 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.673 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.673 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.673 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.673 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38152 2024-09-17T22:41:48.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.674 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:48.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:48.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:48.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3000000000000001e-07 2024-09-17T22:41:48.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764479+0000", 2024-09-17T22:41:48.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8920000000000001e-06 2024-09-17T22:41:48.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764485+0000", 2024-09-17T22:41:48.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:41:48.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764490+0000", 2024-09-17T22:41:48.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6890000000000003e-06 2024-09-17T22:41:48.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788232+0000", 2024-09-17T22:41:48.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023742452000000001 2024-09-17T22:41:48.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788276+0000", 2024-09-17T22:41:48.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3689000000000001e-05 2024-09-17T22:41:48.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788336+0000", 2024-09-17T22:41:48.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0581999999999997e-05 2024-09-17T22:41:48.677 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.677 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.677 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.677 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38184 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 161792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.677 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764589+0000", 2024-09-17T22:41:48.677 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.878818977000002, 2024-09-17T22:41:48.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023956801, 2024-09-17T22:41:48.677 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.677 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.677 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.678 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.678 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38184 2024-09-17T22:41:48.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.678 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764589+0000", 2024-09-17T22:41:48.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764589+0000", 2024-09-17T22:41:48.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764590+0000", 2024-09-17T22:41:48.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7800000000000001e-07 2024-09-17T22:41:48.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764592+0000", 2024-09-17T22:41:48.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6929999999999999e-06 2024-09-17T22:41:48.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764600+0000", 2024-09-17T22:41:48.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1140000000000006e-06 2024-09-17T22:41:48.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764606+0000", 2024-09-17T22:41:48.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6200000000000004e-06 2024-09-17T22:41:48.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788374+0000", 2024-09-17T22:41:48.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023768083999999998 2024-09-17T22:41:48.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788447+0000", 2024-09-17T22:41:48.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2922000000000004e-05 2024-09-17T22:41:48.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788546+0000", 2024-09-17T22:41:48.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8789999999999994e-05 2024-09-17T22:41:48.681 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.681 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.681 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.681 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38186 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.681 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:48.681 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.878794027000001, 2024-09-17T22:41:48.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024156486000000001, 2024-09-17T22:41:48.681 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.681 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.682 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.682 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.682 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.682 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.682 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38186 2024-09-17T22:41:48.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.682 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:48.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:48.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:48.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:41:48.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764621+0000", 2024-09-17T22:41:48.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6409999999999996e-06 2024-09-17T22:41:48.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764628+0000", 2024-09-17T22:41:48.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0360000000000001e-06 2024-09-17T22:41:48.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764633+0000", 2024-09-17T22:41:48.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0300000000000001e-06 2024-09-17T22:41:48.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788594+0000", 2024-09-17T22:41:48.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023960921999999999 2024-09-17T22:41:48.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788676+0000", 2024-09-17T22:41:48.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2123e-05 2024-09-17T22:41:48.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788771+0000", 2024-09-17T22:41:48.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4284999999999996e-05 2024-09-17T22:41:48.685 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.685 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.685 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.685 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38218 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 350208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.685 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764706+0000", 2024-09-17T22:41:48.686 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.878701774, 2024-09-17T22:41:48.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024324732000000002, 2024-09-17T22:41:48.686 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.686 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.686 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.686 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.686 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.686 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.686 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38218 2024-09-17T22:41:48.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.686 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764706+0000", 2024-09-17T22:41:48.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764706+0000", 2024-09-17T22:41:48.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764707+0000", 2024-09-17T22:41:48.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7199999999999999e-07 2024-09-17T22:41:48.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764710+0000", 2024-09-17T22:41:48.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6819999999999999e-06 2024-09-17T22:41:48.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764716+0000", 2024-09-17T22:41:48.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7900000000000002e-06 2024-09-17T22:41:48.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764722+0000", 2024-09-17T22:41:48.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2560000000000002e-06 2024-09-17T22:41:48.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788867+0000", 2024-09-17T22:41:48.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024144982999999998 2024-09-17T22:41:48.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788941+0000", 2024-09-17T22:41:48.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4529000000000004e-05 2024-09-17T22:41:48.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789031+0000", 2024-09-17T22:41:48.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9920000000000006e-05 2024-09-17T22:41:48.689 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.689 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.689 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.689 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38222 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.690 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764765+0000", 2024-09-17T22:41:48.690 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.878642920000001, 2024-09-17T22:41:48.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024443910999999999, 2024-09-17T22:41:48.690 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.690 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.690 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.690 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38222 2024-09-17T22:41:48.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.690 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764765+0000", 2024-09-17T22:41:48.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764765+0000", 2024-09-17T22:41:48.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764766+0000", 2024-09-17T22:41:48.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8999999999999996e-07 2024-09-17T22:41:48.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764769+0000", 2024-09-17T22:41:48.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7659999999999999e-06 2024-09-17T22:41:48.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764776+0000", 2024-09-17T22:41:48.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4229999999999999e-06 2024-09-17T22:41:48.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764783+0000", 2024-09-17T22:41:48.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9600000000000003e-06 2024-09-17T22:41:48.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789060+0000", 2024-09-17T22:41:48.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024277343 2024-09-17T22:41:48.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789129+0000", 2024-09-17T22:41:48.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8367999999999993e-05 2024-09-17T22:41:48.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789209+0000", 2024-09-17T22:41:48.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0461000000000001e-05 2024-09-17T22:41:48.693 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.694 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.694 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.694 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38671 3.12 3:4a5b92ac:::rbd_data.115fe0c30ef6.0000000000000011:5 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.694 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778865+0000", 2024-09-17T22:41:48.694 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.86454286, 2024-09-17T22:41:48.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023227351, 2024-09-17T22:41:48.694 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.694 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.694 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.694 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38671 2024-09-17T22:41:48.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.695 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778865+0000", 2024-09-17T22:41:48.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778865+0000", 2024-09-17T22:41:48.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778866+0000", 2024-09-17T22:41:48.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0900000000000002e-07 2024-09-17T22:41:48.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778868+0000", 2024-09-17T22:41:48.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4339999999999999e-06 2024-09-17T22:41:48.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778875+0000", 2024-09-17T22:41:48.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4559999999999997e-06 2024-09-17T22:41:48.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778880+0000", 2024-09-17T22:41:48.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2689999999999999e-06 2024-09-17T22:41:48.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.801978+0000", 2024-09-17T22:41:48.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023098225999999999 2024-09-17T22:41:48.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802035+0000", 2024-09-17T22:41:48.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7123999999999997e-05 2024-09-17T22:41:48.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802093+0000", 2024-09-17T22:41:48.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7432999999999997e-05 2024-09-17T22:41:48.698 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.698 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.698 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.698 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38673 3.12 3:4a5b92ac:::rbd_data.115fe0c30ef6.0000000000000011:5 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.698 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778886+0000", 2024-09-17T22:41:48.698 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.864521711999998, 2024-09-17T22:41:48.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023389259999999999, 2024-09-17T22:41:48.698 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.698 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.699 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.699 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38673 2024-09-17T22:41:48.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.699 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778886+0000", 2024-09-17T22:41:48.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778886+0000", 2024-09-17T22:41:48.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778887+0000", 2024-09-17T22:41:48.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:41:48.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778889+0000", 2024-09-17T22:41:48.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9379999999999999e-06 2024-09-17T22:41:48.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778895+0000", 2024-09-17T22:41:48.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0360000000000003e-06 2024-09-17T22:41:48.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778900+0000", 2024-09-17T22:41:48.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1220000000000003e-06 2024-09-17T22:41:48.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802126+0000", 2024-09-17T22:41:48.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023226607 2024-09-17T22:41:48.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802208+0000", 2024-09-17T22:41:48.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1157999999999998e-05 2024-09-17T22:41:48.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802276+0000", 2024-09-17T22:41:48.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8122999999999998e-05 2024-09-17T22:41:48.703 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.703 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.703 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.703 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38675 3.12 3:4a5b92ac:::rbd_data.115fe0c30ef6.0000000000000011:5 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.703 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:48.703 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.864486506999999, 2024-09-17T22:41:48.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023484221999999999, 2024-09-17T22:41:48.704 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.704 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.704 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.704 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.704 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.704 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.704 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38675 2024-09-17T22:41:48.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.704 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:48.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:48.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:48.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9999999999999997e-07 2024-09-17T22:41:48.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778924+0000", 2024-09-17T22:41:48.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3039999999999999e-06 2024-09-17T22:41:48.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778931+0000", 2024-09-17T22:41:48.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9519999999999997e-06 2024-09-17T22:41:48.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778937+0000", 2024-09-17T22:41:48.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3639999999999998e-06 2024-09-17T22:41:48.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802300+0000", 2024-09-17T22:41:48.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023363544 2024-09-17T22:41:48.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802347+0000", 2024-09-17T22:41:48.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6897999999999997e-05 2024-09-17T22:41:48.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802406+0000", 2024-09-17T22:41:48.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8560000000000002e-05 2024-09-17T22:41:48.707 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.707 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.707 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.708 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38679 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.708 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778945+0000", 2024-09-17T22:41:48.708 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.864462890999999, 2024-09-17T22:41:48.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023578015000000001, 2024-09-17T22:41:48.708 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.708 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.708 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.708 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38679 2024-09-17T22:41:48.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.708 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778945+0000", 2024-09-17T22:41:48.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778945+0000", 2024-09-17T22:41:48.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778946+0000", 2024-09-17T22:41:48.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.44e-07 2024-09-17T22:41:48.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778948+0000", 2024-09-17T22:41:48.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-17T22:41:48.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778955+0000", 2024-09-17T22:41:48.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.528e-06 2024-09-17T22:41:48.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778960+0000", 2024-09-17T22:41:48.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1919999999999996e-06 2024-09-17T22:41:48.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802429+0000", 2024-09-17T22:41:48.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023469561999999999 2024-09-17T22:41:48.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802470+0000", 2024-09-17T22:41:48.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1068000000000001e-05 2024-09-17T22:41:48.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802523+0000", 2024-09-17T22:41:48.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2852999999999997e-05 2024-09-17T22:41:48.711 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.712 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.712 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.712 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38680 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.712 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778966+0000", 2024-09-17T22:41:48.712 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.864441932999998, 2024-09-17T22:41:48.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023670668999999998, 2024-09-17T22:41:48.712 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.712 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.712 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.712 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.712 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.713 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.713 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38680 2024-09-17T22:41:48.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.713 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778966+0000", 2024-09-17T22:41:48.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778966+0000", 2024-09-17T22:41:48.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778967+0000", 2024-09-17T22:41:48.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.89e-07 2024-09-17T22:41:48.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778969+0000", 2024-09-17T22:41:48.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0219999999999999e-06 2024-09-17T22:41:48.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778975+0000", 2024-09-17T22:41:48.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3779999999999998e-06 2024-09-17T22:41:48.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778980+0000", 2024-09-17T22:41:48.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0699999999999997e-06 2024-09-17T22:41:48.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802545+0000", 2024-09-17T22:41:48.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023565358000000002 2024-09-17T22:41:48.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802585+0000", 2024-09-17T22:41:48.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0136999999999999e-05 2024-09-17T22:41:48.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802637+0000", 2024-09-17T22:41:48.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1314999999999999e-05 2024-09-17T22:41:48.716 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.716 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.716 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.716 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38697 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 675840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.716 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.779074+0000", 2024-09-17T22:41:48.716 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.864333680000001, 2024-09-17T22:41:48.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023676402999999999, 2024-09-17T22:41:48.716 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.717 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.717 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.717 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.717 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.717 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.717 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38697 2024-09-17T22:41:48.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.717 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779074+0000", 2024-09-17T22:41:48.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779074+0000", 2024-09-17T22:41:48.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779075+0000", 2024-09-17T22:41:48.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:41:48.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779077+0000", 2024-09-17T22:41:48.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8810000000000001e-06 2024-09-17T22:41:48.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779084+0000", 2024-09-17T22:41:48.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.691e-06 2024-09-17T22:41:48.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779088+0000", 2024-09-17T22:41:48.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6970000000000001e-06 2024-09-17T22:41:48.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802659+0000", 2024-09-17T22:41:48.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023571214 2024-09-17T22:41:48.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802699+0000", 2024-09-17T22:41:48.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9762000000000003e-05 2024-09-17T22:41:48.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802751+0000", 2024-09-17T22:41:48.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1572e-05 2024-09-17T22:41:48.720 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.720 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.720 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.721 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38698 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 749568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.721 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.779094+0000", 2024-09-17T22:41:48.721 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.864313743, 2024-09-17T22:41:48.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023778845999999999, 2024-09-17T22:41:48.721 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.721 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.721 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.721 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.721 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.721 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.721 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38698 2024-09-17T22:41:48.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.721 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779094+0000", 2024-09-17T22:41:48.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779094+0000", 2024-09-17T22:41:48.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779095+0000", 2024-09-17T22:41:48.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8799999999999998e-07 2024-09-17T22:41:48.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779097+0000", 2024-09-17T22:41:48.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8530000000000001e-06 2024-09-17T22:41:48.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779103+0000", 2024-09-17T22:41:48.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0410000000000003e-06 2024-09-17T22:41:48.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779107+0000", 2024-09-17T22:41:48.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T22:41:48.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802774+0000", 2024-09-17T22:41:48.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023666348 2024-09-17T22:41:48.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802814+0000", 2024-09-17T22:41:48.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0173000000000003e-05 2024-09-17T22:41:48.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802873+0000", 2024-09-17T22:41:48.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9484999999999997e-05 2024-09-17T22:41:48.725 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.725 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.725 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.725 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38825 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.725 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.781521+0000", 2024-09-17T22:41:48.725 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.861886676000001, 2024-09-17T22:41:48.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023214104999999999, 2024-09-17T22:41:48.725 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.725 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.726 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:48.726 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.726 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.726 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.726 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38825 2024-09-17T22:41:48.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.726 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781521+0000", 2024-09-17T22:41:48.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781521+0000", 2024-09-17T22:41:48.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781522+0000", 2024-09-17T22:41:48.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.53e-07 2024-09-17T22:41:48.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781524+0000", 2024-09-17T22:41:48.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0389999999999999e-06 2024-09-17T22:41:48.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781530+0000", 2024-09-17T22:41:48.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8209999999999998e-06 2024-09-17T22:41:48.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781534+0000", 2024-09-17T22:41:48.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6419999999999998e-06 2024-09-17T22:41:48.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804570+0000", 2024-09-17T22:41:48.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023035951999999998 2024-09-17T22:41:48.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:48.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804645+0000", 2024-09-17T22:41:48.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4696000000000004e-05 2024-09-17T22:41:48.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804736+0000", 2024-09-17T22:41:48.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0601999999999998e-05 2024-09-17T22:41:48.729 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.729 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.729 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.729 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38843 3.12 3.c4845672 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:48.729 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:48.730 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.861764466, 2024-09-17T22:41:48.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023236265999999998, 2024-09-17T22:41:48.730 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:48.730 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:48.730 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:48.730 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:48.730 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:48.730 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:48.730 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38843 2024-09-17T22:41:48.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.730 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:48.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:48.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:48.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:48.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:48.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:48.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:48.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:48.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:41:48.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:48.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781646+0000", 2024-09-17T22:41:48.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9290000000000001e-06 2024-09-17T22:41:48.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:48.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781652+0000", 2024-09-17T22:41:48.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7389999999999996e-06 2024-09-17T22:41:48.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:48.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781656+0000", 2024-09-17T22:41:48.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3830000000000002e-06 2024-09-17T22:41:48.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:48.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804871+0000", 2024-09-17T22:41:48.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023215257999999999 2024-09-17T22:41:48.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:48.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:48.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:48.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804880+0000", 2024-09-17T22:41:48.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5769999999999994e-06 2024-09-17T22:41:48.733 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.733 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.733 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.733 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:48.733 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:48.733 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:48.737 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:41:48.738 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:48.735+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:48.838 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:48.835+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:48.881 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:48.881 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:48.881 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:41:48.881 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:48.890 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:41:48.938 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:48.936+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:49.032 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:49.033 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:49.033 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:49.033 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:49.033 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:49.039 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:49.037+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:49.041 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:41:49.096 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is unset 2024-09-17T22:41:49.140 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:49.138+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:49.151 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:49.151 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:49.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36437 3.b 3:d64eaf12:::rbd_data.115f86d8437d.000000000000001c:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:41:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.433115934, 2024-09-17T22:41:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068296390000000002, 2024-09-17T22:41:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.152 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36437 2024-09-17T22:41:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:41:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:41:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715035+0000", 2024-09-17T22:41:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.288e-06 2024-09-17T22:41:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715041+0000", 2024-09-17T22:41:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.643e-06 2024-09-17T22:41:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715051+0000", 2024-09-17T22:41:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0597e-05 2024-09-17T22:41:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715056+0000", 2024-09-17T22:41:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1379999999999998e-06 2024-09-17T22:41:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715101+0000", 2024-09-17T22:41:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4433e-05 2024-09-17T22:41:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715187+0000", 2024-09-17T22:41:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6148000000000003e-05 2024-09-17T22:41:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.716264+0000", 2024-09-17T22:41:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010770179999999999 2024-09-17T22:41:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721735+0000", 2024-09-17T22:41:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0054709019999999997 2024-09-17T22:41:49.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:41:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721754+0000", 2024-09-17T22:41:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9133000000000001e-05 2024-09-17T22:41:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721807+0000", 2024-09-17T22:41:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2831999999999998e-05 2024-09-17T22:41:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721863+0000", 2024-09-17T22:41:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6507e-05 2024-09-17T22:41:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36743 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 188416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735018+0000", 2024-09-17T22:41:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.413131819, 2024-09-17T22:41:49.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0060955260000000004, 2024-09-17T22:41:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36743 2024-09-17T22:41:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735018+0000", 2024-09-17T22:41:49.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735018+0000", 2024-09-17T22:41:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735018+0000", 2024-09-17T22:41:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7199999999999999e-07 2024-09-17T22:41:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735020+0000", 2024-09-17T22:41:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9889999999999999e-06 2024-09-17T22:41:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735027+0000", 2024-09-17T22:41:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3409999999999999e-06 2024-09-17T22:41:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735032+0000", 2024-09-17T22:41:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.904e-06 2024-09-17T22:41:49.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.740910+0000", 2024-09-17T22:41:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.005878266 2024-09-17T22:41:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741009+0000", 2024-09-17T22:41:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8754999999999999e-05 2024-09-17T22:41:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741113+0000", 2024-09-17T22:41:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000104799 2024-09-17T22:41:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36744 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:41:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.413111791999999, 2024-09-17T22:41:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063082559999999999, 2024-09-17T22:41:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.163 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36744 2024-09-17T22:41:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:41:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:41:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:41:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:41:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735040+0000", 2024-09-17T22:41:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9630000000000001e-06 2024-09-17T22:41:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735047+0000", 2024-09-17T22:41:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3439999999999997e-06 2024-09-17T22:41:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735052+0000", 2024-09-17T22:41:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9250000000000003e-06 2024-09-17T22:41:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741147+0000", 2024-09-17T22:41:49.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0060954210000000002 2024-09-17T22:41:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741243+0000", 2024-09-17T22:41:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5674999999999995e-05 2024-09-17T22:41:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741346+0000", 2024-09-17T22:41:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103522 2024-09-17T22:41:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36745 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:41:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.413091808000001, 2024-09-17T22:41:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065186740000000003, 2024-09-17T22:41:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36745 2024-09-17T22:41:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.168 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:41:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:41:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:41:49.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2500000000000001e-07 2024-09-17T22:41:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735061+0000", 2024-09-17T22:41:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.824e-06 2024-09-17T22:41:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735068+0000", 2024-09-17T22:41:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8580000000000002e-06 2024-09-17T22:41:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735074+0000", 2024-09-17T22:41:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5829999999999997e-06 2024-09-17T22:41:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741379+0000", 2024-09-17T22:41:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063051909999999999 2024-09-17T22:41:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741472+0000", 2024-09-17T22:41:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3238000000000004e-05 2024-09-17T22:41:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741577+0000", 2024-09-17T22:41:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000104555 2024-09-17T22:41:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36746 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:41:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.413068842000001, 2024-09-17T22:41:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067275240000000004, 2024-09-17T22:41:49.172 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36746 2024-09-17T22:41:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:41:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:41:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:41:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.03e-07 2024-09-17T22:41:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735083+0000", 2024-09-17T22:41:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T22:41:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735090+0000", 2024-09-17T22:41:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7800000000000003e-06 2024-09-17T22:41:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735095+0000", 2024-09-17T22:41:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8500000000000002e-06 2024-09-17T22:41:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741608+0000", 2024-09-17T22:41:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065131809999999998 2024-09-17T22:41:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741703+0000", 2024-09-17T22:41:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4914999999999995e-05 2024-09-17T22:41:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741808+0000", 2024-09-17T22:41:49.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000105448 2024-09-17T22:41:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36747 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:41:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.413048288999999, 2024-09-17T22:41:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069468790000000004, 2024-09-17T22:41:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:49.177 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36747 2024-09-17T22:41:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:41:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:41:49.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735102+0000", 2024-09-17T22:41:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:41:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735104+0000", 2024-09-17T22:41:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7400000000000001e-06 2024-09-17T22:41:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735111+0000", 2024-09-17T22:41:49.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4189999999999996e-06 2024-09-17T22:41:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735116+0000", 2024-09-17T22:41:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0860000000000001e-06 2024-09-17T22:41:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741847+0000", 2024-09-17T22:41:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067306859999999996 2024-09-17T22:41:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741942+0000", 2024-09-17T22:41:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5180999999999996e-05 2024-09-17T22:41:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742048+0000", 2024-09-17T22:41:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106387 2024-09-17T22:41:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.181 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36748 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:41:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.413026968, 2024-09-17T22:41:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071864119999999997, 2024-09-17T22:41:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36748 2024-09-17T22:41:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:41:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:41:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:41:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.34e-07 2024-09-17T22:41:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735125+0000", 2024-09-17T22:41:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0020000000000001e-06 2024-09-17T22:41:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735132+0000", 2024-09-17T22:41:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3489999999999997e-06 2024-09-17T22:41:49.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735138+0000", 2024-09-17T22:41:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9709999999999998e-06 2024-09-17T22:41:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742087+0000", 2024-09-17T22:41:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069480669999999996 2024-09-17T22:41:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742188+0000", 2024-09-17T22:41:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101279 2024-09-17T22:41:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742309+0000", 2024-09-17T22:41:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012131 2024-09-17T22:41:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36749 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:41:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.413004280999999, 2024-09-17T22:41:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0074112570000000001, 2024-09-17T22:41:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36749 2024-09-17T22:41:49.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:41:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:41:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735146+0000", 2024-09-17T22:41:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9200000000000002e-07 2024-09-17T22:41:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735158+0000", 2024-09-17T22:41:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2508e-05 2024-09-17T22:41:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735166+0000", 2024-09-17T22:41:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6290000000000001e-06 2024-09-17T22:41:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735174+0000", 2024-09-17T22:41:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1000000000000004e-06 2024-09-17T22:41:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742354+0000", 2024-09-17T22:41:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071795660000000001 2024-09-17T22:41:49.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742454+0000", 2024-09-17T22:41:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100408 2024-09-17T22:41:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742557+0000", 2024-09-17T22:41:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102654 2024-09-17T22:41:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.191 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36750 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:41:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.412967155, 2024-09-17T22:41:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0077413890000000004, 2024-09-17T22:41:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36750 2024-09-17T22:41:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:41:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:41:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:41:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2399999999999998e-07 2024-09-17T22:41:49.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735185+0000", 2024-09-17T22:41:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1710000000000001e-06 2024-09-17T22:41:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735192+0000", 2024-09-17T22:41:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.81e-06 2024-09-17T22:41:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735197+0000", 2024-09-17T22:41:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9080000000000003e-06 2024-09-17T22:41:49.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742587+0000", 2024-09-17T22:41:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.007390359 2024-09-17T22:41:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742694+0000", 2024-09-17T22:41:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106901 2024-09-17T22:41:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742924+0000", 2024-09-17T22:41:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022961600000000001 2024-09-17T22:41:49.195 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36751 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:41:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.412945181000001, 2024-09-17T22:41:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079273859999999998, 2024-09-17T22:41:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:49.196 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36751 2024-09-17T22:41:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:41:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:41:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:41:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1600000000000001e-07 2024-09-17T22:41:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735207+0000", 2024-09-17T22:41:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1909999999999999e-06 2024-09-17T22:41:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735214+0000", 2024-09-17T22:41:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9269999999999999e-06 2024-09-17T22:41:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735220+0000", 2024-09-17T22:41:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3739999999999997e-06 2024-09-17T22:41:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742957+0000", 2024-09-17T22:41:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0077371699999999998 2024-09-17T22:41:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.743037+0000", 2024-09-17T22:41:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0370000000000005e-05 2024-09-17T22:41:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.743132+0000", 2024-09-17T22:41:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4838000000000002e-05 2024-09-17T22:41:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37658 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:49.200 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749751+0000", 2024-09-17T22:41:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.398399100999999, 2024-09-17T22:41:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061048459999999997, 2024-09-17T22:41:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37658 2024-09-17T22:41:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749751+0000", 2024-09-17T22:41:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749751+0000", 2024-09-17T22:41:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749751+0000", 2024-09-17T22:41:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8500000000000002e-07 2024-09-17T22:41:49.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749753+0000", 2024-09-17T22:41:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2579999999999998e-06 2024-09-17T22:41:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749760+0000", 2024-09-17T22:41:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9099999999999999e-06 2024-09-17T22:41:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749765+0000", 2024-09-17T22:41:49.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1170000000000003e-06 2024-09-17T22:41:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755677+0000", 2024-09-17T22:41:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059120140000000002 2024-09-17T22:41:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755756+0000", 2024-09-17T22:41:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9013999999999999e-05 2024-09-17T22:41:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755855+0000", 2024-09-17T22:41:49.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9048000000000004e-05 2024-09-17T22:41:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37659 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:41:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.398377537999998, 2024-09-17T22:41:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063144200000000003, 2024-09-17T22:41:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.205 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37659 2024-09-17T22:41:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:41:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:41:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749773+0000", 2024-09-17T22:41:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:41:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749774+0000", 2024-09-17T22:41:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:41:49.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749781+0000", 2024-09-17T22:41:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5740000000000001e-06 2024-09-17T22:41:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749786+0000", 2024-09-17T22:41:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9099999999999996e-06 2024-09-17T22:41:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755889+0000", 2024-09-17T22:41:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061034260000000003 2024-09-17T22:41:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755972+0000", 2024-09-17T22:41:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2795999999999998e-05 2024-09-17T22:41:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756087+0000", 2024-09-17T22:41:49.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000114418 2024-09-17T22:41:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37660 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:41:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.398347699999999, 2024-09-17T22:41:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065587800000000002, 2024-09-17T22:41:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:49.210 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37660 2024-09-17T22:41:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:41:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:41:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:41:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:41:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749805+0000", 2024-09-17T22:41:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-17T22:41:49.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749812+0000", 2024-09-17T22:41:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270000000000002e-06 2024-09-17T22:41:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749819+0000", 2024-09-17T22:41:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.759e-06 2024-09-17T22:41:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756153+0000", 2024-09-17T22:41:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063345809999999997 2024-09-17T22:41:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756257+0000", 2024-09-17T22:41:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103583 2024-09-17T22:41:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756361+0000", 2024-09-17T22:41:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103853 2024-09-17T22:41:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.214 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37661 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:41:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.398319105999999, 2024-09-17T22:41:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067442300000000004, 2024-09-17T22:41:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37661 2024-09-17T22:41:49.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:41:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:41:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:41:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9399999999999995e-07 2024-09-17T22:41:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749835+0000", 2024-09-17T22:41:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1049999999999997e-06 2024-09-17T22:41:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749855+0000", 2024-09-17T22:41:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9343000000000001e-05 2024-09-17T22:41:49.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749860+0000", 2024-09-17T22:41:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:41:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756395+0000", 2024-09-17T22:41:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065349900000000001 2024-09-17T22:41:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756479+0000", 2024-09-17T22:41:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4030999999999996e-05 2024-09-17T22:41:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756575+0000", 2024-09-17T22:41:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5835999999999995e-05 2024-09-17T22:41:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37662 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:41:49.219 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.398265299999998, 2024-09-17T22:41:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006927295, 2024-09-17T22:41:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37662 2024-09-17T22:41:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:41:49.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:41:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749885+0000", 2024-09-17T22:41:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2699999999999999e-07 2024-09-17T22:41:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749888+0000", 2024-09-17T22:41:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6599999999999999e-06 2024-09-17T22:41:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749895+0000", 2024-09-17T22:41:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4200000000000001e-06 2024-09-17T22:41:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749901+0000", 2024-09-17T22:41:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5090000000000001e-06 2024-09-17T22:41:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756605+0000", 2024-09-17T22:41:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067040629999999997 2024-09-17T22:41:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756691+0000", 2024-09-17T22:41:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5846000000000004e-05 2024-09-17T22:41:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756812+0000", 2024-09-17T22:41:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012116999999999999 2024-09-17T22:41:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37663 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:41:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.398241089999999, 2024-09-17T22:41:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071811440000000004, 2024-09-17T22:41:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37663 2024-09-17T22:41:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:41:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:41:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:41:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:41:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749911+0000", 2024-09-17T22:41:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.035e-06 2024-09-17T22:41:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749917+0000", 2024-09-17T22:41:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4370000000000004e-06 2024-09-17T22:41:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749922+0000", 2024-09-17T22:41:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6249999999999998e-06 2024-09-17T22:41:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756845+0000", 2024-09-17T22:41:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069227280000000004 2024-09-17T22:41:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756994+0000", 2024-09-17T22:41:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000149068 2024-09-17T22:41:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757090+0000", 2024-09-17T22:41:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5851000000000001e-05 2024-09-17T22:41:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37664 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:41:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.398220245000001, 2024-09-17T22:41:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.007384743, 2024-09-17T22:41:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.228 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37664 2024-09-17T22:41:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:41:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:41:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749930+0000", 2024-09-17T22:41:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:41:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749933+0000", 2024-09-17T22:41:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8430000000000001e-06 2024-09-17T22:41:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749939+0000", 2024-09-17T22:41:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:41:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749944+0000", 2024-09-17T22:41:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9230000000000001e-06 2024-09-17T22:41:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757124+0000", 2024-09-17T22:41:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071800020000000004 2024-09-17T22:41:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757212+0000", 2024-09-17T22:41:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7383e-05 2024-09-17T22:41:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757314+0000", 2024-09-17T22:41:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010247399999999999 2024-09-17T22:41:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.232 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:39363 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:41:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.356231906000001, 2024-09-17T22:41:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006226245, 2024-09-17T22:41:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 39363 2024-09-17T22:41:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:41:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:41:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:41:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3500000000000002e-07 2024-09-17T22:41:49.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791920+0000", 2024-09-17T22:41:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.869e-06 2024-09-17T22:41:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791926+0000", 2024-09-17T22:41:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8490000000000002e-06 2024-09-17T22:41:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791931+0000", 2024-09-17T22:41:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4269999999999999e-06 2024-09-17T22:41:49.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.797912+0000", 2024-09-17T22:41:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059808819999999999 2024-09-17T22:41:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.798020+0000", 2024-09-17T22:41:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010783800000000001 2024-09-17T22:41:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.798144+0000", 2024-09-17T22:41:49.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012404500000000001 2024-09-17T22:41:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4447.0:51869 3.6 e35/23 3:6037e35f:::rbd_object_map.115fe9ac8535:head v 35'71, mlcod=35'66)", 2024-09-17T22:41:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:41:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.370909606, 2024-09-17T22:41:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0066208669999999999, 2024-09-17T22:41:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:41:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:41:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777241+0000", 2024-09-17T22:41:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4000000000000001e-07 2024-09-17T22:41:49.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777246+0000", 2024-09-17T22:41:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2789999999999998e-06 2024-09-17T22:41:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777255+0000", 2024-09-17T22:41:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6209999999999993e-06 2024-09-17T22:41:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777259+0000", 2024-09-17T22:41:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7650000000000001e-06 2024-09-17T22:41:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777298+0000", 2024-09-17T22:41:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8797999999999997e-05 2024-09-17T22:41:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777311+0000", 2024-09-17T22:41:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2364999999999999e-05 2024-09-17T22:41:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.783814+0000", 2024-09-17T22:41:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065039169999999997 2024-09-17T22:41:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.783861+0000", 2024-09-17T22:41:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6482e-05 2024-09-17T22:41:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:105336 3.b 3:d3549b8d:::rbd_data.115ff9e8d63c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:41:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:43.787529+0000", 2024-09-17T22:41:49.241 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.360620301, 2024-09-17T22:41:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0099864259999999996, 2024-09-17T22:41:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 105336 2024-09-17T22:41:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.787529+0000", 2024-09-17T22:41:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.787529+0000", 2024-09-17T22:41:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.787531+0000", 2024-09-17T22:41:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9759999999999998e-06 2024-09-17T22:41:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788120+0000", 2024-09-17T22:41:49.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00058899799999999997 2024-09-17T22:41:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788138+0000", 2024-09-17T22:41:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7393000000000001e-05 2024-09-17T22:41:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788146+0000", 2024-09-17T22:41:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8220000000000007e-06 2024-09-17T22:41:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788199+0000", 2024-09-17T22:41:49.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3718000000000003e-05 2024-09-17T22:41:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788316+0000", 2024-09-17T22:41:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000116999 2024-09-17T22:41:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.791608+0000", 2024-09-17T22:41:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003291188 2024-09-17T22:41:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797379+0000", 2024-09-17T22:41:49.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057716829999999997 2024-09-17T22:41:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:41:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797401+0000", 2024-09-17T22:41:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1729000000000002e-05 2024-09-17T22:41:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797459+0000", 2024-09-17T22:41:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8115000000000002e-05 2024-09-17T22:41:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797516+0000", 2024-09-17T22:41:49.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6805000000000002e-05 2024-09-17T22:41:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:49.247 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:49.247 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:49.248 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone13 from 0xdd4ba93 (overlap 0x2d8f61) to 0x0 2024-09-17T22:41:49.248 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-17T22:41:49.248 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:49.245+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:49.249 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:41:49.349 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:49.346+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:49.429 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:41:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145714 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.357042+0000", 2024-09-17T22:41:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.049657367000000001, 2024-09-17T22:41:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049676598000000002, 2024-09-17T22:41:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145714 2024-09-17T22:41:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357042+0000", 2024-09-17T22:41:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357042+0000", 2024-09-17T22:41:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357329+0000", 2024-09-17T22:41:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000286774 2024-09-17T22:41:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357334+0000", 2024-09-17T22:41:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5310000000000001e-06 2024-09-17T22:41:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357344+0000", 2024-09-17T22:41:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6730000000000004e-06 2024-09-17T22:41:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357351+0000", 2024-09-17T22:41:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1999999999999997e-06 2024-09-17T22:41:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145778 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.364203+0000", 2024-09-17T22:41:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.042496645999999999, 2024-09-17T22:41:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042551611000000003, 2024-09-17T22:41:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145778 2024-09-17T22:41:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364203+0000", 2024-09-17T22:41:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364203+0000", 2024-09-17T22:41:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364399+0000", 2024-09-17T22:41:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000196386 2024-09-17T22:41:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364403+0000", 2024-09-17T22:41:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2690000000000001e-06 2024-09-17T22:41:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364413+0000", 2024-09-17T22:41:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0183999999999999e-05 2024-09-17T22:41:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364421+0000", 2024-09-17T22:41:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9009999999999994e-06 2024-09-17T22:41:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145842 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.371207+0000", 2024-09-17T22:41:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.035492464000000001, 2024-09-17T22:41:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.035587344, 2024-09-17T22:41:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145842 2024-09-17T22:41:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371207+0000", 2024-09-17T22:41:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371207+0000", 2024-09-17T22:41:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371207+0000", 2024-09-17T22:41:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.84e-07 2024-09-17T22:41:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371209+0000", 2024-09-17T22:41:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2900000000000001e-06 2024-09-17T22:41:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371221+0000", 2024-09-17T22:41:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2048000000000001e-05 2024-09-17T22:41:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371227+0000", 2024-09-17T22:41:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8599999999999998e-06 2024-09-17T22:41:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145907 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.377340+0000", 2024-09-17T22:41:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.029359045, 2024-09-17T22:41:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029508171999999999, 2024-09-17T22:41:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.440 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145907 2024-09-17T22:41:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377340+0000", 2024-09-17T22:41:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377340+0000", 2024-09-17T22:41:49.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379049+0000", 2024-09-17T22:41:49.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017093080000000001 2024-09-17T22:41:49.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379054+0000", 2024-09-17T22:41:49.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9019999999999998e-06 2024-09-17T22:41:49.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379067+0000", 2024-09-17T22:41:49.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2469999999999999e-05 2024-09-17T22:41:49.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379075+0000", 2024-09-17T22:41:49.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7870000000000002e-06 2024-09-17T22:41:49.443 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.443 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.443 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.444 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145968 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.444 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.383219+0000", 2024-09-17T22:41:49.444 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.023480685000000001, 2024-09-17T22:41:49.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023685218000000001, 2024-09-17T22:41:49.444 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.444 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.444 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.444 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.444 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.444 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.444 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145968 2024-09-17T22:41:49.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.445 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383219+0000", 2024-09-17T22:41:49.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383219+0000", 2024-09-17T22:41:49.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383219+0000", 2024-09-17T22:41:49.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0999999999999999e-07 2024-09-17T22:41:49.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383221+0000", 2024-09-17T22:41:49.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3939999999999999e-06 2024-09-17T22:41:49.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383228+0000", 2024-09-17T22:41:49.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1650000000000003e-06 2024-09-17T22:41:49.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383232+0000", 2024-09-17T22:41:49.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7160000000000002e-06 2024-09-17T22:41:49.447 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.447 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.447 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.447 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146032 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.447 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.389387+0000", 2024-09-17T22:41:49.447 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.017312663999999998, 2024-09-17T22:41:49.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017574181000000001, 2024-09-17T22:41:49.447 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.448 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.448 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.448 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.448 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.448 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.448 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146032 2024-09-17T22:41:49.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.448 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389387+0000", 2024-09-17T22:41:49.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389387+0000", 2024-09-17T22:41:49.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389387+0000", 2024-09-17T22:41:49.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:41:49.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389389+0000", 2024-09-17T22:41:49.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3630000000000001e-06 2024-09-17T22:41:49.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389396+0000", 2024-09-17T22:41:49.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-09-17T22:41:49.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389401+0000", 2024-09-17T22:41:49.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2700000000000004e-06 2024-09-17T22:41:49.450 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.450 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.450 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.451 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146097 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.451 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395266+0000", 2024-09-17T22:41:49.451 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.011433254, 2024-09-17T22:41:49.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011748335, 2024-09-17T22:41:49.451 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.451 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.451 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.451 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.451 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.451 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.451 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146097 2024-09-17T22:41:49.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.452 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395266+0000", 2024-09-17T22:41:49.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395266+0000", 2024-09-17T22:41:49.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395267+0000", 2024-09-17T22:41:49.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4199999999999996e-07 2024-09-17T22:41:49.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395269+0000", 2024-09-17T22:41:49.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.937e-06 2024-09-17T22:41:49.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395277+0000", 2024-09-17T22:41:49.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0520000000000004e-06 2024-09-17T22:41:49.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395282+0000", 2024-09-17T22:41:49.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4759999999999997e-06 2024-09-17T22:41:49.454 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.454 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.454 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.454 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146161 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.454 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403131+0000", 2024-09-17T22:41:49.454 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0035679900000000001, 2024-09-17T22:41:49.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039384859999999997, 2024-09-17T22:41:49.454 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.454 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.455 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.455 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.455 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.455 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.455 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146161 2024-09-17T22:41:49.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.455 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403131+0000", 2024-09-17T22:41:49.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403131+0000", 2024-09-17T22:41:49.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403132+0000", 2024-09-17T22:41:49.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:41:49.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403134+0000", 2024-09-17T22:41:49.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000001e-06 2024-09-17T22:41:49.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403140+0000", 2024-09-17T22:41:49.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2310000000000001e-06 2024-09-17T22:41:49.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403145+0000", 2024-09-17T22:41:49.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:41:49.457 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.457 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.457 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.458 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145716 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.458 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.357364+0000", 2024-09-17T22:41:49.458 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.049335253000000003, 2024-09-17T22:41:49.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049760857999999998, 2024-09-17T22:41:49.458 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.458 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.458 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.458 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.458 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.458 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.458 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145716 2024-09-17T22:41:49.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.459 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357364+0000", 2024-09-17T22:41:49.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357364+0000", 2024-09-17T22:41:49.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357474+0000", 2024-09-17T22:41:49.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109781 2024-09-17T22:41:49.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357478+0000", 2024-09-17T22:41:49.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2679999999999996e-06 2024-09-17T22:41:49.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357489+0000", 2024-09-17T22:41:49.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0550000000000001e-05 2024-09-17T22:41:49.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357505+0000", 2024-09-17T22:41:49.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6120000000000002e-05 2024-09-17T22:41:49.461 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.461 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.461 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.461 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145780 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.461 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.364433+0000", 2024-09-17T22:41:49.461 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.042265846000000003, 2024-09-17T22:41:49.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042748135, 2024-09-17T22:41:49.462 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.462 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.462 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.462 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.462 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.462 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.462 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145780 2024-09-17T22:41:49.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.462 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364433+0000", 2024-09-17T22:41:49.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364433+0000", 2024-09-17T22:41:49.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364874+0000", 2024-09-17T22:41:49.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000440767 2024-09-17T22:41:49.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364883+0000", 2024-09-17T22:41:49.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4069999999999993e-06 2024-09-17T22:41:49.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364894+0000", 2024-09-17T22:41:49.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.133e-05 2024-09-17T22:41:49.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364902+0000", 2024-09-17T22:41:49.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7239999999999992e-06 2024-09-17T22:41:49.465 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.465 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.465 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.465 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145843 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.465 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.371235+0000", 2024-09-17T22:41:49.465 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.035464451000000001, 2024-09-17T22:41:49.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036000773999999999, 2024-09-17T22:41:49.465 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.465 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.465 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.465 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.465 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.466 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.466 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145843 2024-09-17T22:41:49.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.466 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371235+0000", 2024-09-17T22:41:49.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371235+0000", 2024-09-17T22:41:49.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371235+0000", 2024-09-17T22:41:49.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9799999999999999e-07 2024-09-17T22:41:49.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371237+0000", 2024-09-17T22:41:49.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.295e-06 2024-09-17T22:41:49.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371244+0000", 2024-09-17T22:41:49.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2400000000000004e-06 2024-09-17T22:41:49.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371249+0000", 2024-09-17T22:41:49.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9699999999999998e-06 2024-09-17T22:41:49.468 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.468 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.468 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.468 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145909 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.468 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.379103+0000", 2024-09-17T22:41:49.469 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.027596605, 2024-09-17T22:41:49.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028189341, 2024-09-17T22:41:49.469 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.469 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.469 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.469 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.469 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.469 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.469 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145909 2024-09-17T22:41:49.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.469 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379103+0000", 2024-09-17T22:41:49.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379103+0000", 2024-09-17T22:41:49.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379103+0000", 2024-09-17T22:41:49.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1500000000000004e-07 2024-09-17T22:41:49.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379106+0000", 2024-09-17T22:41:49.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.796e-06 2024-09-17T22:41:49.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379114+0000", 2024-09-17T22:41:49.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5209999999999997e-06 2024-09-17T22:41:49.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379120+0000", 2024-09-17T22:41:49.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4849999999999997e-06 2024-09-17T22:41:49.471 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.471 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.472 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.472 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145970 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.472 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.383239+0000", 2024-09-17T22:41:49.472 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.023460267999999999, 2024-09-17T22:41:49.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024106767000000001, 2024-09-17T22:41:49.472 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.472 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.472 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.472 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.472 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.472 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.473 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145970 2024-09-17T22:41:49.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.473 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383239+0000", 2024-09-17T22:41:49.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383239+0000", 2024-09-17T22:41:49.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383239+0000", 2024-09-17T22:41:49.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.32e-07 2024-09-17T22:41:49.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383242+0000", 2024-09-17T22:41:49.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2400000000000002e-06 2024-09-17T22:41:49.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383248+0000", 2024-09-17T22:41:49.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1519999999999997e-06 2024-09-17T22:41:49.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383253+0000", 2024-09-17T22:41:49.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9150000000000004e-06 2024-09-17T22:41:49.475 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.476 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.476 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.476 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146034 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.476 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.389408+0000", 2024-09-17T22:41:49.476 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.017291342000000001, 2024-09-17T22:41:49.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017992187, 2024-09-17T22:41:49.476 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.476 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.476 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.476 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.476 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.477 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.477 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146034 2024-09-17T22:41:49.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.477 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389408+0000", 2024-09-17T22:41:49.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389408+0000", 2024-09-17T22:41:49.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389408+0000", 2024-09-17T22:41:49.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9799999999999999e-07 2024-09-17T22:41:49.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389411+0000", 2024-09-17T22:41:49.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2929999999999999e-06 2024-09-17T22:41:49.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389417+0000", 2024-09-17T22:41:49.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-09-17T22:41:49.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389421+0000", 2024-09-17T22:41:49.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8539999999999996e-06 2024-09-17T22:41:49.479 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.479 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.479 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.480 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146099 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.480 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395291+0000", 2024-09-17T22:41:49.480 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.011407778, 2024-09-17T22:41:49.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012161326, 2024-09-17T22:41:49.480 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.480 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.480 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.480 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.480 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.480 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.480 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146099 2024-09-17T22:41:49.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.481 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395291+0000", 2024-09-17T22:41:49.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395291+0000", 2024-09-17T22:41:49.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397050+0000", 2024-09-17T22:41:49.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001758115 2024-09-17T22:41:49.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397054+0000", 2024-09-17T22:41:49.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6859999999999997e-06 2024-09-17T22:41:49.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397064+0000", 2024-09-17T22:41:49.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3209999999999994e-06 2024-09-17T22:41:49.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397070+0000", 2024-09-17T22:41:49.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4130000000000003e-06 2024-09-17T22:41:49.483 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.483 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.483 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.483 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146163 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.483 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403152+0000", 2024-09-17T22:41:49.483 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0035476090000000002, 2024-09-17T22:41:49.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004355332, 2024-09-17T22:41:49.484 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.484 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.484 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.484 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.484 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.484 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.484 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146163 2024-09-17T22:41:49.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.484 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403152+0000", 2024-09-17T22:41:49.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403152+0000", 2024-09-17T22:41:49.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403152+0000", 2024-09-17T22:41:49.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:41:49.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403159+0000", 2024-09-17T22:41:49.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6889999999999998e-06 2024-09-17T22:41:49.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403165+0000", 2024-09-17T22:41:49.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0240000000000003e-06 2024-09-17T22:41:49.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403170+0000", 2024-09-17T22:41:49.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9599999999999999e-06 2024-09-17T22:41:49.486 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.487 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.487 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.487 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145718 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.487 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.357519+0000", 2024-09-17T22:41:49.487 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.049180531999999999, 2024-09-17T22:41:49.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050042479000000001, 2024-09-17T22:41:49.487 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.487 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.487 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.487 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.487 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.487 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.488 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145718 2024-09-17T22:41:49.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.488 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357519+0000", 2024-09-17T22:41:49.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357519+0000", 2024-09-17T22:41:49.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357718+0000", 2024-09-17T22:41:49.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019947000000000001 2024-09-17T22:41:49.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357723+0000", 2024-09-17T22:41:49.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0699999999999997e-06 2024-09-17T22:41:49.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357735+0000", 2024-09-17T22:41:49.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1771e-05 2024-09-17T22:41:49.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357740+0000", 2024-09-17T22:41:49.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8250000000000004e-06 2024-09-17T22:41:49.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.490 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.490 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145782 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.490 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.364923+0000", 2024-09-17T22:41:49.490 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.041776373999999998, 2024-09-17T22:41:49.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042708493, 2024-09-17T22:41:49.491 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.491 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.491 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.491 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145782 2024-09-17T22:41:49.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.491 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364923+0000", 2024-09-17T22:41:49.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364923+0000", 2024-09-17T22:41:49.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364923+0000", 2024-09-17T22:41:49.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7800000000000001e-07 2024-09-17T22:41:49.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364926+0000", 2024-09-17T22:41:49.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8710000000000001e-06 2024-09-17T22:41:49.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364934+0000", 2024-09-17T22:41:49.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3640000000000001e-06 2024-09-17T22:41:49.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364940+0000", 2024-09-17T22:41:49.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1890000000000004e-06 2024-09-17T22:41:49.494 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.494 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.494 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.494 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145845 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.494 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.371255+0000", 2024-09-17T22:41:49.494 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.035444691, 2024-09-17T22:41:49.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036432134999999997, 2024-09-17T22:41:49.494 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.494 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.494 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.494 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.495 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.495 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.495 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145845 2024-09-17T22:41:49.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.495 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371255+0000", 2024-09-17T22:41:49.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371255+0000", 2024-09-17T22:41:49.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371255+0000", 2024-09-17T22:41:49.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3999999999999997e-07 2024-09-17T22:41:49.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371257+0000", 2024-09-17T22:41:49.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9209999999999999e-06 2024-09-17T22:41:49.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371262+0000", 2024-09-17T22:41:49.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2920000000000003e-06 2024-09-17T22:41:49.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371267+0000", 2024-09-17T22:41:49.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5260000000000004e-06 2024-09-17T22:41:49.497 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.497 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.497 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.497 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145911 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.497 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.379129+0000", 2024-09-17T22:41:49.498 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.027570483999999999, 2024-09-17T22:41:49.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028615465, 2024-09-17T22:41:49.498 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.498 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.498 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.498 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.498 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.498 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.498 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145911 2024-09-17T22:41:49.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.498 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379129+0000", 2024-09-17T22:41:49.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379129+0000", 2024-09-17T22:41:49.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379129+0000", 2024-09-17T22:41:49.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.34e-07 2024-09-17T22:41:49.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379132+0000", 2024-09-17T22:41:49.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.373e-06 2024-09-17T22:41:49.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379138+0000", 2024-09-17T22:41:49.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2729999999999999e-06 2024-09-17T22:41:49.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379143+0000", 2024-09-17T22:41:49.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-09-17T22:41:49.501 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.501 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.501 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.501 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145972 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.501 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.383259+0000", 2024-09-17T22:41:49.501 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.02344011, 2024-09-17T22:41:49.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024540457000000002, 2024-09-17T22:41:49.501 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.501 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.501 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.501 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.502 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.502 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.502 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145972 2024-09-17T22:41:49.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.502 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383259+0000", 2024-09-17T22:41:49.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383259+0000", 2024-09-17T22:41:49.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383260+0000", 2024-09-17T22:41:49.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0399999999999996e-07 2024-09-17T22:41:49.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383266+0000", 2024-09-17T22:41:49.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0650000000000004e-06 2024-09-17T22:41:49.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383272+0000", 2024-09-17T22:41:49.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1160000000000004e-06 2024-09-17T22:41:49.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383277+0000", 2024-09-17T22:41:49.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-09-17T22:41:49.504 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.504 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.504 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.504 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146036 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.504 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.389427+0000", 2024-09-17T22:41:49.505 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.017272077, 2024-09-17T22:41:49.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018424867000000001, 2024-09-17T22:41:49.505 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.505 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.505 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.505 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.505 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.505 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.505 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146036 2024-09-17T22:41:49.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.505 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389427+0000", 2024-09-17T22:41:49.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389427+0000", 2024-09-17T22:41:49.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391051+0000", 2024-09-17T22:41:49.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001624068 2024-09-17T22:41:49.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391056+0000", 2024-09-17T22:41:49.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4580000000000003e-06 2024-09-17T22:41:49.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391064+0000", 2024-09-17T22:41:49.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8189999999999998e-06 2024-09-17T22:41:49.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391071+0000", 2024-09-17T22:41:49.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6830000000000002e-06 2024-09-17T22:41:49.508 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.508 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.508 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.508 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146101 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.508 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.397083+0000", 2024-09-17T22:41:49.508 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.009616309, 2024-09-17T22:41:49.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010826258, 2024-09-17T22:41:49.508 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.508 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.508 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.508 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.509 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.509 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.509 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146101 2024-09-17T22:41:49.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.509 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397083+0000", 2024-09-17T22:41:49.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397083+0000", 2024-09-17T22:41:49.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397083+0000", 2024-09-17T22:41:49.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4499999999999997e-07 2024-09-17T22:41:49.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397085+0000", 2024-09-17T22:41:49.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9149999999999999e-06 2024-09-17T22:41:49.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397089+0000", 2024-09-17T22:41:49.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0110000000000002e-06 2024-09-17T22:41:49.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397093+0000", 2024-09-17T22:41:49.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3009999999999998e-06 2024-09-17T22:41:49.511 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.511 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.511 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.511 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146165 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.511 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403176+0000", 2024-09-17T22:41:49.512 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.003522837, 2024-09-17T22:41:49.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047858240000000002, 2024-09-17T22:41:49.512 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.512 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.512 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.512 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.512 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.512 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.512 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146165 2024-09-17T22:41:49.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.512 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403176+0000", 2024-09-17T22:41:49.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403176+0000", 2024-09-17T22:41:49.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403177+0000", 2024-09-17T22:41:49.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.27e-07 2024-09-17T22:41:49.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403179+0000", 2024-09-17T22:41:49.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1380000000000001e-06 2024-09-17T22:41:49.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403185+0000", 2024-09-17T22:41:49.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6540000000000004e-06 2024-09-17T22:41:49.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403189+0000", 2024-09-17T22:41:49.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6179999999999997e-06 2024-09-17T22:41:49.515 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.515 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.515 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.515 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145720 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.515 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.357749+0000", 2024-09-17T22:41:49.515 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.048949811000000003, 2024-09-17T22:41:49.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050266600000000002, 2024-09-17T22:41:49.515 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.515 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.515 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.515 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.516 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145720 2024-09-17T22:41:49.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.516 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357749+0000", 2024-09-17T22:41:49.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357749+0000", 2024-09-17T22:41:49.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357948+0000", 2024-09-17T22:41:49.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019859599999999999 2024-09-17T22:41:49.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357953+0000", 2024-09-17T22:41:49.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0010000000000001e-06 2024-09-17T22:41:49.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357963+0000", 2024-09-17T22:41:49.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0380999999999999e-05 2024-09-17T22:41:49.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357971+0000", 2024-09-17T22:41:49.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6290000000000001e-06 2024-09-17T22:41:49.518 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.518 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.518 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.519 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145784 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.519 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.364949+0000", 2024-09-17T22:41:49.519 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.041749789000000002, 2024-09-17T22:41:49.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043101980999999998, 2024-09-17T22:41:49.519 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.519 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.519 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.519 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.519 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.519 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.519 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145784 2024-09-17T22:41:49.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.519 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364949+0000", 2024-09-17T22:41:49.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.364949+0000", 2024-09-17T22:41:49.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.365054+0000", 2024-09-17T22:41:49.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104184 2024-09-17T22:41:49.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.365057+0000", 2024-09-17T22:41:49.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7670000000000001e-06 2024-09-17T22:41:49.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.365067+0000", 2024-09-17T22:41:49.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1409999999999994e-06 2024-09-17T22:41:49.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.365073+0000", 2024-09-17T22:41:49.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5390000000000004e-06 2024-09-17T22:41:49.522 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.522 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.522 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.522 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145847 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.522 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.371272+0000", 2024-09-17T22:41:49.522 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.035426855, 2024-09-17T22:41:49.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036818442, 2024-09-17T22:41:49.523 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.523 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.523 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.523 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.523 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.523 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.523 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145847 2024-09-17T22:41:49.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.523 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371272+0000", 2024-09-17T22:41:49.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371272+0000", 2024-09-17T22:41:49.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371273+0000", 2024-09-17T22:41:49.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6699999999999999e-07 2024-09-17T22:41:49.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371275+0000", 2024-09-17T22:41:49.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2359999999999999e-06 2024-09-17T22:41:49.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371282+0000", 2024-09-17T22:41:49.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4189999999999996e-06 2024-09-17T22:41:49.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371288+0000", 2024-09-17T22:41:49.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1719999999999998e-06 2024-09-17T22:41:49.525 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.525 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.526 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.526 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145913 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.526 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.379149+0000", 2024-09-17T22:41:49.526 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.027549845, 2024-09-17T22:41:49.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028993096999999999, 2024-09-17T22:41:49.526 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.526 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.526 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.527 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145913 2024-09-17T22:41:49.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.527 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379149+0000", 2024-09-17T22:41:49.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379149+0000", 2024-09-17T22:41:49.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379150+0000", 2024-09-17T22:41:49.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2599999999999998e-07 2024-09-17T22:41:49.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379152+0000", 2024-09-17T22:41:49.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4480000000000001e-06 2024-09-17T22:41:49.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379159+0000", 2024-09-17T22:41:49.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3539999999999996e-06 2024-09-17T22:41:49.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379163+0000", 2024-09-17T22:41:49.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.566e-06 2024-09-17T22:41:49.529 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.529 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.529 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.529 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145974 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.529 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.383283+0000", 2024-09-17T22:41:49.529 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.023416091, 2024-09-17T22:41:49.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024912752, 2024-09-17T22:41:49.530 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.530 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.530 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.530 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.530 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.530 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.530 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145974 2024-09-17T22:41:49.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.530 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383283+0000", 2024-09-17T22:41:49.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383283+0000", 2024-09-17T22:41:49.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385051+0000", 2024-09-17T22:41:49.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001767759 2024-09-17T22:41:49.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385056+0000", 2024-09-17T22:41:49.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8250000000000004e-06 2024-09-17T22:41:49.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385067+0000", 2024-09-17T22:41:49.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1735e-05 2024-09-17T22:41:49.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385074+0000", 2024-09-17T22:41:49.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0140000000000001e-06 2024-09-17T22:41:49.532 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.533 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.533 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.533 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146038 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.533 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.391083+0000", 2024-09-17T22:41:49.533 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015615959, 2024-09-17T22:41:49.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017157748, 2024-09-17T22:41:49.533 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.533 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.533 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.533 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.533 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.533 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.534 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146038 2024-09-17T22:41:49.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.534 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391083+0000", 2024-09-17T22:41:49.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391083+0000", 2024-09-17T22:41:49.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391084+0000", 2024-09-17T22:41:49.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9800000000000004e-07 2024-09-17T22:41:49.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391086+0000", 2024-09-17T22:41:49.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.277e-06 2024-09-17T22:41:49.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391094+0000", 2024-09-17T22:41:49.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1380000000000007e-06 2024-09-17T22:41:49.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391100+0000", 2024-09-17T22:41:49.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5509999999999999e-06 2024-09-17T22:41:49.536 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.536 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.536 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.536 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146103 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.536 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.397097+0000", 2024-09-17T22:41:49.537 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0096023789999999994, 2024-09-17T22:41:49.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011197831, 2024-09-17T22:41:49.537 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.537 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.537 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.537 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.537 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.537 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.537 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146103 2024-09-17T22:41:49.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.537 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397097+0000", 2024-09-17T22:41:49.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397097+0000", 2024-09-17T22:41:49.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397097+0000", 2024-09-17T22:41:49.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2999999999999999e-07 2024-09-17T22:41:49.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397098+0000", 2024-09-17T22:41:49.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3060000000000001e-06 2024-09-17T22:41:49.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397101+0000", 2024-09-17T22:41:49.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0690000000000002e-06 2024-09-17T22:41:49.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397104+0000", 2024-09-17T22:41:49.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6479999999999999e-06 2024-09-17T22:41:49.540 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.540 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.540 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.540 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146167 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.540 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403195+0000", 2024-09-17T22:41:49.540 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.003503932, 2024-09-17T22:41:49.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0051525090000000004, 2024-09-17T22:41:49.540 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.540 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.540 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.541 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.541 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.541 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.541 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146167 2024-09-17T22:41:49.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.541 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403195+0000", 2024-09-17T22:41:49.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403195+0000", 2024-09-17T22:41:49.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403196+0000", 2024-09-17T22:41:49.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:41:49.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403198+0000", 2024-09-17T22:41:49.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2330000000000001e-06 2024-09-17T22:41:49.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403204+0000", 2024-09-17T22:41:49.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6720000000000001e-06 2024-09-17T22:41:49.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403208+0000", 2024-09-17T22:41:49.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6550000000000003e-06 2024-09-17T22:41:49.543 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.543 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.543 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.544 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145722 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.544 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.357983+0000", 2024-09-17T22:41:49.544 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.048716395000000003, 2024-09-17T22:41:49.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050422146000000001, 2024-09-17T22:41:49.544 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.544 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.544 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.544 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145722 2024-09-17T22:41:49.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.545 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357983+0000", 2024-09-17T22:41:49.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.357983+0000", 2024-09-17T22:41:49.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358149+0000", 2024-09-17T22:41:49.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016566899999999999 2024-09-17T22:41:49.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358152+0000", 2024-09-17T22:41:49.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.968e-06 2024-09-17T22:41:49.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358162+0000", 2024-09-17T22:41:49.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8810000000000008e-06 2024-09-17T22:41:49.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358169+0000", 2024-09-17T22:41:49.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7279999999999998e-06 2024-09-17T22:41:49.547 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.547 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.547 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.547 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145786 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.547 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.365084+0000", 2024-09-17T22:41:49.547 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.041615408, 2024-09-17T22:41:49.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043375364999999999, 2024-09-17T22:41:49.548 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.548 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.548 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.548 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.548 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.548 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.548 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145786 2024-09-17T22:41:49.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.548 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.365084+0000", 2024-09-17T22:41:49.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.365084+0000", 2024-09-17T22:41:49.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367051+0000", 2024-09-17T22:41:49.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00196765 2024-09-17T22:41:49.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367057+0000", 2024-09-17T22:41:49.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1270000000000002e-06 2024-09-17T22:41:49.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367070+0000", 2024-09-17T22:41:49.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3709e-05 2024-09-17T22:41:49.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367078+0000", 2024-09-17T22:41:49.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6809999999999998e-06 2024-09-17T22:41:49.550 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.550 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.551 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.551 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145849 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.551 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.371294+0000", 2024-09-17T22:41:49.551 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.035405648999999997, 2024-09-17T22:41:49.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037257783000000003, 2024-09-17T22:41:49.551 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.551 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.551 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.551 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.551 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.551 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.552 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145849 2024-09-17T22:41:49.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.552 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371294+0000", 2024-09-17T22:41:49.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371294+0000", 2024-09-17T22:41:49.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373052+0000", 2024-09-17T22:41:49.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017581999999999999 2024-09-17T22:41:49.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373056+0000", 2024-09-17T22:41:49.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5449999999999997e-06 2024-09-17T22:41:49.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373066+0000", 2024-09-17T22:41:49.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9690000000000006e-06 2024-09-17T22:41:49.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373073+0000", 2024-09-17T22:41:49.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6830000000000002e-06 2024-09-17T22:41:49.554 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.554 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.554 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.554 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145915 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.554 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.379169+0000", 2024-09-17T22:41:49.554 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.027529957000000001, 2024-09-17T22:41:49.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029437764000000002, 2024-09-17T22:41:49.555 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.555 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.555 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.555 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.555 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.555 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.555 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145915 2024-09-17T22:41:49.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.555 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379169+0000", 2024-09-17T22:41:49.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379169+0000", 2024-09-17T22:41:49.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379170+0000", 2024-09-17T22:41:49.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3999999999999997e-07 2024-09-17T22:41:49.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379172+0000", 2024-09-17T22:41:49.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0080000000000001e-06 2024-09-17T22:41:49.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379178+0000", 2024-09-17T22:41:49.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9039999999999997e-06 2024-09-17T22:41:49.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379182+0000", 2024-09-17T22:41:49.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6349999999999997e-06 2024-09-17T22:41:49.558 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.558 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.558 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.558 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145976 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.558 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.385088+0000", 2024-09-17T22:41:49.558 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.021611399, 2024-09-17T22:41:49.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023572692999999999, 2024-09-17T22:41:49.558 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.558 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.558 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.559 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145976 2024-09-17T22:41:49.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.559 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385088+0000", 2024-09-17T22:41:49.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385088+0000", 2024-09-17T22:41:49.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385088+0000", 2024-09-17T22:41:49.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5799999999999999e-07 2024-09-17T22:41:49.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385091+0000", 2024-09-17T22:41:49.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.813e-06 2024-09-17T22:41:49.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385098+0000", 2024-09-17T22:41:49.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9870000000000002e-06 2024-09-17T22:41:49.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385104+0000", 2024-09-17T22:41:49.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3399999999999997e-06 2024-09-17T22:41:49.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.561 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.562 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146040 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.562 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.391107+0000", 2024-09-17T22:41:49.562 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.01559259, 2024-09-17T22:41:49.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017609974, 2024-09-17T22:41:49.562 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.562 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.562 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.562 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146040 2024-09-17T22:41:49.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.562 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391107+0000", 2024-09-17T22:41:49.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391107+0000", 2024-09-17T22:41:49.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391107+0000", 2024-09-17T22:41:49.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-09-17T22:41:49.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391109+0000", 2024-09-17T22:41:49.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4250000000000001e-06 2024-09-17T22:41:49.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391116+0000", 2024-09-17T22:41:49.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.156e-06 2024-09-17T22:41:49.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391121+0000", 2024-09-17T22:41:49.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9830000000000004e-06 2024-09-17T22:41:49.565 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.565 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.565 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.565 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146105 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.565 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.397107+0000", 2024-09-17T22:41:49.565 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0095919669999999999, 2024-09-17T22:41:49.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011662753, 2024-09-17T22:41:49.565 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.565 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.566 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.566 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146105 2024-09-17T22:41:49.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.566 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397107+0000", 2024-09-17T22:41:49.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397107+0000", 2024-09-17T22:41:49.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397107+0000", 2024-09-17T22:41:49.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:41:49.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397109+0000", 2024-09-17T22:41:49.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3209999999999999e-06 2024-09-17T22:41:49.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397125+0000", 2024-09-17T22:41:49.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5781e-05 2024-09-17T22:41:49.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397128+0000", 2024-09-17T22:41:49.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1200000000000002e-06 2024-09-17T22:41:49.568 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.568 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.569 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.569 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146169 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.569 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403214+0000", 2024-09-17T22:41:49.569 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.003485021, 2024-09-17T22:41:49.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056126939999999997, 2024-09-17T22:41:49.569 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.569 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.569 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.569 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.569 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.569 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.569 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146169 2024-09-17T22:41:49.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.570 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403214+0000", 2024-09-17T22:41:49.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403214+0000", 2024-09-17T22:41:49.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403215+0000", 2024-09-17T22:41:49.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.41e-07 2024-09-17T22:41:49.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403217+0000", 2024-09-17T22:41:49.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-09-17T22:41:49.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403223+0000", 2024-09-17T22:41:49.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.037e-06 2024-09-17T22:41:49.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403228+0000", 2024-09-17T22:41:49.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-09-17T22:41:49.572 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.572 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.572 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.572 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145724 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.572 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.358180+0000", 2024-09-17T22:41:49.572 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.048519136999999997, 2024-09-17T22:41:49.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050699838999999997, 2024-09-17T22:41:49.573 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.573 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.573 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.573 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.573 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.573 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.573 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145724 2024-09-17T22:41:49.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.573 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358180+0000", 2024-09-17T22:41:49.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358180+0000", 2024-09-17T22:41:49.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358401+0000", 2024-09-17T22:41:49.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022052500000000001 2024-09-17T22:41:49.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358404+0000", 2024-09-17T22:41:49.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8179999999999997e-06 2024-09-17T22:41:49.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358415+0000", 2024-09-17T22:41:49.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0346000000000001e-05 2024-09-17T22:41:49.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358422+0000", 2024-09-17T22:41:49.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5680000000000002e-06 2024-09-17T22:41:49.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.575 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.576 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.576 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145789 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.576 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.367107+0000", 2024-09-17T22:41:49.576 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.039591927999999998, 2024-09-17T22:41:49.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041828510999999999, 2024-09-17T22:41:49.576 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.576 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.576 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.576 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.576 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.576 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.577 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145789 2024-09-17T22:41:49.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.577 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367107+0000", 2024-09-17T22:41:49.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367107+0000", 2024-09-17T22:41:49.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367108+0000", 2024-09-17T22:41:49.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.75e-07 2024-09-17T22:41:49.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367111+0000", 2024-09-17T22:41:49.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.633e-06 2024-09-17T22:41:49.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367118+0000", 2024-09-17T22:41:49.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7610000000000007e-06 2024-09-17T22:41:49.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367124+0000", 2024-09-17T22:41:49.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9549999999999997e-06 2024-09-17T22:41:49.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.579 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.579 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145851 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.579 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.373085+0000", 2024-09-17T22:41:49.579 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.033614205000000001, 2024-09-17T22:41:49.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.035904160999999997, 2024-09-17T22:41:49.580 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.580 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.580 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.580 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145851 2024-09-17T22:41:49.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.580 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373085+0000", 2024-09-17T22:41:49.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373085+0000", 2024-09-17T22:41:49.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373086+0000", 2024-09-17T22:41:49.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5000000000000003e-07 2024-09-17T22:41:49.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373092+0000", 2024-09-17T22:41:49.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8210000000000004e-06 2024-09-17T22:41:49.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373099+0000", 2024-09-17T22:41:49.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9739999999999996e-06 2024-09-17T22:41:49.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373105+0000", 2024-09-17T22:41:49.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.446e-06 2024-09-17T22:41:49.583 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.583 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.583 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.583 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145917 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.583 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.379249+0000", 2024-09-17T22:41:49.583 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.027450665999999999, 2024-09-17T22:41:49.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029792268, 2024-09-17T22:41:49.583 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.583 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.584 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.584 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.584 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.584 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.584 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145917 2024-09-17T22:41:49.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.584 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379249+0000", 2024-09-17T22:41:49.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379249+0000", 2024-09-17T22:41:49.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379249+0000", 2024-09-17T22:41:49.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8899999999999999e-07 2024-09-17T22:41:49.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379252+0000", 2024-09-17T22:41:49.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0989999999999999e-06 2024-09-17T22:41:49.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379262+0000", 2024-09-17T22:41:49.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6360000000000006e-06 2024-09-17T22:41:49.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379269+0000", 2024-09-17T22:41:49.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8549999999999996e-06 2024-09-17T22:41:49.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.586 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.587 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145978 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.587 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.385111+0000", 2024-09-17T22:41:49.587 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.021588612, 2024-09-17T22:41:49.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023987468000000001, 2024-09-17T22:41:49.587 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.587 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.587 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.587 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145978 2024-09-17T22:41:49.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385111+0000", 2024-09-17T22:41:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385111+0000", 2024-09-17T22:41:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385111+0000", 2024-09-17T22:41:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5999999999999999e-07 2024-09-17T22:41:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385113+0000", 2024-09-17T22:41:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1009999999999999e-06 2024-09-17T22:41:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385120+0000", 2024-09-17T22:41:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1420000000000004e-06 2024-09-17T22:41:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385125+0000", 2024-09-17T22:41:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9060000000000001e-06 2024-09-17T22:41:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146042 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.391127+0000", 2024-09-17T22:41:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015572317, 2024-09-17T22:41:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018012912999999998, 2024-09-17T22:41:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146042 2024-09-17T22:41:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391127+0000", 2024-09-17T22:41:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391127+0000", 2024-09-17T22:41:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391127+0000", 2024-09-17T22:41:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-09-17T22:41:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391133+0000", 2024-09-17T22:41:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1800000000000001e-06 2024-09-17T22:41:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391139+0000", 2024-09-17T22:41:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8769999999999998e-06 2024-09-17T22:41:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391145+0000", 2024-09-17T22:41:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4199999999999998e-06 2024-09-17T22:41:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146107 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.397132+0000", 2024-09-17T22:41:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0095676170000000005, 2024-09-17T22:41:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012048979, 2024-09-17T22:41:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146107 2024-09-17T22:41:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397132+0000", 2024-09-17T22:41:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397132+0000", 2024-09-17T22:41:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397132+0000", 2024-09-17T22:41:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5199999999999998e-07 2024-09-17T22:41:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397133+0000", 2024-09-17T22:41:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.435e-06 2024-09-17T22:41:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397136+0000", 2024-09-17T22:41:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0750000000000002e-06 2024-09-17T22:41:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397139+0000", 2024-09-17T22:41:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5050000000000002e-06 2024-09-17T22:41:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146171 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403234+0000", 2024-09-17T22:41:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0034648790000000001, 2024-09-17T22:41:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059766250000000002, 2024-09-17T22:41:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146171 2024-09-17T22:41:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403234+0000", 2024-09-17T22:41:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403234+0000", 2024-09-17T22:41:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403235+0000", 2024-09-17T22:41:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7800000000000002e-07 2024-09-17T22:41:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403237+0000", 2024-09-17T22:41:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9819999999999998e-06 2024-09-17T22:41:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403243+0000", 2024-09-17T22:41:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9259999999999997e-06 2024-09-17T22:41:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403248+0000", 2024-09-17T22:41:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9779999999999996e-06 2024-09-17T22:41:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145726 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.358433+0000", 2024-09-17T22:41:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.048266170999999997, 2024-09-17T22:41:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050805267000000001, 2024-09-17T22:41:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145726 2024-09-17T22:41:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358433+0000", 2024-09-17T22:41:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358433+0000", 2024-09-17T22:41:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358580+0000", 2024-09-17T22:41:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000146854 2024-09-17T22:41:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358584+0000", 2024-09-17T22:41:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7670000000000001e-06 2024-09-17T22:41:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358600+0000", 2024-09-17T22:41:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6096999999999999e-05 2024-09-17T22:41:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358607+0000", 2024-09-17T22:41:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3830000000000003e-06 2024-09-17T22:41:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145791 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.367133+0000", 2024-09-17T22:41:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.039566399000000002, 2024-09-17T22:41:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042134592999999998, 2024-09-17T22:41:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145791 2024-09-17T22:41:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367133+0000", 2024-09-17T22:41:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367133+0000", 2024-09-17T22:41:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367133+0000", 2024-09-17T22:41:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:41:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367135+0000", 2024-09-17T22:41:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1050000000000002e-06 2024-09-17T22:41:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367142+0000", 2024-09-17T22:41:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3659999999999997e-06 2024-09-17T22:41:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367147+0000", 2024-09-17T22:41:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9250000000000003e-06 2024-09-17T22:41:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145853 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.373114+0000", 2024-09-17T22:41:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.033585470999999999, 2024-09-17T22:41:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036183086000000003, 2024-09-17T22:41:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145853 2024-09-17T22:41:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373114+0000", 2024-09-17T22:41:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373114+0000", 2024-09-17T22:41:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373114+0000", 2024-09-17T22:41:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2500000000000001e-07 2024-09-17T22:41:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373116+0000", 2024-09-17T22:41:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0889999999999998e-06 2024-09-17T22:41:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373126+0000", 2024-09-17T22:41:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6229999999999992e-06 2024-09-17T22:41:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373131+0000", 2024-09-17T22:41:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.899e-06 2024-09-17T22:41:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145919 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.379280+0000", 2024-09-17T22:41:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.027419392000000001, 2024-09-17T22:41:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030046439000000001, 2024-09-17T22:41:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145919 2024-09-17T22:41:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379280+0000", 2024-09-17T22:41:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379280+0000", 2024-09-17T22:41:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379280+0000", 2024-09-17T22:41:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7300000000000001e-07 2024-09-17T22:41:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379283+0000", 2024-09-17T22:41:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3980000000000002e-06 2024-09-17T22:41:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379300+0000", 2024-09-17T22:41:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.749e-05 2024-09-17T22:41:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379310+0000", 2024-09-17T22:41:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7799999999999995e-06 2024-09-17T22:41:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145980 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.385160+0000", 2024-09-17T22:41:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.021539368, 2024-09-17T22:41:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02419344, 2024-09-17T22:41:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145980 2024-09-17T22:41:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385160+0000", 2024-09-17T22:41:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385160+0000", 2024-09-17T22:41:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385160+0000", 2024-09-17T22:41:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.06e-07 2024-09-17T22:41:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385163+0000", 2024-09-17T22:41:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5730000000000002e-06 2024-09-17T22:41:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385170+0000", 2024-09-17T22:41:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2359999999999998e-06 2024-09-17T22:41:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385176+0000", 2024-09-17T22:41:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4049999999999999e-06 2024-09-17T22:41:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146044 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.391156+0000", 2024-09-17T22:41:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015543646, 2024-09-17T22:41:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018241911, 2024-09-17T22:41:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146044 2024-09-17T22:41:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391156+0000", 2024-09-17T22:41:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391156+0000", 2024-09-17T22:41:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391156+0000", 2024-09-17T22:41:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:41:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391158+0000", 2024-09-17T22:41:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3209999999999999e-06 2024-09-17T22:41:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391164+0000", 2024-09-17T22:41:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1190000000000002e-06 2024-09-17T22:41:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391168+0000", 2024-09-17T22:41:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2109999999999998e-06 2024-09-17T22:41:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146109 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.397142+0000", 2024-09-17T22:41:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0095571749999999994, 2024-09-17T22:41:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012291475, 2024-09-17T22:41:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146109 2024-09-17T22:41:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397142+0000", 2024-09-17T22:41:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397142+0000", 2024-09-17T22:41:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397142+0000", 2024-09-17T22:41:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-09-17T22:41:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397143+0000", 2024-09-17T22:41:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1650000000000001e-06 2024-09-17T22:41:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397146+0000", 2024-09-17T22:41:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0599999999999999e-06 2024-09-17T22:41:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397149+0000", 2024-09-17T22:41:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.481e-06 2024-09-17T22:41:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146173 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403253+0000", 2024-09-17T22:41:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0034457749999999999, 2024-09-17T22:41:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062198640000000003, 2024-09-17T22:41:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146173 2024-09-17T22:41:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403253+0000", 2024-09-17T22:41:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403253+0000", 2024-09-17T22:41:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403254+0000", 2024-09-17T22:41:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.65e-07 2024-09-17T22:41:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403256+0000", 2024-09-17T22:41:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9410000000000001e-06 2024-09-17T22:41:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403261+0000", 2024-09-17T22:41:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4729999999999999e-06 2024-09-17T22:41:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403266+0000", 2024-09-17T22:41:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7670000000000003e-06 2024-09-17T22:41:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145728 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.358619+0000", 2024-09-17T22:41:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.048080471999999999, 2024-09-17T22:41:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050909483999999998, 2024-09-17T22:41:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145728 2024-09-17T22:41:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358619+0000", 2024-09-17T22:41:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358619+0000", 2024-09-17T22:41:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358926+0000", 2024-09-17T22:41:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030703600000000001 2024-09-17T22:41:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358930+0000", 2024-09-17T22:41:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3839999999999999e-06 2024-09-17T22:41:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358944+0000", 2024-09-17T22:41:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3689000000000001e-05 2024-09-17T22:41:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358949+0000", 2024-09-17T22:41:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0969999999999996e-06 2024-09-17T22:41:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145793 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.367155+0000", 2024-09-17T22:41:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.039544321, 2024-09-17T22:41:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042408466999999998, 2024-09-17T22:41:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145793 2024-09-17T22:41:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367155+0000", 2024-09-17T22:41:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367155+0000", 2024-09-17T22:41:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367155+0000", 2024-09-17T22:41:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-09-17T22:41:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367157+0000", 2024-09-17T22:41:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.131e-06 2024-09-17T22:41:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367163+0000", 2024-09-17T22:41:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9859999999999999e-06 2024-09-17T22:41:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367175+0000", 2024-09-17T22:41:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1199000000000001e-05 2024-09-17T22:41:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145855 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.373138+0000", 2024-09-17T22:41:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.033560933000000001, 2024-09-17T22:41:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036474098000000003, 2024-09-17T22:41:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145855 2024-09-17T22:41:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373138+0000", 2024-09-17T22:41:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373138+0000", 2024-09-17T22:41:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373139+0000", 2024-09-17T22:41:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1300000000000001e-07 2024-09-17T22:41:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373141+0000", 2024-09-17T22:41:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0439999999999998e-06 2024-09-17T22:41:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373149+0000", 2024-09-17T22:41:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8509999999999999e-06 2024-09-17T22:41:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373153+0000", 2024-09-17T22:41:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6439999999999999e-06 2024-09-17T22:41:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145921 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.379319+0000", 2024-09-17T22:41:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.027380682, 2024-09-17T22:41:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.03034413, 2024-09-17T22:41:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145921 2024-09-17T22:41:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379319+0000", 2024-09-17T22:41:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379319+0000", 2024-09-17T22:41:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379319+0000", 2024-09-17T22:41:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2e-07 2024-09-17T22:41:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379322+0000", 2024-09-17T22:41:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.481e-06 2024-09-17T22:41:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379328+0000", 2024-09-17T22:41:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7880000000000001e-06 2024-09-17T22:41:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379336+0000", 2024-09-17T22:41:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8849999999999999e-06 2024-09-17T22:41:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145982 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.385210+0000", 2024-09-17T22:41:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.021489042, 2024-09-17T22:41:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024484431000000001, 2024-09-17T22:41:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145982 2024-09-17T22:41:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385210+0000", 2024-09-17T22:41:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385210+0000", 2024-09-17T22:41:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385211+0000", 2024-09-17T22:41:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7400000000000003e-07 2024-09-17T22:41:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385214+0000", 2024-09-17T22:41:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1159999999999999e-06 2024-09-17T22:41:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385221+0000", 2024-09-17T22:41:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3690000000000001e-06 2024-09-17T22:41:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385227+0000", 2024-09-17T22:41:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8479999999999997e-06 2024-09-17T22:41:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146046 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.391172+0000", 2024-09-17T22:41:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015527302999999999, 2024-09-17T22:41:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018574889000000001, 2024-09-17T22:41:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146046 2024-09-17T22:41:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391172+0000", 2024-09-17T22:41:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391172+0000", 2024-09-17T22:41:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391172+0000", 2024-09-17T22:41:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2099999999999998e-07 2024-09-17T22:41:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391174+0000", 2024-09-17T22:41:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.795e-06 2024-09-17T22:41:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391179+0000", 2024-09-17T22:41:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.22e-06 2024-09-17T22:41:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391184+0000", 2024-09-17T22:41:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3510000000000004e-06 2024-09-17T22:41:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146111 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.397152+0000", 2024-09-17T22:41:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0095472790000000005, 2024-09-17T22:41:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012645920999999999, 2024-09-17T22:41:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146111 2024-09-17T22:41:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397152+0000", 2024-09-17T22:41:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397152+0000", 2024-09-17T22:41:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397152+0000", 2024-09-17T22:41:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.23e-07 2024-09-17T22:41:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397153+0000", 2024-09-17T22:41:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.22e-06 2024-09-17T22:41:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397156+0000", 2024-09-17T22:41:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9510000000000002e-06 2024-09-17T22:41:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397159+0000", 2024-09-17T22:41:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5040000000000001e-06 2024-09-17T22:41:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146175 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403272+0000", 2024-09-17T22:41:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0034270529999999998, 2024-09-17T22:41:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006575511, 2024-09-17T22:41:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146175 2024-09-17T22:41:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403272+0000", 2024-09-17T22:41:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403272+0000", 2024-09-17T22:41:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405052+0000", 2024-09-17T22:41:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001779379 2024-09-17T22:41:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405056+0000", 2024-09-17T22:41:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:41:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405065+0000", 2024-09-17T22:41:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3200000000000006e-06 2024-09-17T22:41:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405074+0000", 2024-09-17T22:41:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0250000000000002e-06 2024-09-17T22:41:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145730 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.358958+0000", 2024-09-17T22:41:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.047741027999999998, 2024-09-17T22:41:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050953998, 2024-09-17T22:41:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145730 2024-09-17T22:41:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358958+0000", 2024-09-17T22:41:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.358958+0000", 2024-09-17T22:41:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.359141+0000", 2024-09-17T22:41:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018263200000000001 2024-09-17T22:41:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.359145+0000", 2024-09-17T22:41:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7799999999999998e-06 2024-09-17T22:41:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.359192+0000", 2024-09-17T22:41:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.778e-05 2024-09-17T22:41:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.359202+0000", 2024-09-17T22:41:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5359999999999998e-06 2024-09-17T22:41:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145795 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.367183+0000", 2024-09-17T22:41:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.039516369000000003, 2024-09-17T22:41:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042773613000000002, 2024-09-17T22:41:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145795 2024-09-17T22:41:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367183+0000", 2024-09-17T22:41:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367183+0000", 2024-09-17T22:41:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367183+0000", 2024-09-17T22:41:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-09-17T22:41:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367185+0000", 2024-09-17T22:41:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1739999999999999e-06 2024-09-17T22:41:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367202+0000", 2024-09-17T22:41:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6327000000000001e-05 2024-09-17T22:41:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367207+0000", 2024-09-17T22:41:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5350000000000004e-06 2024-09-17T22:41:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145857 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.373160+0000", 2024-09-17T22:41:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.033539378000000002, 2024-09-17T22:41:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036843687, 2024-09-17T22:41:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145857 2024-09-17T22:41:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373160+0000", 2024-09-17T22:41:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373160+0000", 2024-09-17T22:41:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373160+0000", 2024-09-17T22:41:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1300000000000001e-07 2024-09-17T22:41:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373163+0000", 2024-09-17T22:41:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2699999999999999e-06 2024-09-17T22:41:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373168+0000", 2024-09-17T22:41:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.699e-06 2024-09-17T22:41:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373173+0000", 2024-09-17T22:41:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4950000000000002e-06 2024-09-17T22:41:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145923 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.379345+0000", 2024-09-17T22:41:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.027353908999999999, 2024-09-17T22:41:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030695139999999999, 2024-09-17T22:41:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145923 2024-09-17T22:41:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379345+0000", 2024-09-17T22:41:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379345+0000", 2024-09-17T22:41:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379346+0000", 2024-09-17T22:41:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:41:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379348+0000", 2024-09-17T22:41:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.181e-06 2024-09-17T22:41:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379354+0000", 2024-09-17T22:41:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5189999999999998e-06 2024-09-17T22:41:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379361+0000", 2024-09-17T22:41:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3210000000000001e-06 2024-09-17T22:41:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145984 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.385234+0000", 2024-09-17T22:41:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.021465534000000001, 2024-09-17T22:41:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02489574, 2024-09-17T22:41:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145984 2024-09-17T22:41:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385234+0000", 2024-09-17T22:41:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385234+0000", 2024-09-17T22:41:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385234+0000", 2024-09-17T22:41:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5900000000000002e-07 2024-09-17T22:41:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385236+0000", 2024-09-17T22:41:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0940000000000002e-06 2024-09-17T22:41:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385243+0000", 2024-09-17T22:41:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6919999999999996e-06 2024-09-17T22:41:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385248+0000", 2024-09-17T22:41:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7550000000000002e-06 2024-09-17T22:41:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146048 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.391190+0000", 2024-09-17T22:41:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015509438, 2024-09-17T22:41:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018989906000000001, 2024-09-17T22:41:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146048 2024-09-17T22:41:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391190+0000", 2024-09-17T22:41:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391190+0000", 2024-09-17T22:41:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391190+0000", 2024-09-17T22:41:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6400000000000003e-07 2024-09-17T22:41:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391192+0000", 2024-09-17T22:41:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-17T22:41:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391198+0000", 2024-09-17T22:41:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5330000000000002e-06 2024-09-17T22:41:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391202+0000", 2024-09-17T22:41:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4440000000000001e-06 2024-09-17T22:41:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146113 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.397162+0000", 2024-09-17T22:41:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0095374450000000003, 2024-09-17T22:41:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013067491000000001, 2024-09-17T22:41:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146113 2024-09-17T22:41:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397162+0000", 2024-09-17T22:41:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397162+0000", 2024-09-17T22:41:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397162+0000", 2024-09-17T22:41:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0599999999999999e-07 2024-09-17T22:41:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397163+0000", 2024-09-17T22:41:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1039999999999999e-06 2024-09-17T22:41:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397166+0000", 2024-09-17T22:41:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.824e-06 2024-09-17T22:41:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397169+0000", 2024-09-17T22:41:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2540000000000001e-06 2024-09-17T22:41:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146176 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.405086+0000", 2024-09-17T22:41:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001613256, 2024-09-17T22:41:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0051964710000000002, 2024-09-17T22:41:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146176 2024-09-17T22:41:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405086+0000", 2024-09-17T22:41:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405086+0000", 2024-09-17T22:41:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405086+0000", 2024-09-17T22:41:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000002e-07 2024-09-17T22:41:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405089+0000", 2024-09-17T22:41:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3080000000000002e-06 2024-09-17T22:41:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405096+0000", 2024-09-17T22:41:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1250000000000004e-06 2024-09-17T22:41:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405101+0000", 2024-09-17T22:41:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1440000000000002e-06 2024-09-17T22:41:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145732 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.359214+0000", 2024-09-17T22:41:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.047485154000000002, 2024-09-17T22:41:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051118834000000002, 2024-09-17T22:41:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145732 2024-09-17T22:41:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.359214+0000", 2024-09-17T22:41:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.359214+0000", 2024-09-17T22:41:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.359415+0000", 2024-09-17T22:41:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000201218 2024-09-17T22:41:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.359420+0000", 2024-09-17T22:41:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7879999999999997e-06 2024-09-17T22:41:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.359429+0000", 2024-09-17T22:41:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5189999999999992e-06 2024-09-17T22:41:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.359434+0000", 2024-09-17T22:41:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9119999999999997e-06 2024-09-17T22:41:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145797 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.367214+0000", 2024-09-17T22:41:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.039484770000000002, 2024-09-17T22:41:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043169805999999998, 2024-09-17T22:41:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145797 2024-09-17T22:41:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367214+0000", 2024-09-17T22:41:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367214+0000", 2024-09-17T22:41:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367215+0000", 2024-09-17T22:41:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4700000000000002e-07 2024-09-17T22:41:49.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367217+0000", 2024-09-17T22:41:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2919999999999998e-06 2024-09-17T22:41:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367226+0000", 2024-09-17T22:41:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6219999999999998e-06 2024-09-17T22:41:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367231+0000", 2024-09-17T22:41:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.899e-06 2024-09-17T22:41:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145859 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.373179+0000", 2024-09-17T22:41:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.033520354000000002, 2024-09-17T22:41:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037253258999999997, 2024-09-17T22:41:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.691 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145859 2024-09-17T22:41:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373179+0000", 2024-09-17T22:41:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373179+0000", 2024-09-17T22:41:49.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373179+0000", 2024-09-17T22:41:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-09-17T22:41:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373181+0000", 2024-09-17T22:41:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0320000000000002e-06 2024-09-17T22:41:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373189+0000", 2024-09-17T22:41:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2559999999999996e-06 2024-09-17T22:41:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373193+0000", 2024-09-17T22:41:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6439999999999999e-06 2024-09-17T22:41:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145925 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.694 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.379369+0000", 2024-09-17T22:41:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.027330061999999999, 2024-09-17T22:41:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031113536000000001, 2024-09-17T22:41:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145925 2024-09-17T22:41:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379369+0000", 2024-09-17T22:41:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379369+0000", 2024-09-17T22:41:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379370+0000", 2024-09-17T22:41:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3799999999999998e-07 2024-09-17T22:41:49.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379377+0000", 2024-09-17T22:41:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9299999999999997e-06 2024-09-17T22:41:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379385+0000", 2024-09-17T22:41:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0120000000000005e-06 2024-09-17T22:41:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379390+0000", 2024-09-17T22:41:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9790000000000001e-06 2024-09-17T22:41:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145986 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.385254+0000", 2024-09-17T22:41:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.021445281, 2024-09-17T22:41:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025278525, 2024-09-17T22:41:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.698 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145986 2024-09-17T22:41:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385254+0000", 2024-09-17T22:41:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385254+0000", 2024-09-17T22:41:49.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385254+0000", 2024-09-17T22:41:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.08e-07 2024-09-17T22:41:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385256+0000", 2024-09-17T22:41:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.978e-06 2024-09-17T22:41:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385262+0000", 2024-09-17T22:41:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8860000000000001e-06 2024-09-17T22:41:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385267+0000", 2024-09-17T22:41:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8239999999999999e-06 2024-09-17T22:41:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146051 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.391209+0000", 2024-09-17T22:41:49.701 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015490461000000001, 2024-09-17T22:41:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01937527, 2024-09-17T22:41:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146051 2024-09-17T22:41:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391209+0000", 2024-09-17T22:41:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391209+0000", 2024-09-17T22:41:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391209+0000", 2024-09-17T22:41:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2599999999999998e-07 2024-09-17T22:41:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391211+0000", 2024-09-17T22:41:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-09-17T22:41:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391217+0000", 2024-09-17T22:41:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9830000000000001e-06 2024-09-17T22:41:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391223+0000", 2024-09-17T22:41:49.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4840000000000003e-06 2024-09-17T22:41:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146115 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.397190+0000", 2024-09-17T22:41:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0095089400000000004, 2024-09-17T22:41:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013442264000000001, 2024-09-17T22:41:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146115 2024-09-17T22:41:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397190+0000", 2024-09-17T22:41:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397190+0000", 2024-09-17T22:41:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397191+0000", 2024-09-17T22:41:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9599999999999999e-07 2024-09-17T22:41:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397198+0000", 2024-09-17T22:41:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3329999999999999e-06 2024-09-17T22:41:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397206+0000", 2024-09-17T22:41:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8720000000000002e-06 2024-09-17T22:41:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397212+0000", 2024-09-17T22:41:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5430000000000001e-06 2024-09-17T22:41:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146178 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.405108+0000", 2024-09-17T22:41:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0015912330000000001, 2024-09-17T22:41:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005573461, 2024-09-17T22:41:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.708 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146178 2024-09-17T22:41:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405108+0000", 2024-09-17T22:41:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405108+0000", 2024-09-17T22:41:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405108+0000", 2024-09-17T22:41:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9599999999999999e-07 2024-09-17T22:41:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405111+0000", 2024-09-17T22:41:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1900000000000002e-06 2024-09-17T22:41:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405117+0000", 2024-09-17T22:41:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0140000000000004e-06 2024-09-17T22:41:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405121+0000", 2024-09-17T22:41:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.746e-06 2024-09-17T22:41:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145734 3.c 3:36b7e074:::rbd_data.115f14f2d1d5.0000000000000001:e [read 3358720~2048 out=2048b] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.359454+0000", 2024-09-17T22:41:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.047245370000000002, 2024-09-17T22:41:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051283911000000001, 2024-09-17T22:41:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145734 2024-09-17T22:41:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.359454+0000", 2024-09-17T22:41:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.359454+0000", 2024-09-17T22:41:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.359658+0000", 2024-09-17T22:41:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020456200000000001 2024-09-17T22:41:49.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.359663+0000", 2024-09-17T22:41:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.685e-06 2024-09-17T22:41:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.359676+0000", 2024-09-17T22:41:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2444e-05 2024-09-17T22:41:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.359685+0000", 2024-09-17T22:41:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1789999999999997e-06 2024-09-17T22:41:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.410610+0000", 2024-09-17T22:41:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050925318999999997 2024-09-17T22:41:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.410674+0000", 2024-09-17T22:41:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4288000000000005e-05 2024-09-17T22:41:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145799 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.367237+0000", 2024-09-17T22:41:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.039462418999999999, 2024-09-17T22:41:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043565315, 2024-09-17T22:41:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.716 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145799 2024-09-17T22:41:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367237+0000", 2024-09-17T22:41:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367237+0000", 2024-09-17T22:41:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367237+0000", 2024-09-17T22:41:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6600000000000002e-07 2024-09-17T22:41:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367239+0000", 2024-09-17T22:41:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.12e-06 2024-09-17T22:41:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367246+0000", 2024-09-17T22:41:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2299999999999996e-06 2024-09-17T22:41:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367250+0000", 2024-09-17T22:41:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999998e-06 2024-09-17T22:41:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145861 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.373200+0000", 2024-09-17T22:41:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.033499441999999997, 2024-09-17T22:41:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037654178000000003, 2024-09-17T22:41:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.719 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145861 2024-09-17T22:41:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373200+0000", 2024-09-17T22:41:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373200+0000", 2024-09-17T22:41:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373200+0000", 2024-09-17T22:41:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4999999999999998e-07 2024-09-17T22:41:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373202+0000", 2024-09-17T22:41:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1660000000000001e-06 2024-09-17T22:41:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373209+0000", 2024-09-17T22:41:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0190000000000001e-06 2024-09-17T22:41:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373214+0000", 2024-09-17T22:41:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8450000000000002e-06 2024-09-17T22:41:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145928 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.379399+0000", 2024-09-17T22:41:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.027300492999999999, 2024-09-17T22:41:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031513811000000003, 2024-09-17T22:41:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145928 2024-09-17T22:41:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379399+0000", 2024-09-17T22:41:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379399+0000", 2024-09-17T22:41:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379399+0000", 2024-09-17T22:41:49.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9999999999999998e-07 2024-09-17T22:41:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379402+0000", 2024-09-17T22:41:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3190000000000002e-06 2024-09-17T22:41:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379410+0000", 2024-09-17T22:41:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0439999999999996e-06 2024-09-17T22:41:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379414+0000", 2024-09-17T22:41:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8810000000000004e-06 2024-09-17T22:41:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145988 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.385273+0000", 2024-09-17T22:41:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.021426126, 2024-09-17T22:41:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02568894, 2024-09-17T22:41:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.726 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145988 2024-09-17T22:41:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385273+0000", 2024-09-17T22:41:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385273+0000", 2024-09-17T22:41:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385274+0000", 2024-09-17T22:41:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.15e-07 2024-09-17T22:41:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385276+0000", 2024-09-17T22:41:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0870000000000001e-06 2024-09-17T22:41:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385281+0000", 2024-09-17T22:41:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7780000000000004e-06 2024-09-17T22:41:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385286+0000", 2024-09-17T22:41:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4209999999999997e-06 2024-09-17T22:41:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.729 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146053 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.391250+0000", 2024-09-17T22:41:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015448739, 2024-09-17T22:41:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019763563000000001, 2024-09-17T22:41:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.730 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146053 2024-09-17T22:41:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391250+0000", 2024-09-17T22:41:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391250+0000", 2024-09-17T22:41:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391251+0000", 2024-09-17T22:41:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-09-17T22:41:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391255+0000", 2024-09-17T22:41:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9670000000000003e-06 2024-09-17T22:41:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391267+0000", 2024-09-17T22:41:49.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2089999999999999e-05 2024-09-17T22:41:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391275+0000", 2024-09-17T22:41:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.712e-06 2024-09-17T22:41:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146117 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.397222+0000", 2024-09-17T22:41:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00947718, 2024-09-17T22:41:49.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013844594999999999, 2024-09-17T22:41:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146117 2024-09-17T22:41:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397222+0000", 2024-09-17T22:41:49.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.397222+0000", 2024-09-17T22:41:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399051+0000", 2024-09-17T22:41:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018286680000000001 2024-09-17T22:41:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399055+0000", 2024-09-17T22:41:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3919999999999996e-06 2024-09-17T22:41:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399066+0000", 2024-09-17T22:41:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0798000000000001e-05 2024-09-17T22:41:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399074+0000", 2024-09-17T22:41:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6680000000000001e-06 2024-09-17T22:41:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146180 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.405128+0000", 2024-09-17T22:41:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0015717210000000001, 2024-09-17T22:41:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059936429999999999, 2024-09-17T22:41:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146180 2024-09-17T22:41:49.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.738 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405128+0000", 2024-09-17T22:41:49.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405128+0000", 2024-09-17T22:41:49.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405128+0000", 2024-09-17T22:41:49.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.77e-07 2024-09-17T22:41:49.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405130+0000", 2024-09-17T22:41:49.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0700000000000001e-06 2024-09-17T22:41:49.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405135+0000", 2024-09-17T22:41:49.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5169999999999999e-06 2024-09-17T22:41:49.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405141+0000", 2024-09-17T22:41:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9290000000000003e-06 2024-09-17T22:41:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145801 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.367288+0000", 2024-09-17T22:41:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.039411705999999998, 2024-09-17T22:41:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043864101000000003, 2024-09-17T22:41:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.740 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.741 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.741 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.741 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.741 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.741 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145801 2024-09-17T22:41:49.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.741 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367288+0000", 2024-09-17T22:41:49.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367288+0000", 2024-09-17T22:41:49.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367288+0000", 2024-09-17T22:41:49.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7999999999999995e-07 2024-09-17T22:41:49.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367291+0000", 2024-09-17T22:41:49.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9050000000000001e-06 2024-09-17T22:41:49.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367303+0000", 2024-09-17T22:41:49.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2199e-05 2024-09-17T22:41:49.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367311+0000", 2024-09-17T22:41:49.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8909999999999995e-06 2024-09-17T22:41:49.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.743 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.744 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145863 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.744 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.373221+0000", 2024-09-17T22:41:49.744 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.033478292, 2024-09-17T22:41:49.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037959690999999997, 2024-09-17T22:41:49.744 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.744 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145863 2024-09-17T22:41:49.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.744 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373221+0000", 2024-09-17T22:41:49.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373221+0000", 2024-09-17T22:41:49.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373221+0000", 2024-09-17T22:41:49.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6899999999999998e-07 2024-09-17T22:41:49.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373224+0000", 2024-09-17T22:41:49.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2689999999999998e-06 2024-09-17T22:41:49.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373231+0000", 2024-09-17T22:41:49.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7870000000000002e-06 2024-09-17T22:41:49.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373236+0000", 2024-09-17T22:41:49.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8709999999999996e-06 2024-09-17T22:41:49.747 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.747 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.747 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.747 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145930 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.747 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.379443+0000", 2024-09-17T22:41:49.747 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.027256135000000001, 2024-09-17T22:41:49.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031764433000000002, 2024-09-17T22:41:49.747 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.747 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.747 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.747 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.747 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.748 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.748 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145930 2024-09-17T22:41:49.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.748 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379443+0000", 2024-09-17T22:41:49.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379443+0000", 2024-09-17T22:41:49.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379444+0000", 2024-09-17T22:41:49.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0500000000000004e-07 2024-09-17T22:41:49.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379446+0000", 2024-09-17T22:41:49.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6010000000000002e-06 2024-09-17T22:41:49.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379454+0000", 2024-09-17T22:41:49.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5390000000000002e-06 2024-09-17T22:41:49.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379460+0000", 2024-09-17T22:41:49.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7799999999999997e-06 2024-09-17T22:41:49.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.750 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.750 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145990 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.750 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.385292+0000", 2024-09-17T22:41:49.750 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.02140744, 2024-09-17T22:41:49.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025956690000000001, 2024-09-17T22:41:49.751 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.751 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.751 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.751 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145990 2024-09-17T22:41:49.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.751 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385292+0000", 2024-09-17T22:41:49.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385292+0000", 2024-09-17T22:41:49.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385292+0000", 2024-09-17T22:41:49.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-09-17T22:41:49.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385294+0000", 2024-09-17T22:41:49.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1090000000000001e-06 2024-09-17T22:41:49.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385300+0000", 2024-09-17T22:41:49.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4720000000000003e-06 2024-09-17T22:41:49.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385304+0000", 2024-09-17T22:41:49.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5789999999999997e-06 2024-09-17T22:41:49.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.754 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.754 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.754 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146055 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.754 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.391286+0000", 2024-09-17T22:41:49.754 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015412883, 2024-09-17T22:41:49.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019996064000000001, 2024-09-17T22:41:49.754 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.754 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.754 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.754 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146055 2024-09-17T22:41:49.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.755 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391286+0000", 2024-09-17T22:41:49.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391286+0000", 2024-09-17T22:41:49.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391287+0000", 2024-09-17T22:41:49.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4000000000000002e-07 2024-09-17T22:41:49.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391289+0000", 2024-09-17T22:41:49.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4389999999999999e-06 2024-09-17T22:41:49.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391297+0000", 2024-09-17T22:41:49.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4649999999999997e-06 2024-09-17T22:41:49.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391302+0000", 2024-09-17T22:41:49.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6110000000000001e-06 2024-09-17T22:41:49.757 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.757 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.757 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.757 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146119 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.757 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.399087+0000", 2024-09-17T22:41:49.758 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0076117620000000002, 2024-09-17T22:41:49.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012221509, 2024-09-17T22:41:49.758 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.758 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.758 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.758 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.758 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.758 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.758 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146119 2024-09-17T22:41:49.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.758 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399087+0000", 2024-09-17T22:41:49.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399087+0000", 2024-09-17T22:41:49.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399088+0000", 2024-09-17T22:41:49.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7100000000000002e-07 2024-09-17T22:41:49.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399091+0000", 2024-09-17T22:41:49.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5739999999999998e-06 2024-09-17T22:41:49.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399098+0000", 2024-09-17T22:41:49.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6040000000000004e-06 2024-09-17T22:41:49.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399105+0000", 2024-09-17T22:41:49.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8920000000000003e-06 2024-09-17T22:41:49.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.760 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.761 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.761 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146182 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.761 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.405149+0000", 2024-09-17T22:41:49.761 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0015502510000000001, 2024-09-17T22:41:49.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061898109999999999, 2024-09-17T22:41:49.761 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.761 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.761 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.762 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.762 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146182 2024-09-17T22:41:49.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.762 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405149+0000", 2024-09-17T22:41:49.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405149+0000", 2024-09-17T22:41:49.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405149+0000", 2024-09-17T22:41:49.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0699999999999998e-07 2024-09-17T22:41:49.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405152+0000", 2024-09-17T22:41:49.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1720000000000001e-06 2024-09-17T22:41:49.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405159+0000", 2024-09-17T22:41:49.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1969999999999999e-06 2024-09-17T22:41:49.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405164+0000", 2024-09-17T22:41:49.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8579999999999999e-06 2024-09-17T22:41:49.764 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.764 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.764 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.765 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145803 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.765 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.367320+0000", 2024-09-17T22:41:49.765 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.039379619999999997, 2024-09-17T22:41:49.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044046531999999999, 2024-09-17T22:41:49.765 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.765 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.765 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.765 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.765 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.765 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.765 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145803 2024-09-17T22:41:49.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.765 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367320+0000", 2024-09-17T22:41:49.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367320+0000", 2024-09-17T22:41:49.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367320+0000", 2024-09-17T22:41:49.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9999999999999998e-07 2024-09-17T22:41:49.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367322+0000", 2024-09-17T22:41:49.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3690000000000001e-06 2024-09-17T22:41:49.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367331+0000", 2024-09-17T22:41:49.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3850000000000002e-06 2024-09-17T22:41:49.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367336+0000", 2024-09-17T22:41:49.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3789999999999997e-06 2024-09-17T22:41:49.768 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.768 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.768 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.768 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145865 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.768 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.373259+0000", 2024-09-17T22:41:49.768 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.033440154999999999, 2024-09-17T22:41:49.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038135446000000003, 2024-09-17T22:41:49.768 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.769 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.769 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.769 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.769 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.769 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.769 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145865 2024-09-17T22:41:49.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.769 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373259+0000", 2024-09-17T22:41:49.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373259+0000", 2024-09-17T22:41:49.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373259+0000", 2024-09-17T22:41:49.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8700000000000001e-07 2024-09-17T22:41:49.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373262+0000", 2024-09-17T22:41:49.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.48e-06 2024-09-17T22:41:49.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373268+0000", 2024-09-17T22:41:49.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8830000000000003e-06 2024-09-17T22:41:49.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373273+0000", 2024-09-17T22:41:49.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8160000000000001e-06 2024-09-17T22:41:49.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.772 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.772 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.772 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145932 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.772 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.379468+0000", 2024-09-17T22:41:49.772 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.027231333999999999, 2024-09-17T22:41:49.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031959113999999997, 2024-09-17T22:41:49.772 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.772 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.772 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.773 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145932 2024-09-17T22:41:49.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.773 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379468+0000", 2024-09-17T22:41:49.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379468+0000", 2024-09-17T22:41:49.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379468+0000", 2024-09-17T22:41:49.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.58e-07 2024-09-17T22:41:49.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379471+0000", 2024-09-17T22:41:49.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2529999999999999e-06 2024-09-17T22:41:49.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379479+0000", 2024-09-17T22:41:49.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2330000000000006e-06 2024-09-17T22:41:49.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379484+0000", 2024-09-17T22:41:49.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7280000000000003e-06 2024-09-17T22:41:49.775 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.775 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.775 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.775 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145992 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.776 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.385312+0000", 2024-09-17T22:41:49.776 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.021387000999999999, 2024-09-17T22:41:49.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026153566, 2024-09-17T22:41:49.776 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.776 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.776 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.776 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.776 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.776 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.776 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145992 2024-09-17T22:41:49.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.776 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385312+0000", 2024-09-17T22:41:49.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385312+0000", 2024-09-17T22:41:49.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385313+0000", 2024-09-17T22:41:49.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6899999999999997e-07 2024-09-17T22:41:49.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385315+0000", 2024-09-17T22:41:49.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3070000000000001e-06 2024-09-17T22:41:49.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385321+0000", 2024-09-17T22:41:49.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.908e-06 2024-09-17T22:41:49.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385326+0000", 2024-09-17T22:41:49.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0660000000000003e-06 2024-09-17T22:41:49.778 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.778 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.779 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.779 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146057 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.779 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.391310+0000", 2024-09-17T22:41:49.779 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015389686, 2024-09-17T22:41:49.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020183838999999999, 2024-09-17T22:41:49.779 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.779 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.779 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.779 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.779 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.779 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.780 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146057 2024-09-17T22:41:49.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.780 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391310+0000", 2024-09-17T22:41:49.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391310+0000", 2024-09-17T22:41:49.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391310+0000", 2024-09-17T22:41:49.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5900000000000002e-07 2024-09-17T22:41:49.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391312+0000", 2024-09-17T22:41:49.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1169999999999998e-06 2024-09-17T22:41:49.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391319+0000", 2024-09-17T22:41:49.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4119999999999998e-06 2024-09-17T22:41:49.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391324+0000", 2024-09-17T22:41:49.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4199999999999998e-06 2024-09-17T22:41:49.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.782 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.782 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146121 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.782 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.399114+0000", 2024-09-17T22:41:49.782 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0075856500000000002, 2024-09-17T22:41:49.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012408015999999999, 2024-09-17T22:41:49.783 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.783 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.783 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.783 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146121 2024-09-17T22:41:49.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.783 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399114+0000", 2024-09-17T22:41:49.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399114+0000", 2024-09-17T22:41:49.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399114+0000", 2024-09-17T22:41:49.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:41:49.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399116+0000", 2024-09-17T22:41:49.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0729999999999999e-06 2024-09-17T22:41:49.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399125+0000", 2024-09-17T22:41:49.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2890000000000003e-06 2024-09-17T22:41:49.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399185+0000", 2024-09-17T22:41:49.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9919e-05 2024-09-17T22:41:49.785 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.785 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.785 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.786 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146184 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.786 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.405176+0000", 2024-09-17T22:41:49.786 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001523265, 2024-09-17T22:41:49.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063724539999999996, 2024-09-17T22:41:49.786 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.786 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.786 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.786 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.786 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.786 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.786 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146184 2024-09-17T22:41:49.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.786 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405176+0000", 2024-09-17T22:41:49.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405176+0000", 2024-09-17T22:41:49.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405176+0000", 2024-09-17T22:41:49.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:41:49.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405178+0000", 2024-09-17T22:41:49.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0619999999999999e-06 2024-09-17T22:41:49.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405184+0000", 2024-09-17T22:41:49.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6749999999999999e-06 2024-09-17T22:41:49.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405189+0000", 2024-09-17T22:41:49.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0370000000000002e-06 2024-09-17T22:41:49.789 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.789 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.789 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.789 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145805 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.789 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.367343+0000", 2024-09-17T22:41:49.789 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.039356365999999997, 2024-09-17T22:41:49.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044236270000000001, 2024-09-17T22:41:49.789 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.789 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.789 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.790 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145805 2024-09-17T22:41:49.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.790 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367343+0000", 2024-09-17T22:41:49.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367343+0000", 2024-09-17T22:41:49.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367343+0000", 2024-09-17T22:41:49.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2e-07 2024-09-17T22:41:49.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367350+0000", 2024-09-17T22:41:49.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9419999999999998e-06 2024-09-17T22:41:49.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367358+0000", 2024-09-17T22:41:49.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9710000000000004e-06 2024-09-17T22:41:49.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367363+0000", 2024-09-17T22:41:49.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1200000000000001e-06 2024-09-17T22:41:49.792 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.792 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.792 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.792 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145867 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.793 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.373279+0000", 2024-09-17T22:41:49.793 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.033420023, 2024-09-17T22:41:49.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038326928000000003, 2024-09-17T22:41:49.793 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.793 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.793 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.793 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.793 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.793 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.793 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145867 2024-09-17T22:41:49.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.793 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373279+0000", 2024-09-17T22:41:49.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373279+0000", 2024-09-17T22:41:49.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373280+0000", 2024-09-17T22:41:49.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-09-17T22:41:49.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373281+0000", 2024-09-17T22:41:49.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9099999999999999e-06 2024-09-17T22:41:49.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373290+0000", 2024-09-17T22:41:49.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4970000000000001e-06 2024-09-17T22:41:49.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373299+0000", 2024-09-17T22:41:49.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6230000000000003e-06 2024-09-17T22:41:49.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.796 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.796 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145934 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.796 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.379490+0000", 2024-09-17T22:41:49.796 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.027209193, 2024-09-17T22:41:49.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032144526999999999, 2024-09-17T22:41:49.796 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.796 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.797 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.797 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.797 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.797 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.797 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145934 2024-09-17T22:41:49.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.797 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379490+0000", 2024-09-17T22:41:49.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379490+0000", 2024-09-17T22:41:49.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379490+0000", 2024-09-17T22:41:49.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000002e-07 2024-09-17T22:41:49.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379493+0000", 2024-09-17T22:41:49.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1739999999999999e-06 2024-09-17T22:41:49.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379498+0000", 2024-09-17T22:41:49.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6879999999999996e-06 2024-09-17T22:41:49.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379503+0000", 2024-09-17T22:41:49.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.532e-06 2024-09-17T22:41:49.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.799 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.799 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145994 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.800 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.385333+0000", 2024-09-17T22:41:49.800 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.021366296, 2024-09-17T22:41:49.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026331457999999999, 2024-09-17T22:41:49.800 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.800 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.800 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.800 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145994 2024-09-17T22:41:49.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.800 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385333+0000", 2024-09-17T22:41:49.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385333+0000", 2024-09-17T22:41:49.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385333+0000", 2024-09-17T22:41:49.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-09-17T22:41:49.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385336+0000", 2024-09-17T22:41:49.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3309999999999998e-06 2024-09-17T22:41:49.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385342+0000", 2024-09-17T22:41:49.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1489999999999999e-06 2024-09-17T22:41:49.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385347+0000", 2024-09-17T22:41:49.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8899999999999998e-06 2024-09-17T22:41:49.802 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.803 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.803 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146059 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.803 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.391331+0000", 2024-09-17T22:41:49.803 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.015368359999999999, 2024-09-17T22:41:49.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020372922000000002, 2024-09-17T22:41:49.803 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.803 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.804 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.804 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146059 2024-09-17T22:41:49.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.804 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391331+0000", 2024-09-17T22:41:49.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.391331+0000", 2024-09-17T22:41:49.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393057+0000", 2024-09-17T22:41:49.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001726134 2024-09-17T22:41:49.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393062+0000", 2024-09-17T22:41:49.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5970000000000002e-06 2024-09-17T22:41:49.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393071+0000", 2024-09-17T22:41:49.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4690000000000003e-06 2024-09-17T22:41:49.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393078+0000", 2024-09-17T22:41:49.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8009999999999998e-06 2024-09-17T22:41:49.806 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.806 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.806 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.806 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146123 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.807 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.399200+0000", 2024-09-17T22:41:49.807 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0074993999999999998, 2024-09-17T22:41:49.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012531944999999999, 2024-09-17T22:41:49.807 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.807 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.807 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.807 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.807 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.807 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.807 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146123 2024-09-17T22:41:49.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.807 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399200+0000", 2024-09-17T22:41:49.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399200+0000", 2024-09-17T22:41:49.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399200+0000", 2024-09-17T22:41:49.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8699999999999995e-07 2024-09-17T22:41:49.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399204+0000", 2024-09-17T22:41:49.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.303e-06 2024-09-17T22:41:49.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399213+0000", 2024-09-17T22:41:49.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9639999999999992e-06 2024-09-17T22:41:49.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399233+0000", 2024-09-17T22:41:49.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0764e-05 2024-09-17T22:41:49.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.809 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.810 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146186 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.810 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.405197+0000", 2024-09-17T22:41:49.810 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0015023289999999999, 2024-09-17T22:41:49.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0065875789999999997, 2024-09-17T22:41:49.810 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.810 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.810 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.810 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146186 2024-09-17T22:41:49.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.810 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405197+0000", 2024-09-17T22:41:49.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405197+0000", 2024-09-17T22:41:49.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405197+0000", 2024-09-17T22:41:49.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:41:49.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405224+0000", 2024-09-17T22:41:49.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7178999999999998e-05 2024-09-17T22:41:49.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405233+0000", 2024-09-17T22:41:49.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.473e-06 2024-09-17T22:41:49.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405242+0000", 2024-09-17T22:41:49.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8659999999999995e-06 2024-09-17T22:41:49.813 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.813 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.813 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.813 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145807 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.813 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.367371+0000", 2024-09-17T22:41:49.813 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.039328504, 2024-09-17T22:41:49.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044466355999999999, 2024-09-17T22:41:49.813 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.813 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.813 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.814 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.814 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.814 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.814 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145807 2024-09-17T22:41:49.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.814 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367371+0000", 2024-09-17T22:41:49.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.367371+0000", 2024-09-17T22:41:49.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369072+0000", 2024-09-17T22:41:49.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017008329999999999 2024-09-17T22:41:49.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369076+0000", 2024-09-17T22:41:49.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-09-17T22:41:49.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369090+0000", 2024-09-17T22:41:49.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3390999999999999e-05 2024-09-17T22:41:49.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369099+0000", 2024-09-17T22:41:49.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4239999999999999e-06 2024-09-17T22:41:49.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.816 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.816 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145869 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.817 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.373304+0000", 2024-09-17T22:41:49.817 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.033394854000000002, 2024-09-17T22:41:49.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038571853000000003, 2024-09-17T22:41:49.817 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.817 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.817 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.817 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145869 2024-09-17T22:41:49.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.817 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373304+0000", 2024-09-17T22:41:49.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.373304+0000", 2024-09-17T22:41:49.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375050+0000", 2024-09-17T22:41:49.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017452469999999999 2024-09-17T22:41:49.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375053+0000", 2024-09-17T22:41:49.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6200000000000001e-06 2024-09-17T22:41:49.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375061+0000", 2024-09-17T22:41:49.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5270000000000001e-06 2024-09-17T22:41:49.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375065+0000", 2024-09-17T22:41:49.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5430000000000004e-06 2024-09-17T22:41:49.820 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.820 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.820 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.820 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145935 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.820 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.379510+0000", 2024-09-17T22:41:49.820 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.027189722999999999, 2024-09-17T22:41:49.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032402372999999998, 2024-09-17T22:41:49.820 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.820 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.820 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.820 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.821 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145935 2024-09-17T22:41:49.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.821 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379510+0000", 2024-09-17T22:41:49.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379510+0000", 2024-09-17T22:41:49.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379510+0000", 2024-09-17T22:41:49.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:41:49.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379512+0000", 2024-09-17T22:41:49.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1299999999999999e-06 2024-09-17T22:41:49.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379518+0000", 2024-09-17T22:41:49.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7300000000000002e-06 2024-09-17T22:41:49.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379522+0000", 2024-09-17T22:41:49.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3819999999999997e-06 2024-09-17T22:41:49.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.823 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.824 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145996 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.824 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.385356+0000", 2024-09-17T22:41:49.824 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.021342976999999999, 2024-09-17T22:41:49.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026582631999999998, 2024-09-17T22:41:49.824 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.824 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.824 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.824 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.824 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.824 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.824 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145996 2024-09-17T22:41:49.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.824 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385356+0000", 2024-09-17T22:41:49.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.385356+0000", 2024-09-17T22:41:49.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387051+0000", 2024-09-17T22:41:49.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016952270000000001 2024-09-17T22:41:49.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387056+0000", 2024-09-17T22:41:49.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0350000000000003e-06 2024-09-17T22:41:49.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387063+0000", 2024-09-17T22:41:49.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8390000000000007e-06 2024-09-17T22:41:49.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387068+0000", 2024-09-17T22:41:49.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5990000000000004e-06 2024-09-17T22:41:49.827 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.827 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.827 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.827 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146061 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.827 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.393090+0000", 2024-09-17T22:41:49.827 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.013609234, 2024-09-17T22:41:49.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018877244000000001, 2024-09-17T22:41:49.827 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.827 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.828 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.828 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.828 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.828 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.828 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146061 2024-09-17T22:41:49.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.828 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393090+0000", 2024-09-17T22:41:49.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393090+0000", 2024-09-17T22:41:49.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393090+0000", 2024-09-17T22:41:49.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-09-17T22:41:49.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393093+0000", 2024-09-17T22:41:49.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5289999999999998e-06 2024-09-17T22:41:49.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393099+0000", 2024-09-17T22:41:49.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2049999999999999e-06 2024-09-17T22:41:49.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393104+0000", 2024-09-17T22:41:49.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6859999999999997e-06 2024-09-17T22:41:49.830 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.831 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.831 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146125 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.831 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.399250+0000", 2024-09-17T22:41:49.831 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0074491669999999996, 2024-09-17T22:41:49.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.012743877000000001, 2024-09-17T22:41:49.831 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.831 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.831 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.831 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.831 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.831 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.832 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146125 2024-09-17T22:41:49.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.832 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399250+0000", 2024-09-17T22:41:49.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399250+0000", 2024-09-17T22:41:49.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399251+0000", 2024-09-17T22:41:49.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.92e-07 2024-09-17T22:41:49.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399255+0000", 2024-09-17T22:41:49.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0090000000000001e-06 2024-09-17T22:41:49.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399264+0000", 2024-09-17T22:41:49.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1460000000000002e-06 2024-09-17T22:41:49.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399271+0000", 2024-09-17T22:41:49.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6800000000000004e-06 2024-09-17T22:41:49.834 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.834 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.834 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.834 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146188 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.834 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.405250+0000", 2024-09-17T22:41:49.834 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0014489749999999999, 2024-09-17T22:41:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006771895, 2024-09-17T22:41:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146188 2024-09-17T22:41:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405250+0000", 2024-09-17T22:41:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405250+0000", 2024-09-17T22:41:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405251+0000", 2024-09-17T22:41:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1100000000000002e-07 2024-09-17T22:41:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405252+0000", 2024-09-17T22:41:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6640000000000001e-06 2024-09-17T22:41:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405257+0000", 2024-09-17T22:41:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9919999999999998e-06 2024-09-17T22:41:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405260+0000", 2024-09-17T22:41:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.993e-06 2024-09-17T22:41:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145809 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.369116+0000", 2024-09-17T22:41:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.037583660999999997, 2024-09-17T22:41:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042940836000000003, 2024-09-17T22:41:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145809 2024-09-17T22:41:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369116+0000", 2024-09-17T22:41:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369116+0000", 2024-09-17T22:41:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369116+0000", 2024-09-17T22:41:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9599999999999999e-07 2024-09-17T22:41:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369119+0000", 2024-09-17T22:41:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.841e-06 2024-09-17T22:41:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369127+0000", 2024-09-17T22:41:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3399999999999998e-06 2024-09-17T22:41:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369133+0000", 2024-09-17T22:41:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8159999999999999e-06 2024-09-17T22:41:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145871 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.375075+0000", 2024-09-17T22:41:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.031624526, 2024-09-17T22:41:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037010203999999998, 2024-09-17T22:41:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145871 2024-09-17T22:41:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375075+0000", 2024-09-17T22:41:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375075+0000", 2024-09-17T22:41:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375075+0000", 2024-09-17T22:41:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:41:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375077+0000", 2024-09-17T22:41:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.531e-06 2024-09-17T22:41:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375080+0000", 2024-09-17T22:41:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4740000000000001e-06 2024-09-17T22:41:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375083+0000", 2024-09-17T22:41:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.807e-06 2024-09-17T22:41:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145937 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.379528+0000", 2024-09-17T22:41:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.027171560000000001, 2024-09-17T22:41:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032584287000000003, 2024-09-17T22:41:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145937 2024-09-17T22:41:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379528+0000", 2024-09-17T22:41:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.379528+0000", 2024-09-17T22:41:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381055+0000", 2024-09-17T22:41:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001527797 2024-09-17T22:41:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381060+0000", 2024-09-17T22:41:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4059999999999998e-06 2024-09-17T22:41:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381071+0000", 2024-09-17T22:41:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1228e-05 2024-09-17T22:41:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381079+0000", 2024-09-17T22:41:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5410000000000003e-06 2024-09-17T22:41:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145998 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.387077+0000", 2024-09-17T22:41:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019621756000000001, 2024-09-17T22:41:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025085521999999999, 2024-09-17T22:41:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145998 2024-09-17T22:41:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387077+0000", 2024-09-17T22:41:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387077+0000", 2024-09-17T22:41:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387078+0000", 2024-09-17T22:41:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-17T22:41:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387079+0000", 2024-09-17T22:41:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5659999999999999e-06 2024-09-17T22:41:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387083+0000", 2024-09-17T22:41:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3730000000000001e-06 2024-09-17T22:41:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387086+0000", 2024-09-17T22:41:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.835e-06 2024-09-17T22:41:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146063 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.393125+0000", 2024-09-17T22:41:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.013574437, 2024-09-17T22:41:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019088193, 2024-09-17T22:41:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146063 2024-09-17T22:41:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393125+0000", 2024-09-17T22:41:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393125+0000", 2024-09-17T22:41:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393125+0000", 2024-09-17T22:41:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9299999999999999e-07 2024-09-17T22:41:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393127+0000", 2024-09-17T22:41:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9939999999999999e-06 2024-09-17T22:41:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393131+0000", 2024-09-17T22:41:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9509999999999999e-06 2024-09-17T22:41:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393135+0000", 2024-09-17T22:41:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.569e-06 2024-09-17T22:41:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146129 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.399279+0000", 2024-09-17T22:41:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0074200910000000002, 2024-09-17T22:41:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01298828, 2024-09-17T22:41:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146129 2024-09-17T22:41:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399279+0000", 2024-09-17T22:41:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399279+0000", 2024-09-17T22:41:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399280+0000", 2024-09-17T22:41:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4899999999999995e-07 2024-09-17T22:41:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399282+0000", 2024-09-17T22:41:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6249999999999999e-06 2024-09-17T22:41:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399289+0000", 2024-09-17T22:41:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6959999999999999e-06 2024-09-17T22:41:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399294+0000", 2024-09-17T22:41:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9980000000000003e-06 2024-09-17T22:41:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146190 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.405264+0000", 2024-09-17T22:41:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001435174, 2024-09-17T22:41:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0070515480000000004, 2024-09-17T22:41:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146190 2024-09-17T22:41:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405264+0000", 2024-09-17T22:41:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405264+0000", 2024-09-17T22:41:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407048+0000", 2024-09-17T22:41:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017842750000000001 2024-09-17T22:41:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407053+0000", 2024-09-17T22:41:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2390000000000004e-06 2024-09-17T22:41:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407062+0000", 2024-09-17T22:41:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9159999999999996e-06 2024-09-17T22:41:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407068+0000", 2024-09-17T22:41:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0459999999999996e-06 2024-09-17T22:41:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145811 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.369142+0000", 2024-09-17T22:41:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.037556751999999999, 2024-09-17T22:41:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04323105, 2024-09-17T22:41:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145811 2024-09-17T22:41:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369142+0000", 2024-09-17T22:41:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369142+0000", 2024-09-17T22:41:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369143+0000", 2024-09-17T22:41:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1799999999999995e-07 2024-09-17T22:41:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369145+0000", 2024-09-17T22:41:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.446e-06 2024-09-17T22:41:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369152+0000", 2024-09-17T22:41:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6749999999999996e-06 2024-09-17T22:41:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369158+0000", 2024-09-17T22:41:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1670000000000004e-06 2024-09-17T22:41:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145874 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.375087+0000", 2024-09-17T22:41:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.031612574999999997, 2024-09-17T22:41:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037335712, 2024-09-17T22:41:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145874 2024-09-17T22:41:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375087+0000", 2024-09-17T22:41:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375087+0000", 2024-09-17T22:41:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375087+0000", 2024-09-17T22:41:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1e-07 2024-09-17T22:41:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375088+0000", 2024-09-17T22:41:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2190000000000001e-06 2024-09-17T22:41:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375091+0000", 2024-09-17T22:41:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.275e-06 2024-09-17T22:41:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375094+0000", 2024-09-17T22:41:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7149999999999998e-06 2024-09-17T22:41:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145939 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.381092+0000", 2024-09-17T22:41:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.025607681, 2024-09-17T22:41:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031382027999999999, 2024-09-17T22:41:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145939 2024-09-17T22:41:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381092+0000", 2024-09-17T22:41:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381092+0000", 2024-09-17T22:41:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381092+0000", 2024-09-17T22:41:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1900000000000002e-07 2024-09-17T22:41:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381095+0000", 2024-09-17T22:41:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8229999999999999e-06 2024-09-17T22:41:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381109+0000", 2024-09-17T22:41:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3927999999999999e-05 2024-09-17T22:41:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381115+0000", 2024-09-17T22:41:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5729999999999996e-06 2024-09-17T22:41:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146000 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.387090+0000", 2024-09-17T22:41:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019609465999999999, 2024-09-17T22:41:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025435531000000001, 2024-09-17T22:41:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146000 2024-09-17T22:41:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387090+0000", 2024-09-17T22:41:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387090+0000", 2024-09-17T22:41:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387090+0000", 2024-09-17T22:41:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3200000000000001e-07 2024-09-17T22:41:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387091+0000", 2024-09-17T22:41:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2899999999999999e-06 2024-09-17T22:41:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387094+0000", 2024-09-17T22:41:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9979999999999999e-06 2024-09-17T22:41:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387097+0000", 2024-09-17T22:41:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.632e-06 2024-09-17T22:41:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146065 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.393138+0000", 2024-09-17T22:41:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.013560747, 2024-09-17T22:41:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019436568000000001, 2024-09-17T22:41:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146065 2024-09-17T22:41:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393138+0000", 2024-09-17T22:41:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393138+0000", 2024-09-17T22:41:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393139+0000", 2024-09-17T22:41:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.28e-07 2024-09-17T22:41:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393140+0000", 2024-09-17T22:41:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2309999999999999e-06 2024-09-17T22:41:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393143+0000", 2024-09-17T22:41:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.213e-06 2024-09-17T22:41:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393146+0000", 2024-09-17T22:41:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7e-06 2024-09-17T22:41:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146131 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.399301+0000", 2024-09-17T22:41:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0073980210000000003, 2024-09-17T22:41:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013324519999999999, 2024-09-17T22:41:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146131 2024-09-17T22:41:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399301+0000", 2024-09-17T22:41:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399301+0000", 2024-09-17T22:41:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399302+0000", 2024-09-17T22:41:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-09-17T22:41:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399304+0000", 2024-09-17T22:41:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3499999999999999e-06 2024-09-17T22:41:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399311+0000", 2024-09-17T22:41:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7649999999999996e-06 2024-09-17T22:41:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399316+0000", 2024-09-17T22:41:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0139999999999998e-06 2024-09-17T22:41:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146192 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.407077+0000", 2024-09-17T22:41:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9996223, 2024-09-17T22:41:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0055980980000000001, 2024-09-17T22:41:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146192 2024-09-17T22:41:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407077+0000", 2024-09-17T22:41:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407077+0000", 2024-09-17T22:41:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407077+0000", 2024-09-17T22:41:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3500000000000002e-07 2024-09-17T22:41:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407079+0000", 2024-09-17T22:41:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5060000000000001e-06 2024-09-17T22:41:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407083+0000", 2024-09-17T22:41:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.704e-06 2024-09-17T22:41:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407086+0000", 2024-09-17T22:41:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9040000000000001e-06 2024-09-17T22:41:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145813 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.369166+0000", 2024-09-17T22:41:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.037532928, 2024-09-17T22:41:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043560394000000002, 2024-09-17T22:41:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145813 2024-09-17T22:41:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369166+0000", 2024-09-17T22:41:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369166+0000", 2024-09-17T22:41:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369167+0000", 2024-09-17T22:41:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:41:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369169+0000", 2024-09-17T22:41:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4669999999999998e-06 2024-09-17T22:41:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369176+0000", 2024-09-17T22:41:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7710000000000001e-06 2024-09-17T22:41:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369181+0000", 2024-09-17T22:41:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0989999999999998e-06 2024-09-17T22:41:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145876 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.375097+0000", 2024-09-17T22:41:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.031601917, 2024-09-17T22:41:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037678281000000001, 2024-09-17T22:41:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145876 2024-09-17T22:41:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375097+0000", 2024-09-17T22:41:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375097+0000", 2024-09-17T22:41:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375098+0000", 2024-09-17T22:41:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0200000000000001e-07 2024-09-17T22:41:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375102+0000", 2024-09-17T22:41:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.284e-06 2024-09-17T22:41:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375105+0000", 2024-09-17T22:41:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2559999999999998e-06 2024-09-17T22:41:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375108+0000", 2024-09-17T22:41:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7539999999999998e-06 2024-09-17T22:41:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145941 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.381124+0000", 2024-09-17T22:41:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.025575414000000001, 2024-09-17T22:41:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031708966999999998, 2024-09-17T22:41:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145941 2024-09-17T22:41:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381124+0000", 2024-09-17T22:41:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381124+0000", 2024-09-17T22:41:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381124+0000", 2024-09-17T22:41:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9999999999999998e-07 2024-09-17T22:41:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381127+0000", 2024-09-17T22:41:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3700000000000002e-06 2024-09-17T22:41:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381134+0000", 2024-09-17T22:41:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0929999999999997e-06 2024-09-17T22:41:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381140+0000", 2024-09-17T22:41:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1079999999999998e-06 2024-09-17T22:41:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146002 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.387100+0000", 2024-09-17T22:41:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019599174, 2024-09-17T22:41:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025782643000000001, 2024-09-17T22:41:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146002 2024-09-17T22:41:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387100+0000", 2024-09-17T22:41:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387100+0000", 2024-09-17T22:41:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387100+0000", 2024-09-17T22:41:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0699999999999999e-07 2024-09-17T22:41:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387101+0000", 2024-09-17T22:41:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2330000000000001e-06 2024-09-17T22:41:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387104+0000", 2024-09-17T22:41:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9699999999999999e-06 2024-09-17T22:41:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387107+0000", 2024-09-17T22:41:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7080000000000002e-06 2024-09-17T22:41:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146067 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.393149+0000", 2024-09-17T22:41:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.013550370000000001, 2024-09-17T22:41:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019785659000000001, 2024-09-17T22:41:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146067 2024-09-17T22:41:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393149+0000", 2024-09-17T22:41:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393149+0000", 2024-09-17T22:41:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393149+0000", 2024-09-17T22:41:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:41:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393154+0000", 2024-09-17T22:41:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6439999999999999e-06 2024-09-17T22:41:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393157+0000", 2024-09-17T22:41:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2839999999999998e-06 2024-09-17T22:41:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393161+0000", 2024-09-17T22:41:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.411e-06 2024-09-17T22:41:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146133 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.399322+0000", 2024-09-17T22:41:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0073770340000000002, 2024-09-17T22:41:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013661491, 2024-09-17T22:41:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146133 2024-09-17T22:41:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399322+0000", 2024-09-17T22:41:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399322+0000", 2024-09-17T22:41:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399323+0000", 2024-09-17T22:41:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8799999999999998e-07 2024-09-17T22:41:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399325+0000", 2024-09-17T22:41:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:41:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399330+0000", 2024-09-17T22:41:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-09-17T22:41:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399335+0000", 2024-09-17T22:41:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.34e-06 2024-09-17T22:41:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146194 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.407089+0000", 2024-09-17T22:41:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9996099, 2024-09-17T22:41:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060686589999999997, 2024-09-17T22:41:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146194 2024-09-17T22:41:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407089+0000", 2024-09-17T22:41:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407089+0000", 2024-09-17T22:41:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407089+0000", 2024-09-17T22:41:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2700000000000001e-07 2024-09-17T22:41:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407091+0000", 2024-09-17T22:41:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.04e-06 2024-09-17T22:41:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407094+0000", 2024-09-17T22:41:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6119999999999999e-06 2024-09-17T22:41:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407099+0000", 2024-09-17T22:41:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6700000000000002e-06 2024-09-17T22:41:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145815 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.369189+0000", 2024-09-17T22:41:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.037509967999999998, 2024-09-17T22:41:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043999187000000002, 2024-09-17T22:41:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145815 2024-09-17T22:41:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369189+0000", 2024-09-17T22:41:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369189+0000", 2024-09-17T22:41:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369190+0000", 2024-09-17T22:41:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.01e-07 2024-09-17T22:41:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369192+0000", 2024-09-17T22:41:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9819999999999998e-06 2024-09-17T22:41:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369197+0000", 2024-09-17T22:41:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7989999999999999e-06 2024-09-17T22:41:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369202+0000", 2024-09-17T22:41:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.023e-06 2024-09-17T22:41:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145878 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.375111+0000", 2024-09-17T22:41:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.031587918, 2024-09-17T22:41:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038109922999999997, 2024-09-17T22:41:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145878 2024-09-17T22:41:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375111+0000", 2024-09-17T22:41:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375111+0000", 2024-09-17T22:41:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375112+0000", 2024-09-17T22:41:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3300000000000001e-07 2024-09-17T22:41:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375113+0000", 2024-09-17T22:41:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0550000000000001e-06 2024-09-17T22:41:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375116+0000", 2024-09-17T22:41:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9909999999999998e-06 2024-09-17T22:41:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375119+0000", 2024-09-17T22:41:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.915e-06 2024-09-17T22:41:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145943 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.381147+0000", 2024-09-17T22:41:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.025552242999999999, 2024-09-17T22:41:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032122782000000003, 2024-09-17T22:41:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145943 2024-09-17T22:41:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381147+0000", 2024-09-17T22:41:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381147+0000", 2024-09-17T22:41:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381147+0000", 2024-09-17T22:41:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:41:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381150+0000", 2024-09-17T22:41:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1710000000000001e-06 2024-09-17T22:41:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381156+0000", 2024-09-17T22:41:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6290000000000004e-06 2024-09-17T22:41:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381161+0000", 2024-09-17T22:41:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2689999999999999e-06 2024-09-17T22:41:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146004 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.387110+0000", 2024-09-17T22:41:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019589024999999999, 2024-09-17T22:41:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026191662000000001, 2024-09-17T22:41:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146004 2024-09-17T22:41:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387110+0000", 2024-09-17T22:41:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387110+0000", 2024-09-17T22:41:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387110+0000", 2024-09-17T22:41:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0699999999999999e-07 2024-09-17T22:41:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387112+0000", 2024-09-17T22:41:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2049999999999999e-06 2024-09-17T22:41:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387115+0000", 2024-09-17T22:41:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8959999999999999e-06 2024-09-17T22:41:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387117+0000", 2024-09-17T22:41:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.514e-06 2024-09-17T22:41:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146069 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.393164+0000", 2024-09-17T22:41:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.013535289000000001, 2024-09-17T22:41:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020178353, 2024-09-17T22:41:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146069 2024-09-17T22:41:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393164+0000", 2024-09-17T22:41:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393164+0000", 2024-09-17T22:41:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393164+0000", 2024-09-17T22:41:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099999999999999e-07 2024-09-17T22:41:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393165+0000", 2024-09-17T22:41:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.167e-06 2024-09-17T22:41:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393168+0000", 2024-09-17T22:41:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0319999999999999e-06 2024-09-17T22:41:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393171+0000", 2024-09-17T22:41:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.57e-06 2024-09-17T22:41:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146135 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.399340+0000", 2024-09-17T22:41:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.007359132, 2024-09-17T22:41:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014033654, 2024-09-17T22:41:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146135 2024-09-17T22:41:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399340+0000", 2024-09-17T22:41:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399340+0000", 2024-09-17T22:41:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399340+0000", 2024-09-17T22:41:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2099999999999998e-07 2024-09-17T22:41:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399342+0000", 2024-09-17T22:41:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0169999999999999e-06 2024-09-17T22:41:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399349+0000", 2024-09-17T22:41:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-09-17T22:41:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399353+0000", 2024-09-17T22:41:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6120000000000001e-06 2024-09-17T22:41:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146196 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.407105+0000", 2024-09-17T22:41:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9995947, 2024-09-17T22:41:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062991590000000004, 2024-09-17T22:41:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146196 2024-09-17T22:41:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407105+0000", 2024-09-17T22:41:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407105+0000", 2024-09-17T22:41:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407105+0000", 2024-09-17T22:41:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3799999999999998e-07 2024-09-17T22:41:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407107+0000", 2024-09-17T22:41:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-09-17T22:41:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407113+0000", 2024-09-17T22:41:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7470000000000002e-06 2024-09-17T22:41:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407117+0000", 2024-09-17T22:41:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4469999999999999e-06 2024-09-17T22:41:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145817 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.369209+0000", 2024-09-17T22:41:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.037490318000000002, 2024-09-17T22:41:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04422338, 2024-09-17T22:41:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145817 2024-09-17T22:41:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369209+0000", 2024-09-17T22:41:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369209+0000", 2024-09-17T22:41:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369209+0000", 2024-09-17T22:41:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.84e-07 2024-09-17T22:41:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369211+0000", 2024-09-17T22:41:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1739999999999999e-06 2024-09-17T22:41:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369218+0000", 2024-09-17T22:41:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-09-17T22:41:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369222+0000", 2024-09-17T22:41:49.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8319999999999996e-06 2024-09-17T22:41:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145880 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.375139+0000", 2024-09-17T22:41:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.031560715000000003, 2024-09-17T22:41:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038324256000000001, 2024-09-17T22:41:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.938 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145880 2024-09-17T22:41:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375139+0000", 2024-09-17T22:41:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375139+0000", 2024-09-17T22:41:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375139+0000", 2024-09-17T22:41:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6400000000000002e-07 2024-09-17T22:41:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375141+0000", 2024-09-17T22:41:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.221e-06 2024-09-17T22:41:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375147+0000", 2024-09-17T22:41:49.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7529999999999998e-06 2024-09-17T22:41:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375152+0000", 2024-09-17T22:41:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7180000000000004e-06 2024-09-17T22:41:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145945 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.381168+0000", 2024-09-17T22:41:49.941 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.025531170999999998, 2024-09-17T22:41:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032337402000000001, 2024-09-17T22:41:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145945 2024-09-17T22:41:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381168+0000", 2024-09-17T22:41:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381168+0000", 2024-09-17T22:41:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381168+0000", 2024-09-17T22:41:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:41:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381170+0000", 2024-09-17T22:41:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0360000000000001e-06 2024-09-17T22:41:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381177+0000", 2024-09-17T22:41:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4330000000000001e-06 2024-09-17T22:41:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381184+0000", 2024-09-17T22:41:49.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6340000000000003e-06 2024-09-17T22:41:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146006 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.387120+0000", 2024-09-17T22:41:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019579374, 2024-09-17T22:41:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026438581999999999, 2024-09-17T22:41:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146006 2024-09-17T22:41:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387120+0000", 2024-09-17T22:41:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387120+0000", 2024-09-17T22:41:49.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387120+0000", 2024-09-17T22:41:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0100000000000001e-07 2024-09-17T22:41:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387121+0000", 2024-09-17T22:41:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.221e-06 2024-09-17T22:41:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387124+0000", 2024-09-17T22:41:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9189999999999999e-06 2024-09-17T22:41:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387127+0000", 2024-09-17T22:41:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5280000000000002e-06 2024-09-17T22:41:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146071 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.948 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.393174+0000", 2024-09-17T22:41:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.013525381, 2024-09-17T22:41:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020438202999999999, 2024-09-17T22:41:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146071 2024-09-17T22:41:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393174+0000", 2024-09-17T22:41:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393174+0000", 2024-09-17T22:41:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393174+0000", 2024-09-17T22:41:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.16e-07 2024-09-17T22:41:49.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393175+0000", 2024-09-17T22:41:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0470000000000001e-06 2024-09-17T22:41:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393178+0000", 2024-09-17T22:41:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9459999999999998e-06 2024-09-17T22:41:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393181+0000", 2024-09-17T22:41:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7819999999999998e-06 2024-09-17T22:41:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146137 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.399360+0000", 2024-09-17T22:41:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0073395450000000003, 2024-09-17T22:41:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014283272, 2024-09-17T22:41:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.952 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146137 2024-09-17T22:41:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399360+0000", 2024-09-17T22:41:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399360+0000", 2024-09-17T22:41:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399360+0000", 2024-09-17T22:41:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3099999999999998e-07 2024-09-17T22:41:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399362+0000", 2024-09-17T22:41:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0169999999999999e-06 2024-09-17T22:41:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399368+0000", 2024-09-17T22:41:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9229999999999999e-06 2024-09-17T22:41:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399372+0000", 2024-09-17T22:41:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3270000000000002e-06 2024-09-17T22:41:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.955 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146198 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.407124+0000", 2024-09-17T22:41:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9995751, 2024-09-17T22:41:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0065464420000000004, 2024-09-17T22:41:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146198 2024-09-17T22:41:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407124+0000", 2024-09-17T22:41:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407124+0000", 2024-09-17T22:41:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407124+0000", 2024-09-17T22:41:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:41:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407127+0000", 2024-09-17T22:41:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2129999999999998e-06 2024-09-17T22:41:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407133+0000", 2024-09-17T22:41:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0440000000000001e-06 2024-09-17T22:41:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407137+0000", 2024-09-17T22:41:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6260000000000003e-06 2024-09-17T22:41:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.958 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145819 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.369229+0000", 2024-09-17T22:41:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.037470112999999999, 2024-09-17T22:41:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044487553999999999, 2024-09-17T22:41:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.959 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145819 2024-09-17T22:41:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369229+0000", 2024-09-17T22:41:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369229+0000", 2024-09-17T22:41:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369230+0000", 2024-09-17T22:41:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:41:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369232+0000", 2024-09-17T22:41:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2589999999999999e-06 2024-09-17T22:41:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369251+0000", 2024-09-17T22:41:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9145999999999999e-05 2024-09-17T22:41:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369257+0000", 2024-09-17T22:41:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3110000000000002e-06 2024-09-17T22:41:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145882 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.375158+0000", 2024-09-17T22:41:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.031541451999999998, 2024-09-17T22:41:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038592787000000003, 2024-09-17T22:41:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.962 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145882 2024-09-17T22:41:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375158+0000", 2024-09-17T22:41:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375158+0000", 2024-09-17T22:41:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375158+0000", 2024-09-17T22:41:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1e-07 2024-09-17T22:41:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375160+0000", 2024-09-17T22:41:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9520000000000001e-06 2024-09-17T22:41:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375165+0000", 2024-09-17T22:41:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3140000000000003e-06 2024-09-17T22:41:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375170+0000", 2024-09-17T22:41:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4349999999999999e-06 2024-09-17T22:41:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.965 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145947 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.381190+0000", 2024-09-17T22:41:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.025509489, 2024-09-17T22:41:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032589789000000001, 2024-09-17T22:41:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145947 2024-09-17T22:41:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381190+0000", 2024-09-17T22:41:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381190+0000", 2024-09-17T22:41:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381190+0000", 2024-09-17T22:41:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9799999999999999e-07 2024-09-17T22:41:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381192+0000", 2024-09-17T22:41:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0190000000000001e-06 2024-09-17T22:41:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381205+0000", 2024-09-17T22:41:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.271e-05 2024-09-17T22:41:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381211+0000", 2024-09-17T22:41:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6860000000000003e-06 2024-09-17T22:41:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.968 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146008 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.387130+0000", 2024-09-17T22:41:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019569607999999999, 2024-09-17T22:41:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026676831000000002, 2024-09-17T22:41:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.969 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146008 2024-09-17T22:41:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387130+0000", 2024-09-17T22:41:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387130+0000", 2024-09-17T22:41:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387130+0000", 2024-09-17T22:41:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-07 2024-09-17T22:41:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387131+0000", 2024-09-17T22:41:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.167e-06 2024-09-17T22:41:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387134+0000", 2024-09-17T22:41:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2119999999999999e-06 2024-09-17T22:41:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387138+0000", 2024-09-17T22:41:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9600000000000002e-06 2024-09-17T22:41:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146073 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.393184+0000", 2024-09-17T22:41:49.972 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.01351542, 2024-09-17T22:41:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020650895999999998, 2024-09-17T22:41:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146073 2024-09-17T22:41:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393184+0000", 2024-09-17T22:41:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393184+0000", 2024-09-17T22:41:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393184+0000", 2024-09-17T22:41:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-09-17T22:41:49.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393185+0000", 2024-09-17T22:41:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9999999999999995e-07 2024-09-17T22:41:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393189+0000", 2024-09-17T22:41:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5889999999999999e-06 2024-09-17T22:41:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393193+0000", 2024-09-17T22:41:49.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2390000000000004e-06 2024-09-17T22:41:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146139 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.399378+0000", 2024-09-17T22:41:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0073212420000000004, 2024-09-17T22:41:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014483197, 2024-09-17T22:41:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.976 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146139 2024-09-17T22:41:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399378+0000", 2024-09-17T22:41:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.399378+0000", 2024-09-17T22:41:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401050+0000", 2024-09-17T22:41:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00167161 2024-09-17T22:41:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401054+0000", 2024-09-17T22:41:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8609999999999997e-06 2024-09-17T22:41:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401067+0000", 2024-09-17T22:41:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2119e-05 2024-09-17T22:41:49.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401074+0000", 2024-09-17T22:41:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5909999999999998e-06 2024-09-17T22:41:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146200 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.407144+0000", 2024-09-17T22:41:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9995551, 2024-09-17T22:41:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006754716, 2024-09-17T22:41:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.979 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146200 2024-09-17T22:41:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407144+0000", 2024-09-17T22:41:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407144+0000", 2024-09-17T22:41:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407144+0000", 2024-09-17T22:41:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1199999999999998e-07 2024-09-17T22:41:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407146+0000", 2024-09-17T22:41:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1040000000000001e-06 2024-09-17T22:41:49.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407151+0000", 2024-09-17T22:41:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4950000000000002e-06 2024-09-17T22:41:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407154+0000", 2024-09-17T22:41:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9799999999999998e-06 2024-09-17T22:41:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145821 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.369265+0000", 2024-09-17T22:41:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.037434368000000003, 2024-09-17T22:41:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044679178999999999, 2024-09-17T22:41:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145821 2024-09-17T22:41:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.983 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369265+0000", 2024-09-17T22:41:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369265+0000", 2024-09-17T22:41:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369265+0000", 2024-09-17T22:41:49.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6800000000000001e-07 2024-09-17T22:41:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369268+0000", 2024-09-17T22:41:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5450000000000002e-06 2024-09-17T22:41:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369275+0000", 2024-09-17T22:41:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1199999999999996e-06 2024-09-17T22:41:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369282+0000", 2024-09-17T22:41:49.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.635e-06 2024-09-17T22:41:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145884 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.375176+0000", 2024-09-17T22:41:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.031523506999999999, 2024-09-17T22:41:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038821103000000003, 2024-09-17T22:41:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.986 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145884 2024-09-17T22:41:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375176+0000", 2024-09-17T22:41:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375176+0000", 2024-09-17T22:41:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375176+0000", 2024-09-17T22:41:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9200000000000002e-07 2024-09-17T22:41:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375178+0000", 2024-09-17T22:41:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.824e-06 2024-09-17T22:41:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375184+0000", 2024-09-17T22:41:49.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6899999999999997e-06 2024-09-17T22:41:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375188+0000", 2024-09-17T22:41:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6190000000000002e-06 2024-09-17T22:41:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145949 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.381217+0000", 2024-09-17T22:41:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.025482121999999999, 2024-09-17T22:41:49.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.032823815999999999, 2024-09-17T22:41:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145949 2024-09-17T22:41:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381217+0000", 2024-09-17T22:41:49.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381217+0000", 2024-09-17T22:41:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381218+0000", 2024-09-17T22:41:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3099999999999998e-07 2024-09-17T22:41:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381224+0000", 2024-09-17T22:41:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4169999999999997e-06 2024-09-17T22:41:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381231+0000", 2024-09-17T22:41:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8839999999999997e-06 2024-09-17T22:41:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381236+0000", 2024-09-17T22:41:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3059999999999997e-06 2024-09-17T22:41:49.992 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146010 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.387162+0000", 2024-09-17T22:41:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019537266000000001, 2024-09-17T22:41:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026925426999999998, 2024-09-17T22:41:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.993 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146010 2024-09-17T22:41:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387162+0000", 2024-09-17T22:41:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387162+0000", 2024-09-17T22:41:49.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387162+0000", 2024-09-17T22:41:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.39e-07 2024-09-17T22:41:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387165+0000", 2024-09-17T22:41:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.289e-06 2024-09-17T22:41:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387171+0000", 2024-09-17T22:41:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9089999999999997e-06 2024-09-17T22:41:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387175+0000", 2024-09-17T22:41:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6269999999999999e-06 2024-09-17T22:41:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.996 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146075 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.393197+0000", 2024-09-17T22:41:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.013502605000000001, 2024-09-17T22:41:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020945993999999999, 2024-09-17T22:41:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146075 2024-09-17T22:41:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393197+0000", 2024-09-17T22:41:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393197+0000", 2024-09-17T22:41:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393197+0000", 2024-09-17T22:41:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3699999999999999e-07 2024-09-17T22:41:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393198+0000", 2024-09-17T22:41:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1659999999999999e-06 2024-09-17T22:41:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393202+0000", 2024-09-17T22:41:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-06 2024-09-17T22:41:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393206+0000", 2024-09-17T22:41:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3749999999999996e-06 2024-09-17T22:41:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:49.999 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146141 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.401086+0000", 2024-09-17T22:41:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0056128180000000003, 2024-09-17T22:41:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013110787, 2024-09-17T22:41:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146141 2024-09-17T22:41:50.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401086+0000", 2024-09-17T22:41:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401086+0000", 2024-09-17T22:41:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401087+0000", 2024-09-17T22:41:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9200000000000001e-07 2024-09-17T22:41:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401094+0000", 2024-09-17T22:41:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7109999999999998e-06 2024-09-17T22:41:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401101+0000", 2024-09-17T22:41:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.007e-06 2024-09-17T22:41:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401106+0000", 2024-09-17T22:41:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2630000000000003e-06 2024-09-17T22:41:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146202 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.407158+0000", 2024-09-17T22:41:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9995413, 2024-09-17T22:41:50.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0070789750000000004, 2024-09-17T22:41:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146202 2024-09-17T22:41:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407158+0000", 2024-09-17T22:41:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407158+0000", 2024-09-17T22:41:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407158+0000", 2024-09-17T22:41:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4900000000000002e-07 2024-09-17T22:41:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407159+0000", 2024-09-17T22:41:50.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4500000000000001e-06 2024-09-17T22:41:50.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407163+0000", 2024-09-17T22:41:50.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7579999999999998e-06 2024-09-17T22:41:50.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407167+0000", 2024-09-17T22:41:50.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1799999999999998e-06 2024-09-17T22:41:50.006 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.006 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.006 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.007 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145823 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.007 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.369288+0000", 2024-09-17T22:41:50.007 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.037410954000000003, 2024-09-17T22:41:50.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044978971999999999, 2024-09-17T22:41:50.007 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.007 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.007 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.007 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145823 2024-09-17T22:41:50.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.007 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369288+0000", 2024-09-17T22:41:50.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369288+0000", 2024-09-17T22:41:50.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369289+0000", 2024-09-17T22:41:50.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000002e-07 2024-09-17T22:41:50.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369291+0000", 2024-09-17T22:41:50.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1600000000000001e-06 2024-09-17T22:41:50.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369297+0000", 2024-09-17T22:41:50.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4629999999999998e-06 2024-09-17T22:41:50.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369302+0000", 2024-09-17T22:41:50.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9250000000000003e-06 2024-09-17T22:41:50.009 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.009 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.010 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.010 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145886 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.010 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.375195+0000", 2024-09-17T22:41:50.010 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.031504646999999997, 2024-09-17T22:41:50.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039125882000000001, 2024-09-17T22:41:50.010 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.010 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.010 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.010 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.010 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.010 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.010 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145886 2024-09-17T22:41:50.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.011 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375195+0000", 2024-09-17T22:41:50.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.375195+0000", 2024-09-17T22:41:50.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377052+0000", 2024-09-17T22:41:50.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018577579999999999 2024-09-17T22:41:50.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377057+0000", 2024-09-17T22:41:50.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8300000000000003e-06 2024-09-17T22:41:50.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377070+0000", 2024-09-17T22:41:50.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3074000000000001e-05 2024-09-17T22:41:50.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377079+0000", 2024-09-17T22:41:50.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5259999999999993e-06 2024-09-17T22:41:50.013 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.013 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.013 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.013 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145951 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.013 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.381244+0000", 2024-09-17T22:41:50.013 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.025455701000000001, 2024-09-17T22:41:50.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033125927999999999, 2024-09-17T22:41:50.013 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.013 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.014 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.014 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.014 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.014 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.014 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145951 2024-09-17T22:41:50.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.014 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381244+0000", 2024-09-17T22:41:50.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381244+0000", 2024-09-17T22:41:50.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381244+0000", 2024-09-17T22:41:50.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8700000000000001e-07 2024-09-17T22:41:50.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381246+0000", 2024-09-17T22:41:50.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2879999999999999e-06 2024-09-17T22:41:50.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381255+0000", 2024-09-17T22:41:50.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4519999999999997e-06 2024-09-17T22:41:50.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381260+0000", 2024-09-17T22:41:50.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4219999999999999e-06 2024-09-17T22:41:50.016 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.016 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.017 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.017 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146012 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.017 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.387182+0000", 2024-09-17T22:41:50.017 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.019517183, 2024-09-17T22:41:50.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027243566, 2024-09-17T22:41:50.017 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.017 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.017 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.017 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.017 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.018 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.018 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146012 2024-09-17T22:41:50.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.018 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387182+0000", 2024-09-17T22:41:50.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.387182+0000", 2024-09-17T22:41:50.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389161+0000", 2024-09-17T22:41:50.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019785649999999998 2024-09-17T22:41:50.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389165+0000", 2024-09-17T22:41:50.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4780000000000002e-06 2024-09-17T22:41:50.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389176+0000", 2024-09-17T22:41:50.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1382999999999999e-05 2024-09-17T22:41:50.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389184+0000", 2024-09-17T22:41:50.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3259999999999998e-06 2024-09-17T22:41:50.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.020 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.020 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146077 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.020 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.393212+0000", 2024-09-17T22:41:50.020 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.013487307, 2024-09-17T22:41:50.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021267352999999999, 2024-09-17T22:41:50.021 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.021 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.021 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.021 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146077 2024-09-17T22:41:50.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.021 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393212+0000", 2024-09-17T22:41:50.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393212+0000", 2024-09-17T22:41:50.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393212+0000", 2024-09-17T22:41:50.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:41:50.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393214+0000", 2024-09-17T22:41:50.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9230000000000001e-06 2024-09-17T22:41:50.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393221+0000", 2024-09-17T22:41:50.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4119999999999998e-06 2024-09-17T22:41:50.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393226+0000", 2024-09-17T22:41:50.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0549999999999999e-06 2024-09-17T22:41:50.023 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.024 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.024 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.024 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146143 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.024 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.401113+0000", 2024-09-17T22:41:50.024 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0055866980000000002, 2024-09-17T22:41:50.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013422709999999999, 2024-09-17T22:41:50.024 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.024 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.024 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.024 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.024 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.024 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.025 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146143 2024-09-17T22:41:50.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.025 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401113+0000", 2024-09-17T22:41:50.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401113+0000", 2024-09-17T22:41:50.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401113+0000", 2024-09-17T22:41:50.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.65e-07 2024-09-17T22:41:50.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401115+0000", 2024-09-17T22:41:50.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8980000000000001e-06 2024-09-17T22:41:50.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401121+0000", 2024-09-17T22:41:50.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0510000000000002e-06 2024-09-17T22:41:50.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401126+0000", 2024-09-17T22:41:50.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7260000000000002e-06 2024-09-17T22:41:50.027 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.027 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.027 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.027 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146204 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.027 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.407173+0000", 2024-09-17T22:41:50.028 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.999526, 2024-09-17T22:41:50.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0074168400000000001, 2024-09-17T22:41:50.028 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.028 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.028 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.028 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.028 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.028 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.028 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146204 2024-09-17T22:41:50.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.028 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407173+0000", 2024-09-17T22:41:50.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407173+0000", 2024-09-17T22:41:50.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407174+0000", 2024-09-17T22:41:50.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-09-17T22:41:50.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407176+0000", 2024-09-17T22:41:50.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8819999999999999e-06 2024-09-17T22:41:50.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407181+0000", 2024-09-17T22:41:50.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5140000000000001e-06 2024-09-17T22:41:50.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407186+0000", 2024-09-17T22:41:50.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.729e-06 2024-09-17T22:41:50.031 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.031 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.031 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.031 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145825 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.031 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.369309+0000", 2024-09-17T22:41:50.031 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.037390385999999998, 2024-09-17T22:41:50.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045339789999999998, 2024-09-17T22:41:50.031 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.031 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.032 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.032 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.032 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.032 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.032 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145825 2024-09-17T22:41:50.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.032 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369309+0000", 2024-09-17T22:41:50.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369309+0000", 2024-09-17T22:41:50.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369309+0000", 2024-09-17T22:41:50.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.46e-07 2024-09-17T22:41:50.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369312+0000", 2024-09-17T22:41:50.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2850000000000001e-06 2024-09-17T22:41:50.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369319+0000", 2024-09-17T22:41:50.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.357e-06 2024-09-17T22:41:50.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369324+0000", 2024-09-17T22:41:50.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4389999999999999e-06 2024-09-17T22:41:50.034 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.034 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.035 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.035 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145888 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.035 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.377092+0000", 2024-09-17T22:41:50.035 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.029607209999999998, 2024-09-17T22:41:50.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037610324000000001, 2024-09-17T22:41:50.035 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.035 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.035 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.036 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145888 2024-09-17T22:41:50.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.036 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377092+0000", 2024-09-17T22:41:50.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377092+0000", 2024-09-17T22:41:50.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377093+0000", 2024-09-17T22:41:50.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.82e-07 2024-09-17T22:41:50.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377095+0000", 2024-09-17T22:41:50.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.757e-06 2024-09-17T22:41:50.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377103+0000", 2024-09-17T22:41:50.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9510000000000006e-06 2024-09-17T22:41:50.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377109+0000", 2024-09-17T22:41:50.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7590000000000003e-06 2024-09-17T22:41:50.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.038 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.038 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145953 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.038 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.381267+0000", 2024-09-17T22:41:50.039 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.025432395999999999, 2024-09-17T22:41:50.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033492215999999998, 2024-09-17T22:41:50.039 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.039 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.039 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.039 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145953 2024-09-17T22:41:50.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.039 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381267+0000", 2024-09-17T22:41:50.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381267+0000", 2024-09-17T22:41:50.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381267+0000", 2024-09-17T22:41:50.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-09-17T22:41:50.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381269+0000", 2024-09-17T22:41:50.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0990000000000001e-06 2024-09-17T22:41:50.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381276+0000", 2024-09-17T22:41:50.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1409999999999999e-06 2024-09-17T22:41:50.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381282+0000", 2024-09-17T22:41:50.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1209999999999998e-06 2024-09-17T22:41:50.042 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.042 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.042 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.042 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146014 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.042 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.389197+0000", 2024-09-17T22:41:50.042 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.017502292999999999, 2024-09-17T22:41:50.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025616593999999999, 2024-09-17T22:41:50.043 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.043 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.043 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.043 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146014 2024-09-17T22:41:50.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.043 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389197+0000", 2024-09-17T22:41:50.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389197+0000", 2024-09-17T22:41:50.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389198+0000", 2024-09-17T22:41:50.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3099999999999997e-07 2024-09-17T22:41:50.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389200+0000", 2024-09-17T22:41:50.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5749999999999999e-06 2024-09-17T22:41:50.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389207+0000", 2024-09-17T22:41:50.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9639999999999997e-06 2024-09-17T22:41:50.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389212+0000", 2024-09-17T22:41:50.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3299999999999998e-06 2024-09-17T22:41:50.045 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.045 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.045 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.046 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146079 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.046 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.393231+0000", 2024-09-17T22:41:50.046 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.013467761999999999, 2024-09-17T22:41:50.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021636698999999999, 2024-09-17T22:41:50.046 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.046 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.046 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.046 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.046 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.046 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.046 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146079 2024-09-17T22:41:50.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.047 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393231+0000", 2024-09-17T22:41:50.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393231+0000", 2024-09-17T22:41:50.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393232+0000", 2024-09-17T22:41:50.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4200000000000002e-07 2024-09-17T22:41:50.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393234+0000", 2024-09-17T22:41:50.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0430000000000002e-06 2024-09-17T22:41:50.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393240+0000", 2024-09-17T22:41:50.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3999999999999997e-06 2024-09-17T22:41:50.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393247+0000", 2024-09-17T22:41:50.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3940000000000001e-06 2024-09-17T22:41:50.049 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.049 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.049 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.049 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146145 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.049 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.401131+0000", 2024-09-17T22:41:50.049 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0055680019999999998, 2024-09-17T22:41:50.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.013792857, 2024-09-17T22:41:50.049 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.049 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.050 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.050 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.050 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.050 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.050 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146145 2024-09-17T22:41:50.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.050 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401131+0000", 2024-09-17T22:41:50.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401131+0000", 2024-09-17T22:41:50.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401132+0000", 2024-09-17T22:41:50.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8599999999999999e-07 2024-09-17T22:41:50.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401133+0000", 2024-09-17T22:41:50.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8509999999999999e-06 2024-09-17T22:41:50.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401141+0000", 2024-09-17T22:41:50.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3329999999999999e-06 2024-09-17T22:41:50.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401146+0000", 2024-09-17T22:41:50.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0629999999999996e-06 2024-09-17T22:41:50.052 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.052 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.052 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.053 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146206 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.053 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.407192+0000", 2024-09-17T22:41:50.053 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.999507, 2024-09-17T22:41:50.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.007787431, 2024-09-17T22:41:50.053 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.053 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.053 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.053 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.053 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.053 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.053 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146206 2024-09-17T22:41:50.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.053 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407192+0000", 2024-09-17T22:41:50.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407192+0000", 2024-09-17T22:41:50.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407193+0000", 2024-09-17T22:41:50.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0499999999999999e-07 2024-09-17T22:41:50.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407195+0000", 2024-09-17T22:41:50.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.328e-06 2024-09-17T22:41:50.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407201+0000", 2024-09-17T22:41:50.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9399999999999999e-06 2024-09-17T22:41:50.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407205+0000", 2024-09-17T22:41:50.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5070000000000002e-06 2024-09-17T22:41:50.055 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.056 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.056 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145827 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.056 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.369349+0000", 2024-09-17T22:41:50.056 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.037350347999999998, 2024-09-17T22:41:50.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045688982000000003, 2024-09-17T22:41:50.056 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.056 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.056 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.056 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.056 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.057 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.057 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145827 2024-09-17T22:41:50.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.057 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369349+0000", 2024-09-17T22:41:50.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369349+0000", 2024-09-17T22:41:50.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369349+0000", 2024-09-17T22:41:50.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.37e-07 2024-09-17T22:41:50.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369352+0000", 2024-09-17T22:41:50.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5859999999999999e-06 2024-09-17T22:41:50.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369359+0000", 2024-09-17T22:41:50.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8279999999999997e-06 2024-09-17T22:41:50.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369364+0000", 2024-09-17T22:41:50.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3369999999999999e-06 2024-09-17T22:41:50.059 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.059 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.059 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.060 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145890 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.060 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.377118+0000", 2024-09-17T22:41:50.060 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.029581349999999999, 2024-09-17T22:41:50.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037977142999999998, 2024-09-17T22:41:50.060 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.060 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.060 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.060 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.060 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.060 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.060 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145890 2024-09-17T22:41:50.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.060 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377118+0000", 2024-09-17T22:41:50.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377118+0000", 2024-09-17T22:41:50.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377118+0000", 2024-09-17T22:41:50.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2899999999999999e-07 2024-09-17T22:41:50.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377121+0000", 2024-09-17T22:41:50.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3080000000000002e-06 2024-09-17T22:41:50.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377127+0000", 2024-09-17T22:41:50.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5370000000000003e-06 2024-09-17T22:41:50.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377133+0000", 2024-09-17T22:41:50.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0129999999999999e-06 2024-09-17T22:41:50.063 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.063 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.063 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.063 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146016 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.063 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.389220+0000", 2024-09-17T22:41:50.063 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.01747947, 2024-09-17T22:41:50.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025933624999999998, 2024-09-17T22:41:50.063 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.063 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.063 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.063 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.063 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.064 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.064 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146016 2024-09-17T22:41:50.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.064 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389220+0000", 2024-09-17T22:41:50.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389220+0000", 2024-09-17T22:41:50.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389220+0000", 2024-09-17T22:41:50.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3799999999999998e-07 2024-09-17T22:41:50.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389222+0000", 2024-09-17T22:41:50.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2910000000000002e-06 2024-09-17T22:41:50.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389229+0000", 2024-09-17T22:41:50.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2589999999999997e-06 2024-09-17T22:41:50.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389234+0000", 2024-09-17T22:41:50.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0300000000000001e-06 2024-09-17T22:41:50.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.066 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.066 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146081 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.066 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.393254+0000", 2024-09-17T22:41:50.067 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.013444820999999999, 2024-09-17T22:41:50.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021952752999999998, 2024-09-17T22:41:50.067 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.067 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.067 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.067 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.067 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.067 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.067 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146081 2024-09-17T22:41:50.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.067 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393254+0000", 2024-09-17T22:41:50.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.393254+0000", 2024-09-17T22:41:50.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395051+0000", 2024-09-17T22:41:50.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017961839999999999 2024-09-17T22:41:50.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395056+0000", 2024-09-17T22:41:50.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9089999999999999e-06 2024-09-17T22:41:50.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395067+0000", 2024-09-17T22:41:50.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1357e-05 2024-09-17T22:41:50.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395074+0000", 2024-09-17T22:41:50.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3969999999999997e-06 2024-09-17T22:41:50.069 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.069 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.069 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.070 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146146 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.070 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.401153+0000", 2024-09-17T22:41:50.070 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0055466480000000004, 2024-09-17T22:41:50.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014111487000000001, 2024-09-17T22:41:50.070 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.070 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.070 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.070 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.070 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.070 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.070 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146146 2024-09-17T22:41:50.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.070 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401153+0000", 2024-09-17T22:41:50.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401153+0000", 2024-09-17T22:41:50.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401153+0000", 2024-09-17T22:41:50.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8500000000000002e-07 2024-09-17T22:41:50.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401155+0000", 2024-09-17T22:41:50.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.204e-06 2024-09-17T22:41:50.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401162+0000", 2024-09-17T22:41:50.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5010000000000001e-06 2024-09-17T22:41:50.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401172+0000", 2024-09-17T22:41:50.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.815e-06 2024-09-17T22:41:50.073 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.073 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.073 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.073 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146208 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.073 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.407212+0000", 2024-09-17T22:41:50.073 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9994874, 2024-09-17T22:41:50.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.008106828, 2024-09-17T22:41:50.073 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.073 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.074 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.074 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.074 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.074 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.074 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146208 2024-09-17T22:41:50.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.074 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407212+0000", 2024-09-17T22:41:50.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407212+0000", 2024-09-17T22:41:50.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407212+0000", 2024-09-17T22:41:50.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9799999999999999e-07 2024-09-17T22:41:50.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407214+0000", 2024-09-17T22:41:50.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0609999999999998e-06 2024-09-17T22:41:50.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407220+0000", 2024-09-17T22:41:50.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8669999999999999e-06 2024-09-17T22:41:50.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407225+0000", 2024-09-17T22:41:50.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.583e-06 2024-09-17T22:41:50.076 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.076 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.076 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.077 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145829 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.077 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.369372+0000", 2024-09-17T22:41:50.077 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.037327050000000001, 2024-09-17T22:41:50.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046006208999999999, 2024-09-17T22:41:50.077 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.077 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.077 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.077 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.077 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.077 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.077 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145829 2024-09-17T22:41:50.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.078 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369372+0000", 2024-09-17T22:41:50.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369372+0000", 2024-09-17T22:41:50.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369373+0000", 2024-09-17T22:41:50.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-09-17T22:41:50.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369380+0000", 2024-09-17T22:41:50.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9e-06 2024-09-17T22:41:50.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369387+0000", 2024-09-17T22:41:50.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4200000000000001e-06 2024-09-17T22:41:50.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369392+0000", 2024-09-17T22:41:50.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7249999999999997e-06 2024-09-17T22:41:50.080 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.080 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.080 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.080 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145892 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.080 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.377140+0000", 2024-09-17T22:41:50.080 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.029558764000000001, 2024-09-17T22:41:50.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038291723, 2024-09-17T22:41:50.080 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.080 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.080 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.080 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.080 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.081 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.081 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145892 2024-09-17T22:41:50.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.081 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377140+0000", 2024-09-17T22:41:50.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377140+0000", 2024-09-17T22:41:50.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377141+0000", 2024-09-17T22:41:50.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1199999999999998e-07 2024-09-17T22:41:50.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377143+0000", 2024-09-17T22:41:50.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2510000000000001e-06 2024-09-17T22:41:50.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377152+0000", 2024-09-17T22:41:50.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5960000000000004e-06 2024-09-17T22:41:50.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377157+0000", 2024-09-17T22:41:50.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5140000000000001e-06 2024-09-17T22:41:50.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.083 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.083 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146018 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.083 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.389241+0000", 2024-09-17T22:41:50.083 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.017458601000000001, 2024-09-17T22:41:50.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026249637999999999, 2024-09-17T22:41:50.083 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.084 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.084 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.084 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146018 2024-09-17T22:41:50.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.084 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389241+0000", 2024-09-17T22:41:50.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389241+0000", 2024-09-17T22:41:50.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389241+0000", 2024-09-17T22:41:50.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:41:50.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389248+0000", 2024-09-17T22:41:50.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5220000000000004e-06 2024-09-17T22:41:50.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389254+0000", 2024-09-17T22:41:50.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4160000000000001e-06 2024-09-17T22:41:50.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389259+0000", 2024-09-17T22:41:50.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9459999999999997e-06 2024-09-17T22:41:50.086 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.086 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.086 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.086 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146083 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.087 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395087+0000", 2024-09-17T22:41:50.087 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.01161217, 2024-09-17T22:41:50.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020456726000000001, 2024-09-17T22:41:50.087 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.087 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.087 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.087 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.087 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.087 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.087 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146083 2024-09-17T22:41:50.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.087 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395087+0000", 2024-09-17T22:41:50.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395087+0000", 2024-09-17T22:41:50.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395088+0000", 2024-09-17T22:41:50.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1799999999999995e-07 2024-09-17T22:41:50.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395090+0000", 2024-09-17T22:41:50.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-09-17T22:41:50.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395097+0000", 2024-09-17T22:41:50.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0910000000000004e-06 2024-09-17T22:41:50.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395103+0000", 2024-09-17T22:41:50.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.198e-06 2024-09-17T22:41:50.090 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.090 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.090 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.090 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146147 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.090 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.401180+0000", 2024-09-17T22:41:50.090 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0055195879999999998, 2024-09-17T22:41:50.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014420087, 2024-09-17T22:41:50.090 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.090 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.090 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.090 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.090 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.090 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.091 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146147 2024-09-17T22:41:50.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.091 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401180+0000", 2024-09-17T22:41:50.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401180+0000", 2024-09-17T22:41:50.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401180+0000", 2024-09-17T22:41:50.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5400000000000001e-07 2024-09-17T22:41:50.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401183+0000", 2024-09-17T22:41:50.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.345e-06 2024-09-17T22:41:50.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401189+0000", 2024-09-17T22:41:50.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6950000000000003e-06 2024-09-17T22:41:50.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401194+0000", 2024-09-17T22:41:50.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9749999999999998e-06 2024-09-17T22:41:50.093 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.093 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.093 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.093 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146210 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.093 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.407242+0000", 2024-09-17T22:41:50.093 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9994569, 2024-09-17T22:41:50.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0084114819999999996, 2024-09-17T22:41:50.094 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.094 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.094 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.094 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.094 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.094 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.094 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146210 2024-09-17T22:41:50.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.094 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407242+0000", 2024-09-17T22:41:50.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407242+0000", 2024-09-17T22:41:50.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407243+0000", 2024-09-17T22:41:50.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:41:50.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407248+0000", 2024-09-17T22:41:50.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.49e-06 2024-09-17T22:41:50.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407252+0000", 2024-09-17T22:41:50.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7079999999999999e-06 2024-09-17T22:41:50.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407255+0000", 2024-09-17T22:41:50.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0489999999999999e-06 2024-09-17T22:41:50.096 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.096 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.096 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.097 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145831 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.097 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.369398+0000", 2024-09-17T22:41:50.097 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.037301187, 2024-09-17T22:41:50.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046309681999999998, 2024-09-17T22:41:50.097 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.097 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.097 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.097 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.097 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.097 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.097 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145831 2024-09-17T22:41:50.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.097 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369398+0000", 2024-09-17T22:41:50.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369398+0000", 2024-09-17T22:41:50.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369399+0000", 2024-09-17T22:41:50.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2600000000000002e-07 2024-09-17T22:41:50.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369400+0000", 2024-09-17T22:41:50.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.742e-06 2024-09-17T22:41:50.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369405+0000", 2024-09-17T22:41:50.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1270000000000002e-06 2024-09-17T22:41:50.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369410+0000", 2024-09-17T22:41:50.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2899999999999996e-06 2024-09-17T22:41:50.100 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.100 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.100 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.100 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145894 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.100 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.377171+0000", 2024-09-17T22:41:50.100 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.029527781999999999, 2024-09-17T22:41:50.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038591384999999999, 2024-09-17T22:41:50.100 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.100 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.100 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.100 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.101 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.101 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.101 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145894 2024-09-17T22:41:50.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.101 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377171+0000", 2024-09-17T22:41:50.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377171+0000", 2024-09-17T22:41:50.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377172+0000", 2024-09-17T22:41:50.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:41:50.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377175+0000", 2024-09-17T22:41:50.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4849999999999999e-06 2024-09-17T22:41:50.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377181+0000", 2024-09-17T22:41:50.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9800000000000001e-06 2024-09-17T22:41:50.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377187+0000", 2024-09-17T22:41:50.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3990000000000003e-06 2024-09-17T22:41:50.103 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.103 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.103 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.103 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145956 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.104 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.381356+0000", 2024-09-17T22:41:50.104 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.025342746999999999, 2024-09-17T22:41:50.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034461420999999999, 2024-09-17T22:41:50.104 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.104 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.104 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.104 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.104 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.104 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.104 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145956 2024-09-17T22:41:50.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.104 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381356+0000", 2024-09-17T22:41:50.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.381356+0000", 2024-09-17T22:41:50.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383050+0000", 2024-09-17T22:41:50.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001693171 2024-09-17T22:41:50.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383054+0000", 2024-09-17T22:41:50.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6399999999999996e-06 2024-09-17T22:41:50.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383065+0000", 2024-09-17T22:41:50.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0699999999999999e-05 2024-09-17T22:41:50.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383073+0000", 2024-09-17T22:41:50.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9529999999999999e-06 2024-09-17T22:41:50.106 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.107 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.107 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.107 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146020 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.107 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.389267+0000", 2024-09-17T22:41:50.107 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.017432703000000001, 2024-09-17T22:41:50.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026603503000000001, 2024-09-17T22:41:50.107 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.107 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.107 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.107 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.107 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.107 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.107 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146020 2024-09-17T22:41:50.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.108 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389267+0000", 2024-09-17T22:41:50.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389267+0000", 2024-09-17T22:41:50.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389267+0000", 2024-09-17T22:41:50.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2e-07 2024-09-17T22:41:50.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389269+0000", 2024-09-17T22:41:50.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1600000000000001e-06 2024-09-17T22:41:50.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389276+0000", 2024-09-17T22:41:50.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3999999999999997e-06 2024-09-17T22:41:50.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389280+0000", 2024-09-17T22:41:50.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9849999999999997e-06 2024-09-17T22:41:50.110 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.110 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.110 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.110 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146085 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.110 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395121+0000", 2024-09-17T22:41:50.110 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.011578629, 2024-09-17T22:41:50.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020800501999999998, 2024-09-17T22:41:50.110 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.110 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.110 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.111 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.111 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.111 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.111 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146085 2024-09-17T22:41:50.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.111 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395121+0000", 2024-09-17T22:41:50.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395121+0000", 2024-09-17T22:41:50.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395121+0000", 2024-09-17T22:41:50.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8699999999999995e-07 2024-09-17T22:41:50.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395124+0000", 2024-09-17T22:41:50.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5210000000000001e-06 2024-09-17T22:41:50.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395131+0000", 2024-09-17T22:41:50.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3200000000000002e-06 2024-09-17T22:41:50.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395135+0000", 2024-09-17T22:41:50.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2869999999999998e-06 2024-09-17T22:41:50.113 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.113 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.113 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.113 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146149 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.114 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.401209+0000", 2024-09-17T22:41:50.114 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0054901289999999998, 2024-09-17T22:41:50.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01476773, 2024-09-17T22:41:50.114 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.114 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.114 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.114 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.114 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.114 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.114 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146149 2024-09-17T22:41:50.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.114 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401209+0000", 2024-09-17T22:41:50.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401209+0000", 2024-09-17T22:41:50.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401210+0000", 2024-09-17T22:41:50.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5900000000000002e-07 2024-09-17T22:41:50.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401212+0000", 2024-09-17T22:41:50.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4549999999999998e-06 2024-09-17T22:41:50.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401219+0000", 2024-09-17T22:41:50.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0110000000000003e-06 2024-09-17T22:41:50.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401228+0000", 2024-09-17T22:41:50.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0599999999999997e-06 2024-09-17T22:41:50.117 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.117 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.117 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.117 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146212 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.117 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.407259+0000", 2024-09-17T22:41:50.117 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9994402, 2024-09-17T22:41:50.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0087720449999999991, 2024-09-17T22:41:50.117 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.117 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.117 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.117 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.118 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.118 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.118 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146212 2024-09-17T22:41:50.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.118 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407259+0000", 2024-09-17T22:41:50.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407259+0000", 2024-09-17T22:41:50.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407259+0000", 2024-09-17T22:41:50.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:41:50.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407261+0000", 2024-09-17T22:41:50.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0950000000000001e-06 2024-09-17T22:41:50.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407264+0000", 2024-09-17T22:41:50.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9289999999999998e-06 2024-09-17T22:41:50.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407266+0000", 2024-09-17T22:41:50.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6309999999999999e-06 2024-09-17T22:41:50.120 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.120 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.120 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.120 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145833 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.120 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.369414+0000", 2024-09-17T22:41:50.121 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.037284750999999998, 2024-09-17T22:41:50.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046677207999999998, 2024-09-17T22:41:50.121 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.121 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.121 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.121 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.121 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.121 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.121 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145833 2024-09-17T22:41:50.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.121 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369414+0000", 2024-09-17T22:41:50.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.369414+0000", 2024-09-17T22:41:50.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371052+0000", 2024-09-17T22:41:50.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001637166 2024-09-17T22:41:50.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371056+0000", 2024-09-17T22:41:50.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-09-17T22:41:50.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371069+0000", 2024-09-17T22:41:50.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2500999999999999e-05 2024-09-17T22:41:50.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371077+0000", 2024-09-17T22:41:50.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4740000000000005e-06 2024-09-17T22:41:50.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.124 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.124 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145896 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.124 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.377195+0000", 2024-09-17T22:41:50.124 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.029504405000000001, 2024-09-17T22:41:50.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038950049, 2024-09-17T22:41:50.124 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.124 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.124 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.125 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145896 2024-09-17T22:41:50.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.125 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377195+0000", 2024-09-17T22:41:50.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377195+0000", 2024-09-17T22:41:50.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377195+0000", 2024-09-17T22:41:50.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9800000000000004e-07 2024-09-17T22:41:50.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377197+0000", 2024-09-17T22:41:50.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1519999999999999e-06 2024-09-17T22:41:50.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377204+0000", 2024-09-17T22:41:50.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6710000000000002e-06 2024-09-17T22:41:50.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377209+0000", 2024-09-17T22:41:50.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7879999999999997e-06 2024-09-17T22:41:50.127 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.127 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.127 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.127 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145958 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.128 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.383085+0000", 2024-09-17T22:41:50.128 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.023613814, 2024-09-17T22:41:50.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033115313, 2024-09-17T22:41:50.128 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.128 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.128 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.128 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.128 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.128 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.128 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145958 2024-09-17T22:41:50.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.128 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383085+0000", 2024-09-17T22:41:50.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383085+0000", 2024-09-17T22:41:50.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383086+0000", 2024-09-17T22:41:50.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8299999999999997e-07 2024-09-17T22:41:50.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383089+0000", 2024-09-17T22:41:50.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7889999999999999e-06 2024-09-17T22:41:50.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383097+0000", 2024-09-17T22:41:50.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1359999999999997e-06 2024-09-17T22:41:50.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383104+0000", 2024-09-17T22:41:50.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8709999999999999e-06 2024-09-17T22:41:50.131 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.131 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.131 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.131 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146022 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.131 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.389287+0000", 2024-09-17T22:41:50.131 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.017412377999999999, 2024-09-17T22:41:50.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.026967689999999999, 2024-09-17T22:41:50.131 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.132 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.132 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.132 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.132 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.132 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.132 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146022 2024-09-17T22:41:50.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.132 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389287+0000", 2024-09-17T22:41:50.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389287+0000", 2024-09-17T22:41:50.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389287+0000", 2024-09-17T22:41:50.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1300000000000001e-07 2024-09-17T22:41:50.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389289+0000", 2024-09-17T22:41:50.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9520000000000001e-06 2024-09-17T22:41:50.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389295+0000", 2024-09-17T22:41:50.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9050000000000002e-06 2024-09-17T22:41:50.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389300+0000", 2024-09-17T22:41:50.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8969999999999999e-06 2024-09-17T22:41:50.134 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.134 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.134 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.135 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146087 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.135 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395142+0000", 2024-09-17T22:41:50.135 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.011556955000000001, 2024-09-17T22:41:50.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021168816999999999, 2024-09-17T22:41:50.135 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.135 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.135 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.135 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.135 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.135 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.135 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146087 2024-09-17T22:41:50.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.135 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395142+0000", 2024-09-17T22:41:50.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395142+0000", 2024-09-17T22:41:50.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395143+0000", 2024-09-17T22:41:50.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9200000000000001e-07 2024-09-17T22:41:50.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395145+0000", 2024-09-17T22:41:50.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2079999999999999e-06 2024-09-17T22:41:50.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395151+0000", 2024-09-17T22:41:50.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0499999999999997e-06 2024-09-17T22:41:50.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395156+0000", 2024-09-17T22:41:50.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6609999999999999e-06 2024-09-17T22:41:50.138 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.138 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.138 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.138 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146151 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.138 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.401236+0000", 2024-09-17T22:41:50.138 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0054627690000000001, 2024-09-17T22:41:50.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015127824, 2024-09-17T22:41:50.138 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.138 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.139 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.139 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.139 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.139 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.139 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146151 2024-09-17T22:41:50.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.139 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401236+0000", 2024-09-17T22:41:50.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401236+0000", 2024-09-17T22:41:50.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401237+0000", 2024-09-17T22:41:50.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7300000000000001e-07 2024-09-17T22:41:50.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401239+0000", 2024-09-17T22:41:50.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2500000000000001e-06 2024-09-17T22:41:50.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401245+0000", 2024-09-17T22:41:50.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2790000000000004e-06 2024-09-17T22:41:50.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401251+0000", 2024-09-17T22:41:50.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.733e-06 2024-09-17T22:41:50.141 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.141 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.141 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.141 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146214 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.141 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.407269+0000", 2024-09-17T22:41:50.141 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9994302, 2024-09-17T22:41:50.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0091480580000000006, 2024-09-17T22:41:50.142 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.142 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.142 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.142 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146214 2024-09-17T22:41:50.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.142 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407269+0000", 2024-09-17T22:41:50.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.407269+0000", 2024-09-17T22:41:50.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409049+0000", 2024-09-17T22:41:50.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017797690000000001 2024-09-17T22:41:50.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409053+0000", 2024-09-17T22:41:50.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6700000000000002e-06 2024-09-17T22:41:50.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409063+0000", 2024-09-17T22:41:50.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5480000000000007e-06 2024-09-17T22:41:50.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409070+0000", 2024-09-17T22:41:50.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5810000000000002e-06 2024-09-17T22:41:50.144 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.144 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.144 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.145 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145835 3.c 3.2e07ed6c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.145 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.371091+0000", 2024-09-17T22:41:50.145 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.035608488000000001, 2024-09-17T22:41:50.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045381878, 2024-09-17T22:41:50.145 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.145 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.145 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.145 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.145 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.145 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.145 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145835 2024-09-17T22:41:50.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.145 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371091+0000", 2024-09-17T22:41:50.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371091+0000", 2024-09-17T22:41:50.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371091+0000", 2024-09-17T22:41:50.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3099999999999998e-07 2024-09-17T22:41:50.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371094+0000", 2024-09-17T22:41:50.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6450000000000001e-06 2024-09-17T22:41:50.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371101+0000", 2024-09-17T22:41:50.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.764e-06 2024-09-17T22:41:50.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371106+0000", 2024-09-17T22:41:50.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5999999999999997e-06 2024-09-17T22:41:50.147 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.147 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.148 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.148 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145898 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.148 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.377216+0000", 2024-09-17T22:41:50.148 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.029483367, 2024-09-17T22:41:50.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039311263999999999, 2024-09-17T22:41:50.148 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.148 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.148 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.148 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145898 2024-09-17T22:41:50.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.149 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377216+0000", 2024-09-17T22:41:50.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377216+0000", 2024-09-17T22:41:50.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377216+0000", 2024-09-17T22:41:50.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:41:50.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377218+0000", 2024-09-17T22:41:50.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0839999999999999e-06 2024-09-17T22:41:50.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377225+0000", 2024-09-17T22:41:50.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.613e-06 2024-09-17T22:41:50.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377230+0000", 2024-09-17T22:41:50.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0209999999999999e-06 2024-09-17T22:41:50.151 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.151 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.151 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.151 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145960 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.151 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.383112+0000", 2024-09-17T22:41:50.151 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.023587459000000002, 2024-09-17T22:41:50.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033468447999999998, 2024-09-17T22:41:50.151 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.151 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.152 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.152 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.152 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.152 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.152 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145960 2024-09-17T22:41:50.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.152 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383112+0000", 2024-09-17T22:41:50.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383112+0000", 2024-09-17T22:41:50.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383112+0000", 2024-09-17T22:41:50.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:41:50.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383114+0000", 2024-09-17T22:41:50.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3240000000000001e-06 2024-09-17T22:41:50.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383121+0000", 2024-09-17T22:41:50.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2389999999999999e-06 2024-09-17T22:41:50.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383126+0000", 2024-09-17T22:41:50.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.04e-06 2024-09-17T22:41:50.154 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.154 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.154 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.155 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146024 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.155 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.389306+0000", 2024-09-17T22:41:50.155 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.017392828999999999, 2024-09-17T22:41:50.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027328277000000002, 2024-09-17T22:41:50.155 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.155 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.155 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.155 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.155 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.155 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.155 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146024 2024-09-17T22:41:50.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.155 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389306+0000", 2024-09-17T22:41:50.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389306+0000", 2024-09-17T22:41:50.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389307+0000", 2024-09-17T22:41:50.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4799999999999999e-07 2024-09-17T22:41:50.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389309+0000", 2024-09-17T22:41:50.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-09-17T22:41:50.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389315+0000", 2024-09-17T22:41:50.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.533e-06 2024-09-17T22:41:50.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389322+0000", 2024-09-17T22:41:50.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2360000000000001e-06 2024-09-17T22:41:50.158 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.158 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.158 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.158 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146089 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.158 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395162+0000", 2024-09-17T22:41:50.158 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.011537313, 2024-09-17T22:41:50.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021525641000000002, 2024-09-17T22:41:50.158 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.159 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.159 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.159 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.159 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.159 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.159 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146089 2024-09-17T22:41:50.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.159 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395162+0000", 2024-09-17T22:41:50.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395162+0000", 2024-09-17T22:41:50.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395162+0000", 2024-09-17T22:41:50.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8799999999999998e-07 2024-09-17T22:41:50.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395164+0000", 2024-09-17T22:41:50.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.069e-06 2024-09-17T22:41:50.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395171+0000", 2024-09-17T22:41:50.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0829999999999998e-06 2024-09-17T22:41:50.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395176+0000", 2024-09-17T22:41:50.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9980000000000003e-06 2024-09-17T22:41:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146153 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.401258+0000", 2024-09-17T22:41:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0054413380000000004, 2024-09-17T22:41:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.015483696, 2024-09-17T22:41:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.162 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146153 2024-09-17T22:41:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401258+0000", 2024-09-17T22:41:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401258+0000", 2024-09-17T22:41:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401258+0000", 2024-09-17T22:41:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:41:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401260+0000", 2024-09-17T22:41:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-09-17T22:41:50.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401268+0000", 2024-09-17T22:41:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7039999999999994e-06 2024-09-17T22:41:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401273+0000", 2024-09-17T22:41:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0649999999999998e-06 2024-09-17T22:41:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146216 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.409082+0000", 2024-09-17T22:41:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9976172, 2024-09-17T22:41:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0077174599999999998, 2024-09-17T22:41:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.166 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146216 2024-09-17T22:41:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409082+0000", 2024-09-17T22:41:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409082+0000", 2024-09-17T22:41:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409082+0000", 2024-09-17T22:41:50.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6400000000000003e-07 2024-09-17T22:41:50.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409085+0000", 2024-09-17T22:41:50.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4109999999999999e-06 2024-09-17T22:41:50.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409091+0000", 2024-09-17T22:41:50.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1750000000000002e-06 2024-09-17T22:41:50.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409096+0000", 2024-09-17T22:41:50.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.215e-06 2024-09-17T22:41:50.169 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.169 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.169 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.169 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145838 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.169 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.371122+0000", 2024-09-17T22:41:50.169 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.035576952000000002, 2024-09-17T22:41:50.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045731962000000001, 2024-09-17T22:41:50.169 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.169 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.169 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.169 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.170 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.170 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.170 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145838 2024-09-17T22:41:50.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.170 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371122+0000", 2024-09-17T22:41:50.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371122+0000", 2024-09-17T22:41:50.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371123+0000", 2024-09-17T22:41:50.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6199999999999998e-07 2024-09-17T22:41:50.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371125+0000", 2024-09-17T22:41:50.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6539999999999999e-06 2024-09-17T22:41:50.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371132+0000", 2024-09-17T22:41:50.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8689999999999998e-06 2024-09-17T22:41:50.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371143+0000", 2024-09-17T22:41:50.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0971e-05 2024-09-17T22:41:50.172 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.172 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.172 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.173 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145900 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.173 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.377244+0000", 2024-09-17T22:41:50.173 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.029455263999999998, 2024-09-17T22:41:50.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039642315999999997, 2024-09-17T22:41:50.173 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.173 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.173 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.173 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.173 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.173 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.173 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145900 2024-09-17T22:41:50.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.174 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377244+0000", 2024-09-17T22:41:50.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377244+0000", 2024-09-17T22:41:50.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377244+0000", 2024-09-17T22:41:50.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7199999999999999e-07 2024-09-17T22:41:50.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377251+0000", 2024-09-17T22:41:50.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8870000000000003e-06 2024-09-17T22:41:50.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377259+0000", 2024-09-17T22:41:50.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.413e-06 2024-09-17T22:41:50.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377264+0000", 2024-09-17T22:41:50.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2700000000000004e-06 2024-09-17T22:41:50.176 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.176 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.176 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.176 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145962 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.176 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.383132+0000", 2024-09-17T22:41:50.176 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.023566857, 2024-09-17T22:41:50.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.033780952000000003, 2024-09-17T22:41:50.176 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.176 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.176 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.177 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.177 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.177 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.177 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145962 2024-09-17T22:41:50.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.177 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383132+0000", 2024-09-17T22:41:50.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383132+0000", 2024-09-17T22:41:50.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383133+0000", 2024-09-17T22:41:50.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-09-17T22:41:50.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383135+0000", 2024-09-17T22:41:50.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1469999999999999e-06 2024-09-17T22:41:50.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383141+0000", 2024-09-17T22:41:50.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2530000000000001e-06 2024-09-17T22:41:50.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383151+0000", 2024-09-17T22:41:50.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5079999999999994e-06 2024-09-17T22:41:50.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.179 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.179 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146026 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.179 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.389328+0000", 2024-09-17T22:41:50.179 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.017371705000000001, 2024-09-17T22:41:50.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027614046, 2024-09-17T22:41:50.180 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.180 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.180 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.180 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146026 2024-09-17T22:41:50.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.180 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389328+0000", 2024-09-17T22:41:50.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389328+0000", 2024-09-17T22:41:50.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389328+0000", 2024-09-17T22:41:50.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6499999999999999e-07 2024-09-17T22:41:50.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389330+0000", 2024-09-17T22:41:50.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8950000000000001e-06 2024-09-17T22:41:50.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389336+0000", 2024-09-17T22:41:50.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7660000000000004e-06 2024-09-17T22:41:50.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389341+0000", 2024-09-17T22:41:50.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8659999999999997e-06 2024-09-17T22:41:50.182 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.182 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.183 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.183 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146091 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.183 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395183+0000", 2024-09-17T22:41:50.183 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.011516001, 2024-09-17T22:41:50.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02178515, 2024-09-17T22:41:50.183 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.183 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.183 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.183 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.183 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.183 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.183 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146091 2024-09-17T22:41:50.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.184 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395183+0000", 2024-09-17T22:41:50.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395183+0000", 2024-09-17T22:41:50.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395184+0000", 2024-09-17T22:41:50.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4499999999999997e-07 2024-09-17T22:41:50.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395190+0000", 2024-09-17T22:41:50.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0700000000000003e-06 2024-09-17T22:41:50.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395196+0000", 2024-09-17T22:41:50.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.584e-06 2024-09-17T22:41:50.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395201+0000", 2024-09-17T22:41:50.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0250000000000002e-06 2024-09-17T22:41:50.186 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.186 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.186 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.186 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146155 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.186 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.401279+0000", 2024-09-17T22:41:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0054198290000000001, 2024-09-17T22:41:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01572904, 2024-09-17T22:41:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146155 2024-09-17T22:41:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401279+0000", 2024-09-17T22:41:50.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.401279+0000", 2024-09-17T22:41:50.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403049+0000", 2024-09-17T22:41:50.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001769804 2024-09-17T22:41:50.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403054+0000", 2024-09-17T22:41:50.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3340000000000003e-06 2024-09-17T22:41:50.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403065+0000", 2024-09-17T22:41:50.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0963e-05 2024-09-17T22:41:50.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403072+0000", 2024-09-17T22:41:50.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2010000000000002e-06 2024-09-17T22:41:50.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.190 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.190 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146218 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.190 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.409103+0000", 2024-09-17T22:41:50.190 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9975967, 2024-09-17T22:41:50.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079645270000000008, 2024-09-17T22:41:50.190 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.191 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.191 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.191 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146218 2024-09-17T22:41:50.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.191 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409103+0000", 2024-09-17T22:41:50.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409103+0000", 2024-09-17T22:41:50.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409103+0000", 2024-09-17T22:41:50.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3799999999999998e-07 2024-09-17T22:41:50.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409105+0000", 2024-09-17T22:41:50.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.018e-06 2024-09-17T22:41:50.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409112+0000", 2024-09-17T22:41:50.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7170000000000002e-06 2024-09-17T22:41:50.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409117+0000", 2024-09-17T22:41:50.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0359999999999997e-06 2024-09-17T22:41:50.193 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.193 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.193 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.194 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145840 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.194 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.371159+0000", 2024-09-17T22:41:50.194 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.035539822999999998, 2024-09-17T22:41:50.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045968149999999999, 2024-09-17T22:41:50.194 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.194 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.194 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.194 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.194 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.194 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.194 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145840 2024-09-17T22:41:50.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.195 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371159+0000", 2024-09-17T22:41:50.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371159+0000", 2024-09-17T22:41:50.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371160+0000", 2024-09-17T22:41:50.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-09-17T22:41:50.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371163+0000", 2024-09-17T22:41:50.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6249999999999999e-06 2024-09-17T22:41:50.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371170+0000", 2024-09-17T22:41:50.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2320000000000004e-06 2024-09-17T22:41:50.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371175+0000", 2024-09-17T22:41:50.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3000000000000001e-06 2024-09-17T22:41:50.197 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.197 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.197 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.197 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145903 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.197 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.377272+0000", 2024-09-17T22:41:50.197 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.029427167000000001, 2024-09-17T22:41:50.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039908605, 2024-09-17T22:41:50.198 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.198 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.198 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.198 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.198 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.198 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.198 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145903 2024-09-17T22:41:50.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.198 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377272+0000", 2024-09-17T22:41:50.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377272+0000", 2024-09-17T22:41:50.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377273+0000", 2024-09-17T22:41:50.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8299999999999997e-07 2024-09-17T22:41:50.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377275+0000", 2024-09-17T22:41:50.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2970000000000002e-06 2024-09-17T22:41:50.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377284+0000", 2024-09-17T22:41:50.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4199999999999996e-06 2024-09-17T22:41:50.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377291+0000", 2024-09-17T22:41:50.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.131e-06 2024-09-17T22:41:50.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.200 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.201 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.201 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145964 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.201 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.383177+0000", 2024-09-17T22:41:50.201 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.023522128, 2024-09-17T22:41:50.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034059923999999998, 2024-09-17T22:41:50.201 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.201 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.201 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.201 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145964 2024-09-17T22:41:50.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.202 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383177+0000", 2024-09-17T22:41:50.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383177+0000", 2024-09-17T22:41:50.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383178+0000", 2024-09-17T22:41:50.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1500000000000005e-07 2024-09-17T22:41:50.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383180+0000", 2024-09-17T22:41:50.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.526e-06 2024-09-17T22:41:50.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383187+0000", 2024-09-17T22:41:50.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.674e-06 2024-09-17T22:41:50.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383192+0000", 2024-09-17T22:41:50.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0699999999999997e-06 2024-09-17T22:41:50.204 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.204 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.204 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.204 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146028 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.204 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.389346+0000", 2024-09-17T22:41:50.204 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.017352853000000001, 2024-09-17T22:41:50.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.027933499000000001, 2024-09-17T22:41:50.204 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.204 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.204 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.205 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146028 2024-09-17T22:41:50.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.205 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389346+0000", 2024-09-17T22:41:50.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389346+0000", 2024-09-17T22:41:50.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389347+0000", 2024-09-17T22:41:50.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:41:50.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389349+0000", 2024-09-17T22:41:50.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9350000000000001e-06 2024-09-17T22:41:50.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389354+0000", 2024-09-17T22:41:50.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7490000000000004e-06 2024-09-17T22:41:50.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389359+0000", 2024-09-17T22:41:50.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1059999999999999e-06 2024-09-17T22:41:50.207 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.207 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.207 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.208 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146093 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.208 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395208+0000", 2024-09-17T22:41:50.208 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.011490953999999999, 2024-09-17T22:41:50.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022102163000000001, 2024-09-17T22:41:50.208 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.208 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.208 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.208 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146093 2024-09-17T22:41:50.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.208 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395208+0000", 2024-09-17T22:41:50.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395208+0000", 2024-09-17T22:41:50.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395209+0000", 2024-09-17T22:41:50.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.84e-07 2024-09-17T22:41:50.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395211+0000", 2024-09-17T22:41:50.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3439999999999999e-06 2024-09-17T22:41:50.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395217+0000", 2024-09-17T22:41:50.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8590000000000001e-06 2024-09-17T22:41:50.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395223+0000", 2024-09-17T22:41:50.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6509999999999998e-06 2024-09-17T22:41:50.211 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.211 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.211 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.211 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146157 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.211 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403084+0000", 2024-09-17T22:41:50.211 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0036148370000000001, 2024-09-17T22:41:50.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014252999000000001, 2024-09-17T22:41:50.211 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.212 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.212 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.212 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.212 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.212 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.212 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146157 2024-09-17T22:41:50.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.212 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403084+0000", 2024-09-17T22:41:50.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403084+0000", 2024-09-17T22:41:50.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403085+0000", 2024-09-17T22:41:50.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0399999999999996e-07 2024-09-17T22:41:50.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403087+0000", 2024-09-17T22:41:50.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5639999999999999e-06 2024-09-17T22:41:50.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403095+0000", 2024-09-17T22:41:50.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2470000000000002e-06 2024-09-17T22:41:50.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403100+0000", 2024-09-17T22:41:50.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3959999999999997e-06 2024-09-17T22:41:50.214 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.214 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.214 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.215 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145841 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.215 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.371183+0000", 2024-09-17T22:41:50.215 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.035516606999999999, 2024-09-17T22:41:50.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046184498999999997, 2024-09-17T22:41:50.215 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.215 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.215 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.215 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.215 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.215 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.215 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145841 2024-09-17T22:41:50.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.216 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371183+0000", 2024-09-17T22:41:50.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371183+0000", 2024-09-17T22:41:50.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371183+0000", 2024-09-17T22:41:50.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9999999999999998e-07 2024-09-17T22:41:50.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371186+0000", 2024-09-17T22:41:50.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6599999999999999e-06 2024-09-17T22:41:50.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371194+0000", 2024-09-17T22:41:50.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1729999999999995e-06 2024-09-17T22:41:50.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.371199+0000", 2024-09-17T22:41:50.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3240000000000002e-06 2024-09-17T22:41:50.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.218 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.218 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145905 3.c 3.389620ec (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.218 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.377300+0000", 2024-09-17T22:41:50.218 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.029399234, 2024-09-17T22:41:50.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040113419999999997, 2024-09-17T22:41:50.218 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.219 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.219 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.219 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145905 2024-09-17T22:41:50.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.219 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377300+0000", 2024-09-17T22:41:50.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377300+0000", 2024-09-17T22:41:50.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377300+0000", 2024-09-17T22:41:50.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0499999999999999e-07 2024-09-17T22:41:50.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377303+0000", 2024-09-17T22:41:50.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4600000000000002e-06 2024-09-17T22:41:50.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377321+0000", 2024-09-17T22:41:50.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8355999999999998e-05 2024-09-17T22:41:50.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.377332+0000", 2024-09-17T22:41:50.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0594999999999999e-05 2024-09-17T22:41:50.221 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.221 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.221 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.222 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145966 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.222 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.383199+0000", 2024-09-17T22:41:50.222 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.023500686, 2024-09-17T22:41:50.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034269158000000001, 2024-09-17T22:41:50.222 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.222 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.222 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.222 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.222 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.223 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.223 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145966 2024-09-17T22:41:50.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.223 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383199+0000", 2024-09-17T22:41:50.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383199+0000", 2024-09-17T22:41:50.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383199+0000", 2024-09-17T22:41:50.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4700000000000002e-07 2024-09-17T22:41:50.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383201+0000", 2024-09-17T22:41:50.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.103e-06 2024-09-17T22:41:50.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383207+0000", 2024-09-17T22:41:50.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2020000000000001e-06 2024-09-17T22:41:50.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383212+0000", 2024-09-17T22:41:50.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.887e-06 2024-09-17T22:41:50.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.225 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.225 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146030 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.225 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.389366+0000", 2024-09-17T22:41:50.226 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.017333665000000002, 2024-09-17T22:41:50.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028143114, 2024-09-17T22:41:50.226 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.226 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.226 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.226 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146030 2024-09-17T22:41:50.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.226 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389366+0000", 2024-09-17T22:41:50.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389366+0000", 2024-09-17T22:41:50.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389366+0000", 2024-09-17T22:41:50.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3099999999999998e-07 2024-09-17T22:41:50.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389368+0000", 2024-09-17T22:41:50.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.951e-06 2024-09-17T22:41:50.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389376+0000", 2024-09-17T22:41:50.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7430000000000002e-06 2024-09-17T22:41:50.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.389380+0000", 2024-09-17T22:41:50.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6990000000000002e-06 2024-09-17T22:41:50.228 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.228 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.229 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.229 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146095 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.229 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395230+0000", 2024-09-17T22:41:50.229 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.011469465, 2024-09-17T22:41:50.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022306355, 2024-09-17T22:41:50.229 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.229 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.229 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.229 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146095 2024-09-17T22:41:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395230+0000", 2024-09-17T22:41:50.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395230+0000", 2024-09-17T22:41:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395230+0000", 2024-09-17T22:41:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1800000000000001e-07 2024-09-17T22:41:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395232+0000", 2024-09-17T22:41:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0640000000000001e-06 2024-09-17T22:41:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395239+0000", 2024-09-17T22:41:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5479999999999998e-06 2024-09-17T22:41:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395244+0000", 2024-09-17T22:41:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0930000000000002e-06 2024-09-17T22:41:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.232 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146159 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403109+0000", 2024-09-17T22:41:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.003590251, 2024-09-17T22:41:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.014461895000000001, 2024-09-17T22:41:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146159 2024-09-17T22:41:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403109+0000", 2024-09-17T22:41:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403109+0000", 2024-09-17T22:41:50.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403110+0000", 2024-09-17T22:41:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1500000000000004e-07 2024-09-17T22:41:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403112+0000", 2024-09-17T22:41:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.295e-06 2024-09-17T22:41:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403120+0000", 2024-09-17T22:41:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6699999999999994e-06 2024-09-17T22:41:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403125+0000", 2024-09-17T22:41:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0509999999999996e-06 2024-09-17T22:41:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146220 3.c 3.389620ec (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:50.236 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.409324+0000", 2024-09-17T22:41:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9973755, 2024-09-17T22:41:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0082864670000000005, 2024-09-17T22:41:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-09-17T22:41:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146220 2024-09-17T22:41:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:50.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409324+0000", 2024-09-17T22:41:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409324+0000", 2024-09-17T22:41:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409324+0000", 2024-09-17T22:41:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8699999999999995e-07 2024-09-17T22:41:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409327+0000", 2024-09-17T22:41:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0730000000000001e-06 2024-09-17T22:41:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409334+0000", 2024-09-17T22:41:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5039999999999999e-06 2024-09-17T22:41:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.409339+0000", 2024-09-17T22:41:50.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.575e-06 2024-09-17T22:41:50.240 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.240 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:50.240 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.240 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:50.240 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:41:50.240 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 232 2024-09-17T22:41:50.240 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:50.241 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:41:50.242 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T22:41:50.242 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:50.238+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:50.342 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:50.340+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:50.444 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:50.441+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:50.457 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:50.457 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:50.457 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:41:50.457 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:41:50.457 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:50.470 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T22:41:50.473 INFO:teuthology.orchestra.run.smithi115.stdout:osd.0: {} 2024-09-17T22:41:50.474 INFO:teuthology.orchestra.run.smithi115.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T22:41:50.504 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:41:50.504 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:41:50.544 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:50.542+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:50.595 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:41:50.595 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:41:50.627 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:41:50.627 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:41:50.632 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:50.633 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:50.648 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:41:50.649 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:50.643+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:50.661 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:41:50.661 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:41:50.691 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:41:50.691 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:41:50.722 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:41:50.723 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:41:50.747 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:50.744+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:50.756 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:41:50.756 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:41:50.829 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:50.829 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:50.842 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:41:50.886 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:50.884+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:50.949 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:50.946+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:51.008 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:51.008 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:51.020 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:41:51.062 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:51.059+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:51.113 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:41:51.151 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:51.148+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:51.208 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:51.208 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:51.220 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:41:51.264 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:51.261+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:51.278 INFO:teuthology.orchestra.run.smithi115.stdout:1088 write 0x6781028 thru 0x6789455 (0x842e bytes) 2024-09-17T22:41:51.290 INFO:teuthology.orchestra.run.smithi115.stdout:1089 write 0xbfb5a18 thru 0xbfc36a0 (0xdc89 bytes) 2024-09-17T22:41:51.301 INFO:teuthology.orchestra.run.smithi115.stdout:1090 write 0x2ee2edf thru 0x2eec2c2 (0x93e4 bytes) 2024-09-17T22:41:51.308 INFO:teuthology.orchestra.run.smithi115.stdout:1091 read 0x1d7fa7e thru 0x1d8488b (0x4e0e bytes) 2024-09-17T22:41:51.308 INFO:teuthology.orchestra.run.smithi115.stdout:1094 punch from 0x49f106a to 0x49fa195, (0x912b bytes) 2024-09-17T22:41:51.308 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:51.313 INFO:teuthology.orchestra.run.smithi115.stdout:1097 writesame 0xdfef524 thru 0xdff603b data_size 0x2e(0x6b18 bytes) 2024-09-17T22:41:51.314 INFO:teuthology.orchestra.run.smithi115.stdout:1098 punch from 0x560c6f8 to 0x5612b44, (0x644c bytes) 2024-09-17T22:41:51.314 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:51.314 INFO:teuthology.orchestra.run.smithi115.stdout:1100 trunc from 0xdff603c to 0xec1cdd8 2024-09-17T22:41:51.320 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:51.320 INFO:teuthology.orchestra.run.smithi115.stdout:1103 write 0xd10243a thru 0xd1114f3 (0xf0ba bytes) 2024-09-17T22:41:51.330 INFO:teuthology.orchestra.run.smithi115.stdout:1104 writesame 0x3d985f thru 0x3e60ec data_size 0x56(0xc88e bytes) 2024-09-17T22:41:51.345 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:51.345 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:51.345 INFO:teuthology.orchestra.run.smithi115.stdout:1105 trunc from 0xec1cdd8 to 0x86a6187 2024-09-17T22:41:51.352 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:51.349+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:51.357 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:41:51.373 INFO:teuthology.orchestra.run.smithi115.stdout:1106 write 0xd279c44 thru 0xd27c785 (0x2b42 bytes) 2024-09-17T22:41:51.376 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:51.376 INFO:teuthology.orchestra.run.smithi115.stdout:1109 punch from 0xac70475 to 0xac7acf5, (0xa880 bytes) 2024-09-17T22:41:51.377 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:51.377 INFO:teuthology.orchestra.run.smithi115.stdout:1111 writesame 0x686c19a thru 0x68744ee data_size 0x7(0x8355 bytes) 2024-09-17T22:41:51.381 INFO:teuthology.orchestra.run.smithi115.stdout:1113 punch from 0x37e8d01 to 0x37f3953, (0xac52 bytes) 2024-09-17T22:41:51.381 INFO:teuthology.orchestra.run.smithi115.stdout:1114 read 0x248bc4e thru 0x2493706 (0x7ab9 bytes) 2024-09-17T22:41:51.381 INFO:teuthology.orchestra.run.smithi115.stdout:1115 read 0x135201 thru 0x13817f (0x2f7f bytes) 2024-09-17T22:41:51.382 INFO:teuthology.orchestra.run.smithi115.stdout:1116 write 0xb908b6e thru 0xb910b4b (0x7fde bytes) 2024-09-17T22:41:51.386 INFO:teuthology.orchestra.run.smithi115.stdout:1117 write 0xf80279 thru 0xf8df1b (0xdca3 bytes) 2024-09-17T22:41:51.392 INFO:teuthology.orchestra.run.smithi115.stdout:1118 read 0x640a75d thru 0x6411fee (0x7892 bytes) 2024-09-17T22:41:51.392 INFO:teuthology.orchestra.run.smithi115.stdout:1119 writesame 0x106c0fc thru 0x107417b data_size 0x40(0x8080 bytes) 2024-09-17T22:41:51.397 INFO:teuthology.orchestra.run.smithi115.stdout:1120 write 0x572fc56 thru 0x5732353 (0x26fe bytes) 2024-09-17T22:41:51.402 INFO:teuthology.orchestra.run.smithi115.stdout:1121 write 0x9ebb7d2 thru 0x9ebdf1b (0x274a bytes) 2024-09-17T22:41:51.405 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:51.411 INFO:teuthology.orchestra.run.smithi115.stdout:1124 trunc from 0xd27c786 to 0x7f2d2f3 2024-09-17T22:41:51.435 INFO:teuthology.orchestra.run.smithi115.stdout:1125 punch from 0x3026d32 to 0x302db1b, (0x6de9 bytes) 2024-09-17T22:41:51.438 INFO:teuthology.orchestra.run.smithi115.stdout:1126 trunc from 0x7f2d2f3 to 0x50cd8e2 2024-09-17T22:41:51.452 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:51.450+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:51.468 INFO:teuthology.orchestra.run.smithi115.stdout:1132 punch from 0x2caf54d to 0x2cbdc3c, (0xe6ef bytes) 2024-09-17T22:41:51.469 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:51.485 INFO:teuthology.orchestra.run.smithi115.stdout:1135 writesame 0xe50d3d1 thru 0xe50dbed data_size 0x1(0x81d bytes) 2024-09-17T22:41:51.486 INFO:teuthology.orchestra.run.smithi115.stdout:1136 write 0x722f52 thru 0x72aaba (0x7b69 bytes) 2024-09-17T22:41:51.491 INFO:teuthology.orchestra.run.smithi115.stdout:1137 punch from 0x90aa59f to 0x90abf52, (0x19b3 bytes) 2024-09-17T22:41:51.492 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:51.501 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:51.501 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:51.506 INFO:teuthology.orchestra.run.smithi115.stdout:1139 trunc from 0xe50dbee to 0x2052883 2024-09-17T22:41:51.514 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:41:51.542 INFO:teuthology.orchestra.run.smithi115.stdout:1140 write 0x6e44ea2 thru 0x6e48252 (0x33b1 bytes) 2024-09-17T22:41:51.553 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:51.551+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:51.559 INFO:teuthology.orchestra.run.smithi115.stdout:1141 writesame 0xb1363df thru 0xb13d3ce data_size 0x48(0x6ff0 bytes) 2024-09-17T22:41:51.559 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:51.559 INFO:teuthology.orchestra.run.smithi115.stdout:1143 read 0x837eb8 thru 0x83e59d (0x66e6 bytes) 2024-09-17T22:41:51.566 INFO:teuthology.orchestra.run.smithi115.stdout:1144 write 0xd2ad047 thru 0xd2bb383 (0xe33d bytes) 2024-09-17T22:41:51.573 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:51.573 INFO:teuthology.orchestra.run.smithi115.stdout:1147 write 0x7dc32e9 thru 0x7dd0ce2 (0xd9fa bytes) 2024-09-17T22:41:51.579 INFO:teuthology.orchestra.run.smithi115.stdout:1150 punch from 0x6f05bac to 0x6f0f7c8, (0x9c1c bytes) 2024-09-17T22:41:51.579 INFO:teuthology.orchestra.run.smithi115.stdout:1152 write 0x523a36f thru 0x523a6d0 (0x362 bytes) 2024-09-17T22:41:51.597 INFO:teuthology.orchestra.run.smithi115.stdout:1153 trunc from 0xd2bb384 to 0x1b06254 2024-09-17T22:41:51.625 INFO:teuthology.orchestra.run.smithi115.stdout:1154 read 0x785f2c thru 0x78e6c6 (0x879b bytes) 2024-09-17T22:41:51.625 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:51.646 INFO:teuthology.orchestra.run.smithi115.stdout:1159 writesame 0xd0d9083 thru 0xd0e2ef4 data_size 0x11(0x9e72 bytes) 2024-09-17T22:41:51.646 INFO:teuthology.orchestra.run.smithi115.stdout:1160 writesame 0x2e4fbe7 thru 0x2e5dc32 data_size 0x63(0xe04c bytes) 2024-09-17T22:41:51.653 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:51.651+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:51.656 INFO:teuthology.orchestra.run.smithi115.stdout:1161 punch from 0x3b05be3 to 0x3b0b9f9, (0x5e16 bytes) 2024-09-17T22:41:51.656 INFO:teuthology.orchestra.run.smithi115.stdout:1162 read 0x4bed918 thru 0x4bf9e16 (0xc4ff bytes) 2024-09-17T22:41:51.656 INFO:teuthology.orchestra.run.smithi115.stdout:1163 writesame 0x715360c thru 0x715ca8c data_size 0x1(0x9481 bytes) 2024-09-17T22:41:51.674 INFO:teuthology.orchestra.run.smithi115.stdout:1164 trunc from 0xd0e2ef5 to 0x99868e5 2024-09-17T22:41:51.679 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:51.679 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:51.679 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:51.681 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:51.682 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:51.690 INFO:teuthology.orchestra.run.smithi115.stdout:1169 writesame 0xdfbeba4 thru 0xdfc4c70 data_size 0x1(0x60cd bytes) 2024-09-17T22:41:51.691 INFO:teuthology.orchestra.run.smithi115.stdout:1170 write 0x7ab7cdd thru 0x7ac1882 (0x9ba6 bytes) 2024-09-17T22:41:51.694 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:41:51.700 INFO:teuthology.orchestra.run.smithi115.stdout:1171 writesame 0x889283a thru 0x889f11d data_size 0x5c(0xc8e4 bytes) 2024-09-17T22:41:51.707 INFO:teuthology.orchestra.run.smithi115.stdout:1173 writesame 0x3031919 thru 0x3040630 data_size 0x6c(0xed18 bytes) 2024-09-17T22:41:51.715 INFO:teuthology.orchestra.run.smithi115.stdout:1175 writesame 0x4550921 thru 0x4554b8a data_size 0x1(0x426a bytes) 2024-09-17T22:41:51.723 INFO:teuthology.orchestra.run.smithi115.stdout:1176 read 0x346ea96 thru 0x34726f9 (0x3c64 bytes) 2024-09-17T22:41:51.724 INFO:teuthology.orchestra.run.smithi115.stdout:1177 write 0x7a0cab6 thru 0x7a0d566 (0xab1 bytes) 2024-09-17T22:41:51.726 INFO:teuthology.orchestra.run.smithi115.stdout:1178 read 0xcc677e5 thru 0xcc68cd9 (0x14f5 bytes) 2024-09-17T22:41:51.741 INFO:teuthology.orchestra.run.smithi115.stdout:1179 trunc from 0xdfc4c71 to 0x211dad4 2024-09-17T22:41:51.754 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:51.751+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:51.767 INFO:teuthology.orchestra.run.smithi115.stdout:1180 write 0x9f050ab thru 0x9f10b06 (0xba5c bytes) 2024-09-17T22:41:51.775 INFO:teuthology.orchestra.run.smithi115.stdout:1181 write 0x33f5696 thru 0x33fc79c (0x7107 bytes) 2024-09-17T22:41:51.782 INFO:teuthology.orchestra.run.smithi115.stdout:1182 punch from 0x5313b59 to 0x5321ee9, (0xe390 bytes) 2024-09-17T22:41:51.782 INFO:teuthology.orchestra.run.smithi115.stdout:1183 read 0x28541dc thru 0x28639e4 (0xf809 bytes) 2024-09-17T22:41:51.782 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:51.787 INFO:teuthology.orchestra.run.smithi115.stdout:1186 trunc from 0x9f10b07 to 0x5cdf89a 2024-09-17T22:41:51.797 INFO:teuthology.orchestra.run.smithi115.stdout:1187 punch from 0x1e4989c to 0x1e50437, (0x6b9b bytes) 2024-09-17T22:41:51.797 INFO:teuthology.orchestra.run.smithi115.stdout:1188 clone 15 order 23 su 2097152 sc 6 2024-09-17T22:41:51.820 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:51.820 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:51.829 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:41:51.854 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:51.852+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:51.956 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:51.953+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:51.976 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:51.976 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:51.984 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:41:52.057 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:52.054+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:52.088 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:52.088 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:52.096 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:41:52.100 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is unset 2024-09-17T22:41:52.157 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:41:52.155+0000 7f3726da7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33474) UID: 0 2024-09-17T22:41:52.170 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone14 from 0x5cdf89a (overlap 0x1b06254) to 0x0 2024-09-17T22:41:52.231 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:52.231 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:52.244 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:41:52.288 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:52.286+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:52.325 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-17T22:41:52.327 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:52.344 INFO:teuthology.orchestra.run.smithi115.stdout:1190 writesame 0xd8ebfd6 thru 0xd8f47ec data_size 0x4f(0x8817 bytes) 2024-09-17T22:41:52.345 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:52.345 INFO:teuthology.orchestra.run.smithi115.stdout:1192 writesame 0x3b8fef3 thru 0x3b9615c data_size 0x66(0x626a bytes) 2024-09-17T22:41:52.358 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:52.358 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:52.359 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:52.356+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:52.363 INFO:teuthology.orchestra.run.smithi115.stdout:1193 trunc from 0xd8f47ed to 0xd3e3b4 2024-09-17T22:41:52.368 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:41:52.407 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:41:52.407 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:41:52.408 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:41:52.408 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T22:41:52.408 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T22:41:52.412 INFO:teuthology.orchestra.run.smithi115.stdout:1194 write 0xa5b9e thru 0xab474 (0x58d7 bytes) 2024-09-17T22:41:52.422 INFO:teuthology.orchestra.run.smithi115.stdout:1195 trunc from 0xd3e3b4 to 0x830f2ea 2024-09-17T22:41:52.430 INFO:teuthology.orchestra.run.smithi115.stdout:1196 read 0x6992d7d thru 0x699fb4c (0xcdd0 bytes) 2024-09-17T22:41:52.436 INFO:teuthology.orchestra.run.smithi115.stdout:1198 trunc from 0x830f2ea to 0xcdbc412 2024-09-17T22:41:52.439 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T22:41:52.439 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.0 2024-09-17T22:41:52.439 INFO:tasks.thrashosds.thrasher:remote for osd 0 is ubuntu@smithi115.front.sepia.ceph.com 2024-09-17T22:41:52.439 DEBUG:teuthology.orchestra.run.smithi115:> type ceph-objectstore-tool 2024-09-17T22:41:52.443 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:52.443 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:52.443 INFO:teuthology.orchestra.run.smithi115.stdout:1201 read 0x72bc9c5 thru 0x72c137f (0x49bb bytes) 2024-09-17T22:41:52.443 INFO:teuthology.orchestra.run.smithi115.stdout:1202 read 0x49d93cb thru 0x49e27d2 (0x9408 bytes) 2024-09-17T22:41:52.443 INFO:teuthology.orchestra.run.smithi115.stdout:1203 read 0xac8ff12 thru 0xac93a75 (0x3b64 bytes) 2024-09-17T22:41:52.444 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:52.444 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:52.444 INFO:teuthology.orchestra.run.smithi115.stdout:1206 punch from 0x3dc64a4 to 0x3dd1887, (0xb3e3 bytes) 2024-09-17T22:41:52.444 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:52.454 INFO:teuthology.orchestra.run.smithi115.stdout:1212 trunc from 0xcdbc412 to 0x30e4e25 2024-09-17T22:41:52.459 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:52.457+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:52.460 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:52.460 INFO:teuthology.orchestra.run.smithi115.stdout:1214 punch from 0x296f628 to 0x2973bde, (0x45b6 bytes) 2024-09-17T22:41:52.460 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T22:41:52.461 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op list-pgs 2024-09-17T22:41:52.464 INFO:teuthology.orchestra.run.smithi115.stdout:1215 trunc from 0x30e4e25 to 0x65eaa66 2024-09-17T22:41:52.470 INFO:teuthology.orchestra.run.smithi115.stdout:1216 read 0x5544ac7 thru 0x55543fc (0xf936 bytes) 2024-09-17T22:41:52.470 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:52.474 INFO:teuthology.orchestra.run.smithi115.stdout:1219 trunc from 0x65eaa66 to 0xa2f71fe 2024-09-17T22:41:52.479 INFO:teuthology.orchestra.run.smithi115.stdout:1220 punch from 0x5c10065 to 0x5c11d29, (0x1cc4 bytes) 2024-09-17T22:41:52.485 INFO:teuthology.orchestra.run.smithi115.stdout:1221 write 0xa7e800a thru 0xa7e8f86 (0xf7d bytes) 2024-09-17T22:41:52.514 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:52.514 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:52.523 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:41:52.560 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:52.558+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:52.626 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:52.626 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:52.635 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:41:52.678 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:52.676+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:52.737 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:52.737 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:52.745 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:41:52.762 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:52.759+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:52.773 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:41:52.863 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:52.861+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:52.889 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:52.889 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:52.897 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:41:52.965 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:52.962+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:53.001 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:53.001 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:53.010 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:41:53.096 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:53.092+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:53.155 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:53.155 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:53.164 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:41:53.205 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:53.203+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:53.267 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:53.264+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:53.308 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:53.308 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:53.317 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:41:53.367 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:53.365+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:53.421 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:53.421 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:53.430 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:41:53.468 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:53.466+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:53.569 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:53.567+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:53.573 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:53.573 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:53.582 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:41:53.670 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:53.668+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:53.686 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:53.686 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:53.695 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:41:53.771 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:53.768+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:53.797 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:53.797 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:53.806 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:41:53.872 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:53.870+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:53.956 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:53.957 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:53.965 INFO:teuthology.orchestra.run.smithi115.stdout:3.e 2024-09-17T22:41:53.965 INFO:teuthology.orchestra.run.smithi115.stdout:3.c 2024-09-17T22:41:53.965 INFO:teuthology.orchestra.run.smithi115.stdout:3.1d 2024-09-17T22:41:53.965 INFO:teuthology.orchestra.run.smithi115.stdout:3.1c 2024-09-17T22:41:53.965 INFO:teuthology.orchestra.run.smithi115.stdout:3.1b 2024-09-17T22:41:53.965 INFO:teuthology.orchestra.run.smithi115.stdout:3.15 2024-09-17T22:41:53.965 INFO:teuthology.orchestra.run.smithi115.stdout:3.13 2024-09-17T22:41:53.965 INFO:teuthology.orchestra.run.smithi115.stdout:3.f 2024-09-17T22:41:53.965 INFO:teuthology.orchestra.run.smithi115.stdout:2.7 2024-09-17T22:41:53.966 INFO:teuthology.orchestra.run.smithi115.stdout:3.3 2024-09-17T22:41:53.966 INFO:teuthology.orchestra.run.smithi115.stdout:2.6 2024-09-17T22:41:53.966 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:41:54.009 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:54.007+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:54.069 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:54.069 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:54.074 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:54.071+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:54.078 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:41:54.114 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:41:54.121 INFO:teuthology.orchestra.run.smithi115.stdout:1222 read 0x1dc5037 thru 0x1dccb4f (0x7b19 bytes) 2024-09-17T22:41:54.122 INFO:teuthology.orchestra.run.smithi115.stdout:1223 read 0xbf1297 thru 0xbf8aa5 (0x780f bytes) 2024-09-17T22:41:54.123 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.123 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.123 INFO:teuthology.orchestra.run.smithi115.stdout:1228 write 0x51fa27d thru 0x51fe606 (0x438a bytes) 2024-09-17T22:41:54.128 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.133 INFO:teuthology.orchestra.run.smithi115.stdout:1230 trunc from 0xa7e8f87 to 0x64b4ad5 2024-09-17T22:41:54.143 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.143 INFO:teuthology.orchestra.run.smithi115.stdout:1234 punch from 0x51f2366 to 0x51f9073, (0x6d0d bytes) 2024-09-17T22:41:54.143 INFO:teuthology.orchestra.run.smithi115.stdout:1235 read 0x5a1ae5f thru 0x5a1ef23 (0x40c5 bytes) 2024-09-17T22:41:54.143 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.143 INFO:teuthology.orchestra.run.smithi115.stdout:1237 write 0x175a1e0 thru 0x175fc7d (0x5a9e bytes) 2024-09-17T22:41:54.147 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.147 INFO:teuthology.orchestra.run.smithi115.stdout:1239 read 0x52c4b7b thru 0x52ced99 (0xa21f bytes) 2024-09-17T22:41:54.157 INFO:teuthology.orchestra.run.smithi115.stdout:1240 write 0x98dbf42 thru 0x98eaeca (0xef89 bytes) 2024-09-17T22:41:54.160 INFO:teuthology.orchestra.run.smithi115.stdout:1241 write 0x195dd80 thru 0x196b519 (0xd79a bytes) 2024-09-17T22:41:54.162 INFO:teuthology.orchestra.run.smithi115.stdout:1242 read 0x681fd86 thru 0x682882a (0x8aa5 bytes) 2024-09-17T22:41:54.162 INFO:teuthology.orchestra.run.smithi115.stdout:1243 read 0x38b3c64 thru 0x38b9dcc (0x6169 bytes) 2024-09-17T22:41:54.170 INFO:teuthology.orchestra.run.smithi115.stdout:1245 write 0xb267715 thru 0xb26a064 (0x2950 bytes) 2024-09-17T22:41:54.175 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:54.172+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:54.182 INFO:teuthology.orchestra.run.smithi115.stdout:1248 writesame 0xeb1248e thru 0xeb149b4 data_size 0x1(0x2527 bytes) 2024-09-17T22:41:54.182 INFO:teuthology.orchestra.run.smithi115.stdout:1249 write 0x5d9afca thru 0x5da3a58 (0x8a8f bytes) 2024-09-17T22:41:54.199 INFO:teuthology.orchestra.run.smithi115.stdout:1252 trunc from 0xeb149b5 to 0x1c4ae20 2024-09-17T22:41:54.217 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.235 INFO:teuthology.orchestra.run.smithi115.stdout:1254 write 0xd044eda thru 0xd047182 (0x22a9 bytes) 2024-09-17T22:41:54.239 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.239 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.240 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:54.240 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:54.248 INFO:teuthology.orchestra.run.smithi115.stdout:1257 trunc from 0xd047183 to 0x5a31660 2024-09-17T22:41:54.253 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:41:54.269 INFO:teuthology.orchestra.run.smithi115.stdout:1258 write 0xaa21371 thru 0xaa2f6c2 (0xe352 bytes) 2024-09-17T22:41:54.273 INFO:teuthology.orchestra.run.smithi115.stdout:1260 write 0x16ce23e thru 0x16d7d2f (0x9af2 bytes) 2024-09-17T22:41:54.276 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:54.274+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:54.278 INFO:teuthology.orchestra.run.smithi115.stdout:1262 trunc from 0xaa2f6c3 to 0xd449fc7 2024-09-17T22:41:54.283 INFO:teuthology.orchestra.run.smithi115.stdout:1264 writesame 0x8e5c1c7 thru 0x8e672a9 data_size 0x1(0xb0e3 bytes) 2024-09-17T22:41:54.306 INFO:teuthology.orchestra.run.smithi115.stdout:1265 writesame 0x3abd495 thru 0x3ac3dc8 data_size 0x2(0x6934 bytes) 2024-09-17T22:41:54.312 INFO:teuthology.orchestra.run.smithi115.stdout:1266 read 0x2b1d398 thru 0x2b2b7db (0xe444 bytes) 2024-09-17T22:41:54.317 INFO:teuthology.orchestra.run.smithi115.stdout:1268 trunc from 0xd449fc7 to 0x9870ca2 2024-09-17T22:41:54.324 INFO:teuthology.orchestra.run.smithi115.stdout:1269 write 0x40989d7 thru 0x409f60b (0x6c35 bytes) 2024-09-17T22:41:54.329 INFO:teuthology.orchestra.run.smithi115.stdout:1270 trunc from 0x9870ca2 to 0x7dc940a 2024-09-17T22:41:54.340 INFO:teuthology.orchestra.run.smithi115.stdout:1273 punch from 0x100125f to 0x100ad55, (0x9af6 bytes) 2024-09-17T22:41:54.340 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.340 INFO:teuthology.orchestra.run.smithi115.stdout:1279 punch from 0x1fcab70 to 0x1fd349f, (0x892f bytes) 2024-09-17T22:41:54.341 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.341 INFO:teuthology.orchestra.run.smithi115.stdout:1281 read 0x6037c43 thru 0x603b7c9 (0x3b87 bytes) 2024-09-17T22:41:54.341 INFO:teuthology.orchestra.run.smithi115.stdout:1282 read 0x75f5f93 thru 0x75fb78c (0x57fa bytes) 2024-09-17T22:41:54.346 INFO:teuthology.orchestra.run.smithi115.stdout:1288 trunc from 0x7dc940a to 0x2bebb10 2024-09-17T22:41:54.374 INFO:teuthology.orchestra.run.smithi115.stdout:1289 write 0xc2bfee6 thru 0xc2c5380 (0x549b bytes) 2024-09-17T22:41:54.377 INFO:teuthology.orchestra.run.smithi115.stdout:1290 write 0x97e62c1 thru 0x97f2b2d (0xc86d bytes) 2024-09-17T22:41:54.377 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:54.375+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:54.380 INFO:teuthology.orchestra.run.smithi115.stdout:1293 trunc from 0xc2c5381 to 0xbf6ce4b 2024-09-17T22:41:54.385 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:54.385 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:54.391 INFO:teuthology.orchestra.run.smithi115.stdout:1294 trunc from 0xbf6ce4b to 0xd7f69e6 2024-09-17T22:41:54.396 INFO:teuthology.orchestra.run.smithi115.stdout:1295 write 0xa94c975 thru 0xa953468 (0x6af4 bytes) 2024-09-17T22:41:54.397 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:41:54.399 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.401 INFO:teuthology.orchestra.run.smithi115.stdout:1297 trunc from 0xd7f69e6 to 0xc7594f8 2024-09-17T22:41:54.406 INFO:teuthology.orchestra.run.smithi115.stdout:1298 punch from 0x9ba78a5 to 0x9bb62b9, (0xea14 bytes) 2024-09-17T22:41:54.406 INFO:teuthology.orchestra.run.smithi115.stdout:1299 write 0x77e4a4c thru 0x77ef03e (0xa5f3 bytes) 2024-09-17T22:41:54.410 INFO:teuthology.orchestra.run.smithi115.stdout:1300 write 0x659972b thru 0x65a91ac (0xfa82 bytes) 2024-09-17T22:41:54.412 INFO:teuthology.orchestra.run.smithi115.stdout:1301 writesame 0x4595abe thru 0x459792f data_size 0x9(0x1e72 bytes) 2024-09-17T22:41:54.414 INFO:teuthology.orchestra.run.smithi115.stdout:1302 punch from 0xa437803 to 0xa43e5ba, (0x6db7 bytes) 2024-09-17T22:41:54.414 INFO:teuthology.orchestra.run.smithi115.stdout:1305 punch from 0xa5d0d27 to 0xa5e0403, (0xf6dc bytes) 2024-09-17T22:41:54.420 INFO:teuthology.orchestra.run.smithi115.stdout:1307 trunc from 0xc7594f8 to 0x7e32680 2024-09-17T22:41:54.436 INFO:teuthology.orchestra.run.smithi115.stdout:1308 trunc from 0x7e32680 to 0xc67da9d 2024-09-17T22:41:54.440 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.440 INFO:teuthology.orchestra.run.smithi115.stdout:1314 punch from 0x2d4cbea to 0x2d57050, (0xa466 bytes) 2024-09-17T22:41:54.441 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.441 INFO:teuthology.orchestra.run.smithi115.stdout:1316 read 0x9b2075e thru 0x9b2a5a5 (0x9e48 bytes) 2024-09-17T22:41:54.441 INFO:teuthology.orchestra.run.smithi115.stdout:1317 read 0xee1722 thru 0xee6e1c (0x56fb bytes) 2024-09-17T22:41:54.441 INFO:teuthology.orchestra.run.smithi115.stdout:1318 writesame 0xa588fd1 thru 0xa58d88e data_size 0x1(0x48be bytes) 2024-09-17T22:41:54.450 INFO:teuthology.orchestra.run.smithi115.stdout:1319 trunc from 0xc67da9d to 0xd72e5ae 2024-09-17T22:41:54.455 INFO:teuthology.orchestra.run.smithi115.stdout:1320 write 0x1bb8299 thru 0x1bb9520 (0x1288 bytes) 2024-09-17T22:41:54.457 INFO:teuthology.orchestra.run.smithi115.stdout:1321 read 0x94f19c4 thru 0x94f7539 (0x5b76 bytes) 2024-09-17T22:41:54.457 INFO:teuthology.orchestra.run.smithi115.stdout:1322 read 0x6de7b67 thru 0x6df1d57 (0xa1f1 bytes) 2024-09-17T22:41:54.457 INFO:teuthology.orchestra.run.smithi115.stdout:1325 read 0x8d0b56e thru 0x8d0d2c4 (0x1d57 bytes) 2024-09-17T22:41:54.457 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.457 INFO:teuthology.orchestra.run.smithi115.stdout:1327 writesame 0x7494660 thru 0x74a38ae data_size 0x57(0xf24f bytes) 2024-09-17T22:41:54.461 INFO:teuthology.orchestra.run.smithi115.stdout:1328 read 0x91d6dfc thru 0x91e13eb (0xa5f0 bytes) 2024-09-17T22:41:54.461 INFO:teuthology.orchestra.run.smithi115.stdout:1329 write 0xa50d81c thru 0xa50e764 (0xf49 bytes) 2024-09-17T22:41:54.473 INFO:teuthology.orchestra.run.smithi115.stdout:1330 trunc from 0xd72e5ae to 0x44857ff 2024-09-17T22:41:54.478 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:54.475+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:54.485 INFO:teuthology.orchestra.run.smithi115.stdout:1332 punch from 0x1036d88 to 0x103dff3, (0x726b bytes) 2024-09-17T22:41:54.485 INFO:teuthology.orchestra.run.smithi115.stdout:1333 write 0x17a5664 thru 0x17ab605 (0x5fa2 bytes) 2024-09-17T22:41:54.489 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op export --pgid 3.13 --file /var/log/ceph/exp.3.13.0 2024-09-17T22:41:54.500 INFO:teuthology.orchestra.run.smithi115.stdout:1337 write 0x910519a thru 0x91052c2 (0x129 bytes) 2024-09-17T22:41:54.502 INFO:teuthology.orchestra.run.smithi115.stdout:1338 writesame 0x18a7a8a thru 0x18b7968 data_size 0x27(0xfedf bytes) 2024-09-17T22:41:54.505 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.505 INFO:teuthology.orchestra.run.smithi115.stdout:1342 write 0x1275c3a thru 0x127fd3a (0xa101 bytes) 2024-09-17T22:41:54.507 INFO:teuthology.orchestra.run.smithi115.stdout:1343 read 0x6b9e55c thru 0x6ba4600 (0x60a5 bytes) 2024-09-17T22:41:54.507 INFO:teuthology.orchestra.run.smithi115.stdout:1344 write 0x63171a1 thru 0x6319d3d (0x2b9d bytes) 2024-09-17T22:41:54.509 INFO:teuthology.orchestra.run.smithi115.stdout:1345 write 0x8ce861d thru 0x8cedaf9 (0x54dd bytes) 2024-09-17T22:41:54.513 INFO:teuthology.orchestra.run.smithi115.stdout:1348 trunc from 0x91052c3 to 0x8495dfa 2024-09-17T22:41:54.519 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:54.519 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:54.524 INFO:teuthology.orchestra.run.smithi115.stdout:1349 read 0x83ddc56 thru 0x83e370b (0x5ab6 bytes) 2024-09-17T22:41:54.524 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.524 INFO:teuthology.orchestra.run.smithi115.stdout:1353 read 0x701aa1 thru 0x702382 (0x8e2 bytes) 2024-09-17T22:41:54.525 INFO:teuthology.orchestra.run.smithi115.stdout:1354 write 0x5f31983 thru 0x5f33589 (0x1c07 bytes) 2024-09-17T22:41:54.529 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:41:54.542 INFO:teuthology.orchestra.run.smithi115.stdout:1355 write 0xe303d31 thru 0xe3121c0 (0xe490 bytes) 2024-09-17T22:41:54.545 INFO:teuthology.orchestra.run.smithi115.stdout:1356 writesame 0x2a62338 thru 0x2a71e26 data_size 0x45(0xfaef bytes) 2024-09-17T22:41:54.546 INFO:teuthology.orchestra.run.smithi115.stdout:1357 writesame 0x57de22e thru 0x57e625c data_size 0x1(0x802f bytes) 2024-09-17T22:41:54.565 INFO:teuthology.orchestra.run.smithi115.stdout:1358 trunc from 0xe3121c1 to 0xccd781c 2024-09-17T22:41:54.571 INFO:teuthology.orchestra.run.smithi115.stdout:1360 read 0x8cfe33b thru 0x8d0c1bc (0xde82 bytes) 2024-09-17T22:41:54.572 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.572 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.572 INFO:teuthology.orchestra.run.smithi115.stdout:1363 write 0x1935273 thru 0x193d7d2 (0x8560 bytes) 2024-09-17T22:41:54.573 INFO:teuthology.orchestra.run.smithi115.stdout:1364 read 0x2177b01 thru 0x2180a13 (0x8f13 bytes) 2024-09-17T22:41:54.574 INFO:teuthology.orchestra.run.smithi115.stdout:1365 write 0x7875848 thru 0x7877a61 (0x221a bytes) 2024-09-17T22:41:54.578 INFO:teuthology.orchestra.run.smithi115.stdout:1366 write 0x83c986d thru 0x83d042b (0x6bbf bytes) 2024-09-17T22:41:54.578 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:54.576+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:54.581 INFO:teuthology.orchestra.run.smithi115.stdout:1367 write 0x2b2cd7 thru 0x2c0c1c (0xdf46 bytes) 2024-09-17T22:41:54.582 INFO:teuthology.orchestra.run.smithi115.stdout:1369 write 0x214e2be thru 0x215503d (0x6d80 bytes) 2024-09-17T22:41:54.594 INFO:teuthology.orchestra.run.smithi115.stdout:1375 trunc from 0xccd781c to 0x3eecf33 2024-09-17T22:41:54.613 INFO:teuthology.orchestra.run.smithi115.stdout:1376 write 0x64d6d18 thru 0x64dc0ef (0x53d8 bytes) 2024-09-17T22:41:54.622 INFO:teuthology.orchestra.run.smithi115.stdout:1377 write 0x7654e0a thru 0x765ad0f (0x5f06 bytes) 2024-09-17T22:41:54.636 INFO:teuthology.orchestra.run.smithi115.stdout:1378 writesame 0xc70d495 thru 0xc711e67 data_size 0x1(0x49d3 bytes) 2024-09-17T22:41:54.636 INFO:teuthology.orchestra.run.smithi115.stdout:1381 write 0x12c5809 thru 0x12c7503 (0x1cfb bytes) 2024-09-17T22:41:54.637 INFO:teuthology.orchestra.run.smithi115.stdout:1383 read 0x1105d4b thru 0x11159ae (0xfc64 bytes) 2024-09-17T22:41:54.638 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.639 INFO:teuthology.orchestra.run.smithi115.stdout:1385 punch from 0x70eae0e to 0x70f35f7, (0x87e9 bytes) 2024-09-17T22:41:54.639 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:54.639 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:54.646 INFO:teuthology.orchestra.run.smithi115.stdout:1388 write 0xe315400 thru 0xe31fa17 (0xa618 bytes) 2024-09-17T22:41:54.648 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:41:54.649 INFO:teuthology.orchestra.run.smithi115.stdout:1389 write 0x8273965 thru 0x8277617 (0x3cb3 bytes) 2024-09-17T22:41:54.650 INFO:teuthology.orchestra.run.smithi115.stdout:1390 write 0x7c7ab4d thru 0x7c7d68d (0x2b41 bytes) 2024-09-17T22:41:54.651 INFO:teuthology.orchestra.run.smithi115.stdout:1391 read 0x568dd3c thru 0x5696177 (0x843c bytes) 2024-09-17T22:41:54.651 INFO:teuthology.orchestra.run.smithi115.stdout:1392 read 0x8e8148d thru 0x8e88c7a (0x77ee bytes) 2024-09-17T22:41:54.652 INFO:teuthology.orchestra.run.smithi115.stdout:1393 punch from 0x7ff44af to 0x7ffc3f7, (0x7f48 bytes) 2024-09-17T22:41:54.652 INFO:teuthology.orchestra.run.smithi115.stdout:1394 read 0xd8e2056 thru 0xd8ed6a4 (0xb64f bytes) 2024-09-17T22:41:54.652 INFO:teuthology.orchestra.run.smithi115.stdout:1395 write 0x66434e0 thru 0x664ceb8 (0x99d9 bytes) 2024-09-17T22:41:54.655 INFO:teuthology.orchestra.run.smithi115.stdout:1397 writesame 0x8bc49ce thru 0x8bc928f data_size 0x2(0x48c2 bytes) 2024-09-17T22:41:54.660 INFO:teuthology.orchestra.run.smithi115.stdout:1398 read 0x37996e7 thru 0x379cac2 (0x33dc bytes) 2024-09-17T22:41:54.660 INFO:teuthology.orchestra.run.smithi115.stdout:1399 read 0xba1f446 thru 0xba2e583 (0xf13e bytes) 2024-09-17T22:41:54.660 INFO:teuthology.orchestra.run.smithi115.stdout:1400 write 0xcac65c8 thru 0xcad1de8 (0xb821 bytes) 2024-09-17T22:41:54.662 INFO:teuthology.orchestra.run.smithi115.stdout:1401 punch from 0x859f4e9 to 0x85aad98, (0xb8af bytes) 2024-09-17T22:41:54.672 INFO:teuthology.orchestra.run.smithi115.stdout:1402 trunc from 0xe31fa18 to 0x588dbe8 2024-09-17T22:41:54.685 INFO:teuthology.orchestra.run.smithi115.stdout:1403 trunc from 0x588dbe8 to 0x20bcd38 2024-09-17T22:41:54.689 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:54.687+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:54.710 INFO:teuthology.orchestra.run.smithi115.stdout:1407 writesame 0x92bae0a thru 0x92c9c9a data_size 0x1(0xee91 bytes) 2024-09-17T22:41:54.711 INFO:teuthology.orchestra.run.smithi115.stdout:1409 trunc from 0x92c9c9b to 0x8893496 2024-09-17T22:41:54.724 INFO:teuthology.orchestra.run.smithi115.stdout:1410 trunc from 0x8893496 to 0x196f9cf 2024-09-17T22:41:54.732 INFO:teuthology.orchestra.run.smithi115.stdout:1411 trunc from 0x196f9cf to 0x2798c34 2024-09-17T22:41:54.737 INFO:teuthology.orchestra.run.smithi115.stdout:1413 read 0xb71a87 thru 0xb793d6 (0x7950 bytes) 2024-09-17T22:41:54.747 INFO:teuthology.orchestra.run.smithi115.stdout:1414 write 0x63d2889 thru 0x63dc0e1 (0x9859 bytes) 2024-09-17T22:41:54.750 INFO:teuthology.orchestra.run.smithi115.stdout:1415 punch from 0x2887838 to 0x2896e4c, (0xf614 bytes) 2024-09-17T22:41:54.753 INFO:teuthology.orchestra.run.smithi115.stdout:1417 trunc from 0x63dc0e2 to 0x89dbacb 2024-09-17T22:41:54.758 INFO:teuthology.orchestra.run.smithi115.stdout:1418 punch from 0x245a123 to 0x2468cfc, (0xebd9 bytes) 2024-09-17T22:41:54.758 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.758 INFO:teuthology.orchestra.run.smithi115.stdout:1420 read 0x60c7cc5 thru 0x60d2a1a (0xad56 bytes) 2024-09-17T22:41:54.769 INFO:teuthology.orchestra.run.smithi115.stdout:1421 writesame 0xd3484a9 thru 0xd354c0e data_size 0x1(0xc766 bytes) 2024-09-17T22:41:54.769 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.769 INFO:teuthology.orchestra.run.smithi115.stdout:1423 write 0x75681d2 thru 0x756d7c6 (0x55f5 bytes) 2024-09-17T22:41:54.772 INFO:teuthology.orchestra.run.smithi115.stdout:1424 read 0xab773a6 thru 0xab84a54 (0xd6af bytes) 2024-09-17T22:41:54.772 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.772 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.779 INFO:teuthology.orchestra.run.smithi115.stdout:1427 trunc from 0xd354c0f to 0x716e71a 2024-09-17T22:41:54.779 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:54.777+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:54.784 INFO:teuthology.orchestra.run.smithi115.stdout:1428 read 0x6a76706 thru 0x6a80f4a (0xa845 bytes) 2024-09-17T22:41:54.785 INFO:teuthology.orchestra.run.smithi115.stdout:1429 trunc from 0x716e71a to 0x76861a2 2024-09-17T22:41:54.789 INFO:teuthology.orchestra.run.smithi115.stdout:1430 read 0x9e9442 thru 0x9f9393 (0xff52 bytes) 2024-09-17T22:41:54.790 INFO:teuthology.orchestra.run.smithi115.stdout:1431 read 0x109b839 thru 0x109c58d (0xd55 bytes) 2024-09-17T22:41:54.799 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:54.799 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:54.801 INFO:teuthology.orchestra.run.smithi115.stdout:1432 writesame 0xbd43c48 thru 0xbd4d38f data_size 0x2f(0x9748 bytes) 2024-09-17T22:41:54.801 INFO:teuthology.orchestra.run.smithi115.stdout:1433 read 0x98b70b thru 0x98ce00 (0x16f6 bytes) 2024-09-17T22:41:54.801 INFO:teuthology.orchestra.run.smithi115.stdout:1434 punch from 0x351fb05 to 0x35289e9, (0x8ee4 bytes) 2024-09-17T22:41:54.801 INFO:teuthology.orchestra.run.smithi115.stdout:1435 trunc from 0xbd4d390 to 0xbd3ab01 2024-09-17T22:41:54.805 INFO:teuthology.orchestra.run.smithi115.stdout:1436 read 0x8d75981 thru 0x8d7f957 (0x9fd7 bytes) 2024-09-17T22:41:54.806 INFO:teuthology.orchestra.run.smithi115.stdout:1438 writesame 0x4bb4cb7 thru 0x4bbc9e9 data_size 0x1(0x7d33 bytes) 2024-09-17T22:41:54.808 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:41:54.822 INFO:teuthology.orchestra.run.smithi115.stdout:1439 read 0xfbb7e9 thru 0xfc3685 (0x7e9d bytes) 2024-09-17T22:41:54.822 INFO:teuthology.orchestra.run.smithi115.stdout:1440 punch from 0x646b25e to 0x64717b0, (0x6552 bytes) 2024-09-17T22:41:54.823 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.823 INFO:teuthology.orchestra.run.smithi115.stdout:1442 read 0x99acc0f thru 0x99b264f (0x5a41 bytes) 2024-09-17T22:41:54.827 INFO:teuthology.orchestra.run.smithi115.stdout:1443 trunc from 0xbd3ab01 to 0x7d620cf 2024-09-17T22:41:54.834 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.836 INFO:teuthology.orchestra.run.smithi115.stdout:1445 trunc from 0x7d620cf to 0x604a330 2024-09-17T22:41:54.850 INFO:teuthology.orchestra.run.smithi115.stdout:1446 trunc from 0x604a330 to 0xb0df635 2024-09-17T22:41:54.855 INFO:teuthology.orchestra.run.smithi115.stdout:1447 punch from 0x49592ac to 0x495bd07, (0x2a5b bytes) 2024-09-17T22:41:54.855 INFO:teuthology.orchestra.run.smithi115.stdout:1448 write 0x7ce9bfe thru 0x7cf6e52 (0xd255 bytes) 2024-09-17T22:41:54.858 INFO:teuthology.orchestra.run.smithi115.stdout:1449 write 0x205a921 thru 0x2067282 (0xc962 bytes) 2024-09-17T22:41:54.860 INFO:teuthology.orchestra.run.smithi115.stdout:1451 writesame 0x7031323 thru 0x703e2e5 data_size 0x1(0xcfc3 bytes) 2024-09-17T22:41:54.881 INFO:teuthology.orchestra.run.smithi115.stdout:1453 trunc from 0xb0df635 to 0xaa4afd4 2024-09-17T22:41:54.886 INFO:teuthology.orchestra.run.smithi115.stdout:1455 write 0x20eff9 thru 0x21452a (0x5532 bytes) 2024-09-17T22:41:54.887 INFO:teuthology.orchestra.run.smithi115.stdout:1456 read 0x65e8119 thru 0x65eeaf9 (0x69e1 bytes) 2024-09-17T22:41:54.892 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:54.890+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:54.897 INFO:teuthology.orchestra.run.smithi115.stdout:1458 write 0xcc5f4ed thru 0xcc6ebe6 (0xf6fa bytes) 2024-09-17T22:41:54.900 INFO:teuthology.orchestra.run.smithi115.stdout:1460 writesame 0x2b54a04 thru 0x2b5be28 data_size 0x35(0x7425 bytes) 2024-09-17T22:41:54.902 INFO:teuthology.orchestra.run.smithi115.stdout:1461 read 0x6c1b234 thru 0x6c1f7b0 (0x457d bytes) 2024-09-17T22:41:54.902 INFO:teuthology.orchestra.run.smithi115.stdout:1462 write 0x12c72f3 thru 0x12c753b (0x249 bytes) 2024-09-17T22:41:54.904 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.904 INFO:teuthology.orchestra.run.smithi115.stdout:1467 read 0x3f6468b thru 0x3f6dfe1 (0x9957 bytes) 2024-09-17T22:41:54.905 INFO:teuthology.orchestra.run.smithi115.stdout:1470 writesame 0x3ab2ef0 thru 0x3ac0867 data_size 0x4(0xd978 bytes) 2024-09-17T22:41:54.919 INFO:teuthology.orchestra.run.smithi115.stdout:1472 write 0xd852572 thru 0xd859013 (0x6aa2 bytes) 2024-09-17T22:41:54.920 INFO:teuthology.orchestra.run.smithi115.stdout:1473 read 0x5e0001e thru 0x5e0a1e0 (0xa1c3 bytes) 2024-09-17T22:41:54.920 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.920 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:54.920 INFO:teuthology.orchestra.run.smithi115.stdout:1478 clone 16 order 19 su 262144 sc 13 2024-09-17T22:41:54.955 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:54.956 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:54.964 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:41:55.005 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:55.003+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:55.082 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:55.080+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:55.108 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:55.108 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:55.117 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:41:55.122 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is set 2024-09-17T22:41:55.201 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:55.199+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:55.260 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:55.260 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:55.269 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:41:55.309 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:55.307+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:55.385 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:55.383+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:55.411 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:55.411 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:55.420 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:41:55.486 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:55.484+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:55.523 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:55.523 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:55.532 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:41:55.586 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:55.584+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:55.636 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:55.636 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:55.645 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:41:55.687 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:55.684+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:55.787 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:55.785+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:55.789 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:55.789 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:55.798 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:41:55.888 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:55.885+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:55.890 INFO:teuthology.orchestra.run.smithi115.stderr:Exporting 3.13 info 3.13( v 50'249 (0'0,50'249] local-lis/les=23/24 n=32 ec=23/23 lis/c=23/23 les/c/f=24/24/0 sis=23) 2024-09-17T22:41:55.899 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:c828dad6:::rbd_data.115fbc01575.0000000000000073:head# 2024-09-17T22:41:55.899 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:c879d723:::rbd_data.115fe0c30ef6.0000000000000059:head# 2024-09-17T22:41:55.899 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:c905ebf9:::rbd_data.115fe0c30ef6.0000000000000016:head# 2024-09-17T22:41:55.899 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:c90a79a2:::rbd_data.115fe0c30ef6.000000000000005c:head# 2024-09-17T22:41:55.899 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:c94dd37f:::rbd_data.115fe0c30ef6.00000000000001af:5# 2024-09-17T22:41:55.899 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:c94dd37f:::rbd_data.115fe0c30ef6.00000000000001af:head# 2024-09-17T22:41:55.900 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:c9ce73c5:::rbd_data.115fe0c30ef6.000000000000004b:head# 2024-09-17T22:41:55.900 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:c9ed77b7:::rbd_data.115ff9e8d63c.00000000000000aa:f# 2024-09-17T22:41:55.902 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:c9ed77b7:::rbd_data.115ff9e8d63c.00000000000000aa:head# 2024-09-17T22:41:55.902 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:c9f51540:::rbd_data.115f450dd2d3.000000000000023a:d# 2024-09-17T22:41:55.902 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:c9f51540:::rbd_data.115f450dd2d3.000000000000023a:head# 2024-09-17T22:41:55.902 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:ca2b7776:::rbd_data.115fbc01575.0000000000000095:b# 2024-09-17T22:41:55.903 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:ca2b7776:::rbd_data.115fbc01575.0000000000000095:head# 2024-09-17T22:41:55.903 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:cacd6ade:::rbd_data.115fe0c30ef6.00000000000001ae:5# 2024-09-17T22:41:55.903 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:cacd6ade:::rbd_data.115fe0c30ef6.00000000000001ae:head# 2024-09-17T22:41:55.903 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:cb013b4b:::rbd_data.115fe0c30ef6.000000000000008d:head# 2024-09-17T22:41:55.903 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:cb9daead:::rbd_object_map.115f86d8437d:head# 2024-09-17T22:41:55.903 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:cbac0069:::rbd_data.115f450dd2d3.000000000000005b:head# 2024-09-17T22:41:55.903 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:cc5087aa:::rbd_data.115fabba65f5.0000000000000004:head# 2024-09-17T22:41:55.903 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:cc524d94:::rbd_data.115fc131703f.0000000000000040:head# 2024-09-17T22:41:55.903 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:ccb197fc:::rbd_data.115f2bd7bdd2.0000000000000005:head# 2024-09-17T22:41:55.910 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:ccbe4cb7:::rbd_id.image_client.0-clone10:head# 2024-09-17T22:41:55.911 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:cd38a6e0:::rbd_data.115fe0c30ef6.000000000000008b:head# 2024-09-17T22:41:55.911 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:cd4faa88:::rbd_object_map.115f2bd7bdd2:head# 2024-09-17T22:41:55.911 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:cd72b7a5:::rbd_data.115fabba65f5.0000000000000066:head# 2024-09-17T22:41:55.911 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:cdcab361:::rbd_data.115fe0c30ef6.0000000000000027:head# 2024-09-17T22:41:55.911 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:cef44135:::rbd_data.115f450dd2d3.0000000000000036:head# 2024-09-17T22:41:55.911 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:cf231ad5:::rbd_data.115fe0c30ef6.00000000000000b9:head# 2024-09-17T22:41:55.911 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:cf679955:::rbd_data.115f450dd2d3.0000000000000055:head# 2024-09-17T22:41:55.911 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:cf856a34:::rbd_data.115fe0c30ef6.000000000000008c:head# 2024-09-17T22:41:55.911 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:cfb9cce5:::rbd_data.115fe0c30ef6.000000000000001d:5# 2024-09-17T22:41:55.911 INFO:teuthology.orchestra.run.smithi115.stderr:Read #3:cfb9cce5:::rbd_data.115fe0c30ef6.000000000000001d:head# 2024-09-17T22:41:55.911 INFO:teuthology.orchestra.run.smithi115.stderr:Export successful 2024-09-17T22:41:55.941 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:55.941 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:55.953 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:41:55.996 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:55.994+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:56.057 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:56.057 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:56.066 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:41:56.089 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:56.087+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:56.190 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:56.188+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:56.210 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:56.210 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:56.220 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:41:56.291 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:56.289+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:56.330 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:56.330 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:56.339 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:41:56.423 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:56.421+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:56.434 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --force --op remove --pgid 3.13 2024-09-17T22:41:56.484 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:56.484 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:56.493 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:41:56.493 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:56.490+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:56.593 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:56.590+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:56.638 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:56.638 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:56.646 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:41:56.694 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:56.692+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:56.790 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:56.790 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:56.795 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:56.792+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:56.798 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:41:56.896 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:56.893+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:56.942 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:56.942 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:56.951 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:41:56.996 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:56.994+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:57.094 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:57.094 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:57.097 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:57.095+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:57.103 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:41:57.139 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:41:57.195 INFO:teuthology.orchestra.run.smithi115.stdout:flattening image image_client.0-clone15 2024-09-17T22:41:57.198 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:57.196+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:57.265 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:57.265 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:57.280 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:41:57.323 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:57.320+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:57.351 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-17T22:41:57.353 INFO:teuthology.orchestra.run.smithi115.stdout:1480 write 0x12e1760 thru 0x12e929a (0x7b3b bytes) 2024-09-17T22:41:57.360 INFO:teuthology.orchestra.run.smithi115.stdout:1481 read 0x6546408 thru 0x65532dd (0xced6 bytes) 2024-09-17T22:41:57.361 INFO:teuthology.orchestra.run.smithi115.stdout:1482 read 0x310bf24 thru 0x31107bb (0x4898 bytes) 2024-09-17T22:41:57.361 INFO:teuthology.orchestra.run.smithi115.stdout:1485 punch from 0xa0639b2 to 0xa06c7bb, (0x8e09 bytes) 2024-09-17T22:41:57.362 INFO:teuthology.orchestra.run.smithi115.stdout:1486 read 0x931795a thru 0x93209aa (0x9051 bytes) 2024-09-17T22:41:57.362 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:57.376 INFO:teuthology.orchestra.run.smithi115.stdout:1488 trunc from 0xd859014 to 0xa2bc93 2024-09-17T22:41:57.400 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:57.397+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:57.405 INFO:teuthology.orchestra.run.smithi115.stdout:1489 read 0x76ff9a thru 0x77d5d7 (0xd63e bytes) 2024-09-17T22:41:57.405 INFO:teuthology.orchestra.run.smithi115.stdout:1490 read 0x9fab30 thru 0xa08605 (0xdad6 bytes) 2024-09-17T22:41:57.406 INFO:teuthology.orchestra.run.smithi115.stdout:1491 punch from 0x8fd7d7 to 0x8fe02d, (0x856 bytes) 2024-09-17T22:41:57.406 INFO:teuthology.orchestra.run.smithi115.stdout:1492 punch from 0x9832ed to 0x98ccfd, (0x9a10 bytes) 2024-09-17T22:41:57.406 INFO:teuthology.orchestra.run.smithi115.stdout:1493 read 0x1ff0a thru 0x2d931 (0xda28 bytes) 2024-09-17T22:41:57.407 INFO:teuthology.orchestra.run.smithi115.stdout:1496 punch from 0x7ab7bd to 0x7baa87, (0xf2ca bytes) 2024-09-17T22:41:57.407 INFO:teuthology.orchestra.run.smithi115.stdout:1497 punch from 0x68b7c4 to 0x698c72, (0xd4ae bytes) 2024-09-17T22:41:57.407 INFO:teuthology.orchestra.run.smithi115.stdout:1498 punch from 0xa1c914 to 0xa218fa, (0x4fe6 bytes) 2024-09-17T22:41:57.429 INFO:teuthology.orchestra.run.smithi115.stdout:1499 write 0xcc8f0cf thru 0xcc94577 (0x54a9 bytes) 2024-09-17T22:41:57.439 INFO:teuthology.orchestra.run.smithi115.stdout:1503 write 0xdc67537 thru 0xdc6c9aa (0x5474 bytes) 2024-09-17T22:41:57.443 INFO:teuthology.orchestra.run.smithi115.stdout:1504 trunc from 0xdc6c9ab to 0xd192254 2024-09-17T22:41:57.451 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:57.451 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:57.453 INFO:teuthology.orchestra.run.smithi115.stdout:1505 writesame 0x2d070e thru 0x2d47d3 data_size 0x1(0x40c6 bytes) 2024-09-17T22:41:57.460 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:41:57.467 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:57.467 INFO:teuthology.orchestra.run.smithi115.stdout:1508 writesame 0xb57c60f thru 0xb5817a1 data_size 0x1(0x5193 bytes) 2024-09-17T22:41:57.475 INFO:teuthology.orchestra.run.smithi115.stdout:1509 read 0x8bf5682 thru 0x8bfcee0 (0x785f bytes) 2024-09-17T22:41:57.475 INFO:teuthology.orchestra.run.smithi115.stdout:1512 write 0x6c86b22 thru 0x6c95c9b (0xf17a bytes) 2024-09-17T22:41:57.479 INFO:teuthology.orchestra.run.smithi115.stdout:1514 punch from 0xd0e60f3 to 0xd0e7f8b, (0x1e98 bytes) 2024-09-17T22:41:57.479 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:57.479 INFO:teuthology.orchestra.run.smithi115.stdout:1517 write 0x193e957 thru 0x194dd52 (0xf3fc bytes) 2024-09-17T22:41:57.483 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:57.483 INFO:teuthology.orchestra.run.smithi115.stdout:1519 writesame 0x5217286 thru 0x52227c0 data_size 0xf(0xb53b bytes) 2024-09-17T22:41:57.486 INFO:teuthology.orchestra.run.smithi115.stdout:1520 write 0x92b7b29 thru 0x92bcf2e (0x5406 bytes) 2024-09-17T22:41:57.489 INFO:teuthology.orchestra.run.smithi115.stdout:1521 write 0x2ffa7e8 thru 0x300263a (0x7e53 bytes) 2024-09-17T22:41:57.492 INFO:teuthology.orchestra.run.smithi115.stdout:1522 writesame 0x935b496 thru 0x9365647 data_size 0x3(0xa1b2 bytes) 2024-09-17T22:41:57.500 INFO:teuthology.orchestra.run.smithi115.stdout:1524 write 0x7c64590 thru 0x7c68419 (0x3e8a bytes) 2024-09-17T22:41:57.501 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:41:57.498+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:41:57.504 INFO:teuthology.orchestra.run.smithi115.stdout:1525 punch from 0xa1aa701 to 0xa1b1443, (0x6d42 bytes) 2024-09-17T22:41:57.504 INFO:teuthology.orchestra.run.smithi115.stdout:1526 read 0xa9b30e2 thru 0xa9b32fa (0x219 bytes) 2024-09-17T22:41:57.504 INFO:teuthology.orchestra.run.smithi115.stdout:1528 punch from 0x81c4db2 to 0x81c631a, (0x1568 bytes) 2024-09-17T22:41:57.504 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:57.504 INFO:teuthology.orchestra.run.smithi115.stdout:1531 write 0x2ad7f31 thru 0x2ade299 (0x6369 bytes) 2024-09-17T22:41:57.506 INFO:teuthology.orchestra.run.smithi115.stdout:1532 read 0x5381f0f thru 0x538419d (0x228f bytes) 2024-09-17T22:41:57.513 INFO:teuthology.orchestra.run.smithi115.stdout:1534 trunc from 0xd192254 to 0x7966d77 2024-09-17T22:41:57.522 INFO:teuthology.orchestra.run.smithi115.stdout:1535 read 0x71aaff9 thru 0x71b42e0 (0x92e8 bytes) 2024-09-17T22:41:57.522 INFO:teuthology.orchestra.run.smithi115.stdout:1536 read 0x55508c3 thru 0x55536c1 (0x2dff bytes) 2024-09-17T22:41:57.530 INFO:teuthology.orchestra.run.smithi115.stdout:1537 trunc from 0x7966d77 to 0x90e0dc 2024-09-17T22:41:57.551 INFO:teuthology.orchestra.run.smithi115.stdout:1538 read 0x6b9c3b thru 0x6c6453 (0xc819 bytes) 2024-09-17T22:41:57.569 INFO:teuthology.orchestra.run.smithi115.stdout:1539 write 0x9ec7515 thru 0x9ed0e53 (0x993f bytes) 2024-09-17T22:41:57.572 INFO:teuthology.orchestra.run.smithi115.stdout:1541 read 0x60099ec thru 0x6015beb (0xc200 bytes) 2024-09-17T22:41:57.572 INFO:teuthology.orchestra.run.smithi115.stdout:1542 read 0x5b1683b thru 0x5b1c8c4 (0x608a bytes) 2024-09-17T22:41:57.572 INFO:teuthology.orchestra.run.smithi115.stdout:1545 read 0x2b4b5df thru 0x2b549ab (0x93cd bytes) 2024-09-17T22:41:57.575 INFO:teuthology.orchestra.run.smithi115.stdout:1546 trunc from 0x9ed0e54 to 0xcaf6ee6 2024-09-17T22:41:57.589 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:57.589 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:57.592 INFO:teuthology.orchestra.run.smithi115.stdout:1547 trunc from 0xcaf6ee6 to 0x2dc0ca0 2024-09-17T22:41:57.600 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:41:57.601 INFO:teuthology.orchestra.run.smithi115.stdout:1548 read 0x1de5101 thru 0x1deb666 (0x6566 bytes) 2024-09-17T22:41:57.602 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:57.599+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:57.612 INFO:teuthology.orchestra.run.smithi115.stdout:1549 trunc from 0x2dc0ca0 to 0xc8fba69 2024-09-17T22:41:57.617 INFO:teuthology.orchestra.run.smithi115.stdout:1551 read 0x66f751 thru 0x67eeec (0xf79c bytes) 2024-09-17T22:41:57.618 INFO:teuthology.orchestra.run.smithi115.stdout:1553 read 0x8002a10 thru 0x800f62c (0xcc1d bytes) 2024-09-17T22:41:57.619 INFO:teuthology.orchestra.run.smithi115.stdout:1554 write 0x6c5dfcb thru 0x6c603b1 (0x23e7 bytes) 2024-09-17T22:41:57.621 INFO:teuthology.orchestra.run.smithi115.stdout:1555 write 0x6241724 thru 0x6241c24 (0x501 bytes) 2024-09-17T22:41:57.624 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:57.624 INFO:teuthology.orchestra.run.smithi115.stdout:1557 read 0x5bc2776 thru 0x5bd253f (0xfdca bytes) 2024-09-17T22:41:57.625 INFO:teuthology.orchestra.run.smithi115.stdout:1558 read 0x4636d93 thru 0x4638021 (0x128f bytes) 2024-09-17T22:41:57.631 INFO:teuthology.orchestra.run.smithi115.stdout:1559 write 0xdedc7b3 thru 0xdee822b (0xba79 bytes) 2024-09-17T22:41:57.634 INFO:teuthology.orchestra.run.smithi115.stdout:1560 punch from 0x2107d8c to 0x2109598, (0x180c bytes) 2024-09-17T22:41:57.634 INFO:teuthology.orchestra.run.smithi115.stdout:1561 read 0x4779ece thru 0x477c53e (0x2671 bytes) 2024-09-17T22:41:57.634 INFO:teuthology.orchestra.run.smithi115.stdout:1562 read 0x95bc6be thru 0x95c0819 (0x415c bytes) 2024-09-17T22:41:57.640 INFO:teuthology.orchestra.run.smithi115.stdout:1565 writesame 0xece18f4 thru 0xece28f5 data_size 0x3(0x1002 bytes) 2024-09-17T22:41:57.640 INFO:teuthology.orchestra.run.smithi115.stdout:1567 punch from 0xbdb2522 to 0xbdb33f1, (0xecf bytes) 2024-09-17T22:41:57.640 INFO:teuthology.orchestra.run.smithi115.stdout:1568 punch from 0x30799e6 to 0x30804b2, (0x6acc bytes) 2024-09-17T22:41:57.640 INFO:teuthology.orchestra.run.smithi115.stdout:1569 writesame 0x7a9ac65 thru 0x7a9c0bb data_size 0x1(0x1457 bytes) 2024-09-17T22:41:57.645 INFO:teuthology.orchestra.run.smithi115.stdout:1570 write 0xa325fb0 thru 0xa32fb79 (0x9bca bytes) 2024-09-17T22:41:57.648 INFO:teuthology.orchestra.run.smithi115.stdout:1573 read 0x53b232 thru 0x5441bd (0x8f8c bytes) 2024-09-17T22:41:57.649 INFO:teuthology.orchestra.run.smithi115.stdout:1574 read 0x5f89d80 thru 0x5f919cb (0x7c4c bytes) 2024-09-17T22:41:57.649 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:57.649 INFO:teuthology.orchestra.run.smithi115.stdout:1577 read 0x63896c thru 0x64239b (0x9a30 bytes) 2024-09-17T22:41:57.649 INFO:teuthology.orchestra.run.smithi115.stdout:1578 read 0x36c9734 thru 0x36d3b99 (0xa466 bytes) 2024-09-17T22:41:57.649 INFO:teuthology.orchestra.run.smithi115.stdout:1580 write 0xd3b26f6 thru 0xd3bee8f (0xc79a bytes) 2024-09-17T22:41:57.659 INFO:teuthology.orchestra.run.smithi115.stdout:1581 trunc from 0xece28f6 to 0x9ffa915 2024-09-17T22:41:57.668 INFO:teuthology.orchestra.run.smithi115.stdout:1584 writesame 0x8117f6b thru 0x812004e data_size 0x92(0x80e4 bytes) 2024-09-17T22:41:57.674 INFO:teuthology.orchestra.run.smithi115.stdout:1585 trunc from 0x9ffa915 to 0x777d91d 2024-09-17T22:41:57.693 INFO:teuthology.orchestra.run.smithi115.stdout:1586 write 0xb38aea2 thru 0xb391fe7 (0x7146 bytes) 2024-09-17T22:41:57.696 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:57.696 INFO:teuthology.orchestra.run.smithi115.stdout:1588 read 0x71029c9 thru 0x710f9ef (0xd027 bytes) 2024-09-17T22:41:57.696 INFO:teuthology.orchestra.run.smithi115.stdout:1591 read 0xa300704 thru 0xa30d630 (0xcf2d bytes) 2024-09-17T22:41:57.696 INFO:teuthology.orchestra.run.smithi115.stdout:1594 read 0x70431d9 thru 0x704372e (0x556 bytes) 2024-09-17T22:41:57.696 INFO:teuthology.orchestra.run.smithi115.stdout:1596 write 0x1d156ff thru 0x1d22c2c (0xd52e bytes) 2024-09-17T22:41:57.699 INFO:teuthology.orchestra.run.smithi115.stdout:1597 punch from 0x7e214c1 to 0x7e22618, (0x1157 bytes) 2024-09-17T22:41:57.699 INFO:teuthology.orchestra.run.smithi115.stdout:1598 read 0x8b40389 thru 0x8b4d09a (0xcd12 bytes) 2024-09-17T22:41:57.699 INFO:teuthology.orchestra.run.smithi115.stdout:1599 read 0x570c09e thru 0x5716298 (0xa1fb bytes) 2024-09-17T22:41:57.699 INFO:teuthology.orchestra.run.smithi115.stdout:1601 write 0xaf0d468 thru 0xaf0dee3 (0xa7c bytes) 2024-09-17T22:41:57.702 INFO:teuthology.orchestra.run.smithi115.stdout:1602 punch from 0xa3cd832 to 0xa3dc923, (0xf0f1 bytes) 2024-09-17T22:41:57.702 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:57.699+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:57.709 INFO:teuthology.orchestra.run.smithi115.stdout:1604 writesame 0xd41409c thru 0xd419ea3 data_size 0x76(0x5e08 bytes) 2024-09-17T22:41:57.709 INFO:teuthology.orchestra.run.smithi115.stdout:1606 read 0x325a4e9 thru 0x3261372 (0x6e8a bytes) 2024-09-17T22:41:57.709 INFO:teuthology.orchestra.run.smithi115.stdout:1607 read 0x1f90466 thru 0x1f975a1 (0x713c bytes) 2024-09-17T22:41:57.709 INFO:teuthology.orchestra.run.smithi115.stdout:1609 read 0x3f22ee2 thru 0x3f2fbd6 (0xccf5 bytes) 2024-09-17T22:41:57.710 INFO:teuthology.orchestra.run.smithi115.stdout:1611 writesame 0x3d61ebc thru 0x3d6694e data_size 0x1(0x4a93 bytes) 2024-09-17T22:41:57.710 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:57.711 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:57.721 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:57.721 INFO:teuthology.orchestra.run.smithi115.stdout:1613 writesame 0x9c25f34 thru 0x9c3047f data_size 0x4(0xa54c bytes) 2024-09-17T22:41:57.722 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:41:57.726 INFO:teuthology.orchestra.run.smithi115.stdout:1615 trunc from 0xd419ea4 to 0xde0d601 2024-09-17T22:41:57.731 INFO:teuthology.orchestra.run.smithi115.stdout:1616 read 0x229be98 thru 0x22a3139 (0x72a2 bytes) 2024-09-17T22:41:57.731 INFO:teuthology.orchestra.run.smithi115.stdout:1619 write 0x6f378eb thru 0x6f39b50 (0x2266 bytes) 2024-09-17T22:41:57.741 INFO:teuthology.orchestra.run.smithi115.stdout:1622 trunc from 0xde0d601 to 0x7b99807 2024-09-17T22:41:57.752 INFO:teuthology.orchestra.run.smithi115.stdout:1624 trunc from 0x7b99807 to 0x5efaa07 2024-09-17T22:41:57.762 INFO:teuthology.orchestra.run.smithi115.stdout:1625 punch from 0xf28832 to 0xf331d5, (0xa9a3 bytes) 2024-09-17T22:41:57.772 INFO:teuthology.orchestra.run.smithi115.stdout:1626 trunc from 0x5efaa07 to 0xe93e579 2024-09-17T22:41:57.792 INFO:teuthology.orchestra.run.smithi115.stdout:1627 trunc from 0xe93e579 to 0x10a7f1f 2024-09-17T22:41:57.802 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:57.800+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:57.806 INFO:teuthology.orchestra.run.smithi115.stdout:1628 trunc from 0x10a7f1f to 0x4e04f2b 2024-09-17T22:41:57.810 INFO:teuthology.orchestra.run.smithi115.stdout:1630 read 0x5069c8 thru 0x513f26 (0xd55f bytes) 2024-09-17T22:41:57.811 INFO:teuthology.orchestra.run.smithi115.stdout:1631 read 0x1eda692 thru 0x1ee6c36 (0xc5a5 bytes) 2024-09-17T22:41:57.811 INFO:teuthology.orchestra.run.smithi115.stdout:1632 read 0x17cb4f9 thru 0x17d1d21 (0x6829 bytes) 2024-09-17T22:41:57.817 INFO:teuthology.orchestra.run.smithi115.stdout:1633 write 0x606d758 thru 0x6071d50 (0x45f9 bytes) 2024-09-17T22:41:57.821 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:57.821 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:57.821 INFO:teuthology.orchestra.run.smithi115.stdout:1636 punch from 0x19d4031 to 0x19e14ae, (0xd47d bytes) 2024-09-17T22:41:57.821 INFO:teuthology.orchestra.run.smithi115.stdout:1637 read 0x153dc18 thru 0x154633e (0x8727 bytes) 2024-09-17T22:41:57.821 INFO:teuthology.orchestra.run.smithi115.stdout:1638 read 0x2459213 thru 0x245ad83 (0x1b71 bytes) 2024-09-17T22:41:57.821 INFO:teuthology.orchestra.run.smithi115.stdout:1639 trunc from 0x6071d51 to 0x6f5d92d 2024-09-17T22:41:57.828 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:41:57.828 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:41:57.828 INFO:teuthology.orchestra.run.smithi115.stdout:1640 trunc from 0x6f5d92d to 0x8969e64 2024-09-17T22:41:57.833 INFO:teuthology.orchestra.run.smithi115.stdout:1641 read 0x7a8efea thru 0x7a9c602 (0xd619 bytes) 2024-09-17T22:41:57.833 INFO:teuthology.orchestra.run.smithi115.stdout:1642 read 0x25e935b thru 0x25efdff (0x6aa5 bytes) 2024-09-17T22:41:57.837 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:41:57.843 INFO:teuthology.orchestra.run.smithi115.stdout:1643 write 0xcebe02e thru 0xcecd131 (0xf104 bytes) 2024-09-17T22:41:57.846 INFO:teuthology.orchestra.run.smithi115.stdout:1645 punch from 0x7b46c17 to 0x7b50c9a, (0xa083 bytes) 2024-09-17T22:41:57.847 INFO:teuthology.orchestra.run.smithi115.stdout:1647 punch from 0x4e1f7dd to 0x4e27c8d, (0x84b0 bytes) 2024-09-17T22:41:57.847 INFO:teuthology.orchestra.run.smithi115.stdout:1648 write 0x392d9d7 thru 0x3931698 (0x3cc2 bytes) 2024-09-17T22:41:57.849 INFO:teuthology.orchestra.run.smithi115.stdout:1649 write 0xc95d16c thru 0xc96ae02 (0xdc97 bytes) 2024-09-17T22:41:57.852 INFO:teuthology.orchestra.run.smithi115.stdout:1651 punch from 0x7adac75 to 0x7ae41d4, (0x955f bytes) 2024-09-17T22:41:57.852 INFO:teuthology.orchestra.run.smithi115.stdout:1652 writesame 0x1ad7479 thru 0x1ae4dc2 data_size 0x3(0xd94a bytes) 2024-09-17T22:41:57.869 INFO:teuthology.orchestra.run.smithi115.stdout:1654 trunc from 0xcecd132 to 0x5f09744 2024-09-17T22:41:57.877 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:41:57.879 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:57.881 INFO:teuthology.orchestra.run.smithi115.stdout:1657 trunc from 0x5f09744 to 0x47ef00b 2024-09-17T22:41:57.890 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:57.890 INFO:teuthology.orchestra.run.smithi115.stdout:1659 read 0x3f62c8f thru 0x3f6a8a3 (0x7c15 bytes) 2024-09-17T22:41:57.894 INFO:teuthology.orchestra.run.smithi115.stdout:1661 trunc from 0x47ef00b to 0x173da81 2024-09-17T22:41:57.896 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:41:57.897 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:41:57.904 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:57.901+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:57.907 INFO:teuthology.orchestra.run.smithi115.stdout: marking collection for removal 2024-09-17T22:41:57.907 INFO:teuthology.orchestra.run.smithi115.stdout:setting '_remove' omap key 2024-09-17T22:41:57.907 INFO:teuthology.orchestra.run.smithi115.stdout:finish_remove_pgs 3.13_head removing 3.13 2024-09-17T22:41:57.916 INFO:teuthology.orchestra.run.smithi115.stdout:Remove successful 2024-09-17T22:41:57.920 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:41:57.920 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:41:57.922 INFO:teuthology.orchestra.run.smithi115.stdout:1663 writesame 0xa449acc thru 0xa44a113 data_size 0xc(0x648 bytes) 2024-09-17T22:41:57.923 INFO:teuthology.orchestra.run.smithi115.stdout:1664 read 0x7b33ab8 thru 0x7b38b49 (0x5092 bytes) 2024-09-17T22:41:57.923 INFO:teuthology.orchestra.run.smithi115.stdout:1665 punch from 0x3ee628b to 0x3ef33e1, (0xd156 bytes) 2024-09-17T22:41:57.926 INFO:teuthology.orchestra.run.smithi115.stdout:1667 trunc from 0xa44a114 to 0xd1c7116 2024-09-17T22:41:57.931 INFO:teuthology.orchestra.run.smithi115.stdout:1668 flatten 2024-09-17T22:41:57.939 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:41:57.940 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:41:57.949 INFO:teuthology.orchestra.run.smithi115.stdout:1669 read 0x96cffbc thru 0x96d1a49 (0x1a8e bytes) 2024-09-17T22:41:57.949 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:57.950 INFO:teuthology.orchestra.run.smithi115.stdout:1671 writesame 0x83e3e4a thru 0x83eccf5 data_size 0x2e(0x8eac bytes) 2024-09-17T22:41:57.953 INFO:teuthology.orchestra.run.smithi115.stdout:1672 write 0xc8882eb thru 0xc893db7 (0xbacd bytes) 2024-09-17T22:41:57.957 INFO:teuthology.orchestra.run.smithi115.stdout:1674 read 0x628375a thru 0x62858e8 (0x218f bytes) 2024-09-17T22:41:57.957 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:57.957 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:57.957 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:41:57.958 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:41:57.968 INFO:teuthology.orchestra.run.smithi115.stdout:1678 trunc from 0xd1c7116 to 0x3f2b64a 2024-09-17T22:41:57.975 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:41:57.975 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:41:57.988 INFO:teuthology.orchestra.run.smithi115.stdout:1679 write 0x978a8bb thru 0x978f2fe (0x4a44 bytes) 2024-09-17T22:41:57.993 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:57.993 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:57.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:57.993 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:57.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:57.993 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158673 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:57.993 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:41:57.993 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7450946329999999, 2024-09-17T22:41:57.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039400614, 2024-09-17T22:41:57.993 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:57.993 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:57.994 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:57.994 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:57.994 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:57.994 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:57.994 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158673 2024-09-17T22:41:57.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:57.994 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:57.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:57.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:57.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:41:57.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:57.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:57.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:57.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:57.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:41:57.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:57.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:57.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:57.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:57.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:41:57.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.04e-07 2024-09-17T22:41:57.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:57.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:57.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:57.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241563+0000", 2024-09-17T22:41:57.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5799999999999998e-07 2024-09-17T22:41:57.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:57.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:57.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:57.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241566+0000", 2024-09-17T22:41:57.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8279999999999999e-06 2024-09-17T22:41:57.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:57.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:57.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:57.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241568+0000", 2024-09-17T22:41:57.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2709999999999999e-06 2024-09-17T22:41:57.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:57.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:57.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:57.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280954+0000", 2024-09-17T22:41:57.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039386051999999998 2024-09-17T22:41:57.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:57.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:57.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:57.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280962+0000", 2024-09-17T22:41:57.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3010000000000007e-06 2024-09-17T22:41:57.997 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:57.997 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:57.997 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:57.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:57.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:57.997 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158675 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:57.997 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:41:57.997 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7450853420000003, 2024-09-17T22:41:57.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039408033000000002, 2024-09-17T22:41:57.998 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:57.998 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:57.998 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:57.998 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:57.998 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:57.998 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:57.998 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158675 2024-09-17T22:41:57.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:57.998 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:57.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:57.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:57.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:41:57.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:57.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:57.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:57.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:57.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:41:57.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:57.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:57.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:57.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:57.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:41:57.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.36e-07 2024-09-17T22:41:57.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:57.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:57.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:57.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241572+0000", 2024-09-17T22:41:57.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6599999999999994e-07 2024-09-17T22:41:58.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241575+0000", 2024-09-17T22:41:58.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8389999999999998e-06 2024-09-17T22:41:58.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241577+0000", 2024-09-17T22:41:58.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4119999999999999e-06 2024-09-17T22:41:58.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280975+0000", 2024-09-17T22:41:58.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039397205999999997 2024-09-17T22:41:58.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280979+0000", 2024-09-17T22:41:58.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.374e-06 2024-09-17T22:41:58.001 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.001 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.001 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.001 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158677 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.001 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:41:58.002 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7450628989999997, 2024-09-17T22:41:58.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039409307999999997, 2024-09-17T22:41:58.002 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.002 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.002 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:58.002 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.002 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.002 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.002 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158677 2024-09-17T22:41:58.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.002 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:41:58.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:41:58.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241594+0000", 2024-09-17T22:41:58.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:41:58.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241595+0000", 2024-09-17T22:41:58.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2729999999999999e-06 2024-09-17T22:41:58.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241602+0000", 2024-09-17T22:41:58.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:41:58.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241606+0000", 2024-09-17T22:41:58.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3019999999999999e-06 2024-09-17T22:41:58.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280994+0000", 2024-09-17T22:41:58.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039388484000000001 2024-09-17T22:41:58.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.005 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281003+0000", 2024-09-17T22:41:58.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5459999999999992e-06 2024-09-17T22:41:58.005 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.005 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.005 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.006 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158679 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.006 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:41:58.006 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7450472210000001, 2024-09-17T22:41:58.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039413403999999999, 2024-09-17T22:41:58.006 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.006 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.006 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:58.006 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.006 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.006 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.006 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158679 2024-09-17T22:41:58.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.006 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:41:58.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:41:58.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:41:58.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4999999999999999e-07 2024-09-17T22:41:58.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241613+0000", 2024-09-17T22:41:58.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.207e-06 2024-09-17T22:41:58.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241616+0000", 2024-09-17T22:41:58.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.275e-06 2024-09-17T22:41:58.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241618+0000", 2024-09-17T22:41:58.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:41:58.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281017+0000", 2024-09-17T22:41:58.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039398385000000001 2024-09-17T22:41:58.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281022+0000", 2024-09-17T22:41:58.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.896e-06 2024-09-17T22:41:58.009 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.009 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.009 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.010 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158681 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.010 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:41:58.010 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7450350429999997, 2024-09-17T22:41:58.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039425292000000001, 2024-09-17T22:41:58.010 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.010 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.010 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:58.010 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.010 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.010 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.010 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158681 2024-09-17T22:41:58.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.011 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:41:58.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:41:58.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:41:58.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:41:58.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241622+0000", 2024-09-17T22:41:58.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:41:58.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241625+0000", 2024-09-17T22:41:58.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9629999999999998e-06 2024-09-17T22:41:58.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241628+0000", 2024-09-17T22:41:58.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4339999999999999e-06 2024-09-17T22:41:58.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281039+0000", 2024-09-17T22:41:58.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039410672000000001 2024-09-17T22:41:58.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281047+0000", 2024-09-17T22:41:58.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9969999999999998e-06 2024-09-17T22:41:58.013 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.013 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.014 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.014 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158682 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.014 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:41:58.014 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7450257589999998, 2024-09-17T22:41:58.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039436455000000002, 2024-09-17T22:41:58.014 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.014 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.014 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:58.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.014 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.014 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158682 2024-09-17T22:41:58.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.015 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:41:58.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:41:58.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:41:58.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:41:58.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241632+0000", 2024-09-17T22:41:58.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:41:58.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241635+0000", 2024-09-17T22:41:58.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8940000000000002e-06 2024-09-17T22:41:58.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241637+0000", 2024-09-17T22:41:58.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3659999999999999e-06 2024-09-17T22:41:58.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281061+0000", 2024-09-17T22:41:58.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039423664999999997 2024-09-17T22:41:58.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281067+0000", 2024-09-17T22:41:58.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.285e-06 2024-09-17T22:41:58.018 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.018 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.018 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.018 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158684 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.018 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:41:58.018 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7450164130000001, 2024-09-17T22:41:58.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462980000000002, 2024-09-17T22:41:58.018 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.018 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.018 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:58.018 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.019 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158684 2024-09-17T22:41:58.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.019 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:41:58.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.019 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.019 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:41:58.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:41:58.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:41:58.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241641+0000", 2024-09-17T22:41:58.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:41:58.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241644+0000", 2024-09-17T22:41:58.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8870000000000001e-06 2024-09-17T22:41:58.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241646+0000", 2024-09-17T22:41:58.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3530000000000002e-06 2024-09-17T22:41:58.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281093+0000", 2024-09-17T22:41:58.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039446947000000003 2024-09-17T22:41:58.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281103+0000", 2024-09-17T22:41:58.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6299999999999993e-06 2024-09-17T22:41:58.021 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.022 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.022 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.022 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158686 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.022 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:41:58.022 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7450072429999999, 2024-09-17T22:41:58.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039474214000000001, 2024-09-17T22:41:58.022 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.022 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.022 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:58.022 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.023 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.023 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.023 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158686 2024-09-17T22:41:58.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.023 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:41:58.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:41:58.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:41:58.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:41:58.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241650+0000", 2024-09-17T22:41:58.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2999999999999999e-07 2024-09-17T22:41:58.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241653+0000", 2024-09-17T22:41:58.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9689999999999999e-06 2024-09-17T22:41:58.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241655+0000", 2024-09-17T22:41:58.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3460000000000001e-06 2024-09-17T22:41:58.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281118+0000", 2024-09-17T22:41:58.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462876000000001 2024-09-17T22:41:58.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281123+0000", 2024-09-17T22:41:58.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8840000000000002e-06 2024-09-17T22:41:58.026 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.026 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.026 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.026 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158688 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.026 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:41:58.026 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7449980119999999, 2024-09-17T22:41:58.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039490458999999999, 2024-09-17T22:41:58.026 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.027 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.027 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:58.027 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.027 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.027 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.027 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158688 2024-09-17T22:41:58.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.027 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:41:58.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:41:58.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:41:58.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:41:58.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241659+0000", 2024-09-17T22:41:58.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3699999999999999e-07 2024-09-17T22:41:58.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241662+0000", 2024-09-17T22:41:58.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8499999999999998e-06 2024-09-17T22:41:58.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241665+0000", 2024-09-17T22:41:58.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.356e-06 2024-09-17T22:41:58.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281141+0000", 2024-09-17T22:41:58.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039476031000000002 2024-09-17T22:41:58.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281149+0000", 2024-09-17T22:41:58.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0709999999999995e-06 2024-09-17T22:41:58.030 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.030 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.030 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.030 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158690 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.030 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:41:58.030 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.744988953, 2024-09-17T22:41:58.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039502378999999997, 2024-09-17T22:41:58.031 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.031 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.031 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:58.031 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.031 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.031 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.031 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158690 2024-09-17T22:41:58.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.031 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:41:58.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:41:58.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241668+0000", 2024-09-17T22:41:58.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:41:58.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241669+0000", 2024-09-17T22:41:58.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1009999999999999e-06 2024-09-17T22:41:58.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241672+0000", 2024-09-17T22:41:58.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9059999999999998e-06 2024-09-17T22:41:58.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241674+0000", 2024-09-17T22:41:58.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4439999999999998e-06 2024-09-17T22:41:58.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281162+0000", 2024-09-17T22:41:58.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039488189 2024-09-17T22:41:58.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281170+0000", 2024-09-17T22:41:58.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5310000000000004e-06 2024-09-17T22:41:58.034 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.034 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.034 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.034 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158692 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.035 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:41:58.035 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7449795200000002, 2024-09-17T22:41:58.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039531064999999997, 2024-09-17T22:41:58.035 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.035 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.035 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:58.035 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.035 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.035 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.035 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158692 2024-09-17T22:41:58.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.035 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:41:58.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:41:58.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:41:58.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:41:58.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241678+0000", 2024-09-17T22:41:58.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0800000000000003e-07 2024-09-17T22:41:58.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241681+0000", 2024-09-17T22:41:58.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8729999999999999e-06 2024-09-17T22:41:58.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241683+0000", 2024-09-17T22:41:58.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.447e-06 2024-09-17T22:41:58.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281199+0000", 2024-09-17T22:41:58.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039515388999999998 2024-09-17T22:41:58.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281208+0000", 2024-09-17T22:41:58.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2399999999999996e-06 2024-09-17T22:41:58.038 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.038 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.038 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.039 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158694 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.039 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:41:58.039 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7449703489999999, 2024-09-17T22:41:58.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039543986000000003, 2024-09-17T22:41:58.039 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.039 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.039 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:58.039 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.039 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.039 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.039 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158694 2024-09-17T22:41:58.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.039 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:41:58.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:41:58.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:41:58.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:41:58.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241687+0000", 2024-09-17T22:41:58.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:41:58.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241691+0000", 2024-09-17T22:41:58.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.811e-06 2024-09-17T22:41:58.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241693+0000", 2024-09-17T22:41:58.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.385e-06 2024-09-17T22:41:58.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281223+0000", 2024-09-17T22:41:58.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039529332 2024-09-17T22:41:58.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281230+0000", 2024-09-17T22:41:58.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2139999999999999e-06 2024-09-17T22:41:58.042 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.042 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.042 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.043 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158696 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.043 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:41:58.043 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7449599610000002, 2024-09-17T22:41:58.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039570458000000003, 2024-09-17T22:41:58.043 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.043 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.043 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:58.043 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.043 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.043 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.043 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158696 2024-09-17T22:41:58.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.043 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:41:58.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:41:58.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:41:58.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:41:58.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:41:58.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.33e-07 2024-09-17T22:41:58.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241700+0000", 2024-09-17T22:41:58.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8949999999999998e-06 2024-09-17T22:41:58.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241703+0000", 2024-09-17T22:41:58.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3159999999999999e-06 2024-09-17T22:41:58.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281257+0000", 2024-09-17T22:41:58.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039553862000000002 2024-09-17T22:41:58.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281267+0000", 2024-09-17T22:41:58.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.024e-05 2024-09-17T22:41:58.046 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.046 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.046 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.047 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158698 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.047 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:41:58.047 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7449498830000003, 2024-09-17T22:41:58.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039576493999999997, 2024-09-17T22:41:58.047 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.047 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.047 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:58.047 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.047 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.047 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.047 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158698 2024-09-17T22:41:58.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.047 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:41:58.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:41:58.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241707+0000", 2024-09-17T22:41:58.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:41:58.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241708+0000", 2024-09-17T22:41:58.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5000000000000001e-07 2024-09-17T22:41:58.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241711+0000", 2024-09-17T22:41:58.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9670000000000001e-06 2024-09-17T22:41:58.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241713+0000", 2024-09-17T22:41:58.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2689999999999998e-06 2024-09-17T22:41:58.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281278+0000", 2024-09-17T22:41:58.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039565206999999998 2024-09-17T22:41:58.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281283+0000", 2024-09-17T22:41:58.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:41:58.050 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.050 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.051 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.051 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158700 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.051 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:41:58.051 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7449406720000002, 2024-09-17T22:41:58.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039604292999999999, 2024-09-17T22:41:58.051 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.051 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.051 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:58.051 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.051 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.051 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.051 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158700 2024-09-17T22:41:58.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.052 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:41:58.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:41:58.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:41:58.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:41:58.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241717+0000", 2024-09-17T22:41:58.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.78e-07 2024-09-17T22:41:58.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241720+0000", 2024-09-17T22:41:58.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8119999999999999e-06 2024-09-17T22:41:58.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241722+0000", 2024-09-17T22:41:58.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-17T22:41:58.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281311+0000", 2024-09-17T22:41:58.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039588894999999999 2024-09-17T22:41:58.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281320+0000", 2024-09-17T22:41:58.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9989999999999997e-06 2024-09-17T22:41:58.054 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.055 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.055 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.055 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158702 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.055 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:41:58.055 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7449293499999996, 2024-09-17T22:41:58.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039616142, 2024-09-17T22:41:58.055 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.055 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.055 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:58.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.055 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.056 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158702 2024-09-17T22:41:58.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.056 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:41:58.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:41:58.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:41:58.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4400000000000001e-07 2024-09-17T22:41:58.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241728+0000", 2024-09-17T22:41:58.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0839999999999999e-06 2024-09-17T22:41:58.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241731+0000", 2024-09-17T22:41:58.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0690000000000002e-06 2024-09-17T22:41:58.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241734+0000", 2024-09-17T22:41:58.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4159999999999998e-06 2024-09-17T22:41:58.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281334+0000", 2024-09-17T22:41:58.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039600110000000001 2024-09-17T22:41:58.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281343+0000", 2024-09-17T22:41:58.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2189999999999993e-06 2024-09-17T22:41:58.059 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.059 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.059 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.059 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158704 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.059 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:41:58.059 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7449194859999997, 2024-09-17T22:41:58.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039641485999999997, 2024-09-17T22:41:58.059 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.059 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.059 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:58.059 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.059 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.060 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.060 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158704 2024-09-17T22:41:58.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.060 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:41:58.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:41:58.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:41:58.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:41:58.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241738+0000", 2024-09-17T22:41:58.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0589999999999999e-06 2024-09-17T22:41:58.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241741+0000", 2024-09-17T22:41:58.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.943e-06 2024-09-17T22:41:58.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241744+0000", 2024-09-17T22:41:58.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4819999999999999e-06 2024-09-17T22:41:58.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281370+0000", 2024-09-17T22:41:58.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039625661 2024-09-17T22:41:58.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281378+0000", 2024-09-17T22:41:58.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1310000000000006e-06 2024-09-17T22:41:58.063 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.063 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.063 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.063 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158705 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.063 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:41:58.063 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7449089359999999, 2024-09-17T22:41:58.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039652637999999997, 2024-09-17T22:41:58.063 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.063 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.063 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:58.063 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.063 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.064 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.064 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158705 2024-09-17T22:41:58.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.064 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:41:58.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:41:58.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:41:58.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-07 2024-09-17T22:41:58.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:41:58.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0599999999999999e-07 2024-09-17T22:41:58.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241751+0000", 2024-09-17T22:41:58.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.926e-06 2024-09-17T22:41:58.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241754+0000", 2024-09-17T22:41:58.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4360000000000001e-06 2024-09-17T22:41:58.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281392+0000", 2024-09-17T22:41:58.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039638566 2024-09-17T22:41:58.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281400+0000", 2024-09-17T22:41:58.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.571e-06 2024-09-17T22:41:58.067 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.067 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.067 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.067 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158707 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.067 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:41:58.067 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7448878350000001, 2024-09-17T22:41:58.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039654809999999999, 2024-09-17T22:41:58.067 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.067 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.067 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:58.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.068 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.068 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158707 2024-09-17T22:41:58.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.068 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:41:58.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:41:58.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241769+0000", 2024-09-17T22:41:58.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4299999999999999e-07 2024-09-17T22:41:58.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241770+0000", 2024-09-17T22:41:58.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3769999999999999e-06 2024-09-17T22:41:58.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241773+0000", 2024-09-17T22:41:58.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2109999999999998e-06 2024-09-17T22:41:58.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241776+0000", 2024-09-17T22:41:58.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5069999999999999e-06 2024-09-17T22:41:58.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281415+0000", 2024-09-17T22:41:58.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039639499000000002 2024-09-17T22:41:58.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281423+0000", 2024-09-17T22:41:58.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8730000000000007e-06 2024-09-17T22:41:58.071 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.071 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.071 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.071 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158709 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.071 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:41:58.071 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.7448772259999998, 2024-09-17T22:41:58.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039661937000000001, 2024-09-17T22:41:58.071 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.071 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.072 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:58.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.072 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.072 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158709 2024-09-17T22:41:58.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.072 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:41:58.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:41:58.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242843+0000", 2024-09-17T22:41:58.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010634780000000001 2024-09-17T22:41:58.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242845+0000", 2024-09-17T22:41:58.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7209999999999999e-06 2024-09-17T22:41:58.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242851+0000", 2024-09-17T22:41:58.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6949999999999997e-06 2024-09-17T22:41:58.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242855+0000", 2024-09-17T22:41:58.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7960000000000002e-06 2024-09-17T22:41:58.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281434+0000", 2024-09-17T22:41:58.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038579519 2024-09-17T22:41:58.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.075 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281441+0000", 2024-09-17T22:41:58.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7279999999999998e-06 2024-09-17T22:41:58.075 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.075 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.075 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.075 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.075 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.075 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:58.076 INFO:teuthology.orchestra.run.smithi115.stdout:1680 write 0x876cb8f thru 0x8779a9f (0xcf11 bytes) 2024-09-17T22:41:58.077 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:41:58.077 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:41:58.078 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:41:58.079 INFO:teuthology.orchestra.run.smithi115.stdout:1681 write 0x798d7b0 thru 0x799cba3 (0xf3f4 bytes) 2024-09-17T22:41:58.079 INFO:teuthology.orchestra.run.smithi115.stdout:1682 read 0x4c07ed6 thru 0x4c0d5b4 (0x56df bytes) 2024-09-17T22:41:58.079 INFO:teuthology.orchestra.run.smithi115.stdout:1683 read 0x554b27d thru 0x5555a56 (0xa7da bytes) 2024-09-17T22:41:58.079 INFO:teuthology.orchestra.run.smithi115.stdout:1685 trunc from 0x978f2ff to 0xbab835f 2024-09-17T22:41:58.079 INFO:teuthology.orchestra.run.smithi115.stdout:1686 read 0x304119d thru 0x304e8c0 (0xd724 bytes) 2024-09-17T22:41:58.079 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:58.079 INFO:teuthology.orchestra.run.smithi115.stdout:1690 write 0x71d51f7 thru 0x71e0a41 (0xb84b bytes) 2024-09-17T22:41:58.080 INFO:teuthology.orchestra.run.smithi115.stdout:1691 read 0x3b86913 thru 0x3b926d5 (0xbdc3 bytes) 2024-09-17T22:41:58.080 INFO:teuthology.orchestra.run.smithi115.stdout:1694 trunc from 0xbab835f to 0xac75a34 2024-09-17T22:41:58.080 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:41:58.080 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:41:58.081 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:41:58.081 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:41:58.082 INFO:teuthology.orchestra.run.smithi115.stdout:1695 write 0xa759a07 thru 0xa75b546 (0x1b40 bytes) 2024-09-17T22:41:58.082 INFO:teuthology.orchestra.run.smithi115.stdout:1696 trunc from 0xac75a34 to 0x77cf7a3 2024-09-17T22:41:58.082 INFO:teuthology.orchestra.run.smithi115.stdout:1697 write 0x2cf96bf thru 0x2cfda25 (0x4367 bytes) 2024-09-17T22:41:58.082 INFO:teuthology.orchestra.run.smithi115.stdout:1699 write 0x995c49a thru 0x9964798 (0x82ff bytes) 2024-09-17T22:41:58.082 INFO:teuthology.orchestra.run.smithi115.stdout:1700 writesame 0x8ba1059 thru 0x8bae169 data_size 0x17(0xd111 bytes) 2024-09-17T22:41:58.082 INFO:teuthology.orchestra.run.smithi115.stdout:1701 writesame 0x977f505 thru 0x9781c7e data_size 0x1f(0x277a bytes) 2024-09-17T22:41:58.082 INFO:teuthology.orchestra.run.smithi115.stdout:1704 trunc from 0x9964799 to 0xa19c26a 2024-09-17T22:41:58.083 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:58.083 INFO:teuthology.orchestra.run.smithi115.stdout:1706 read 0x30a8f0f thru 0x30a9b7b (0xc6d bytes) 2024-09-17T22:41:58.083 INFO:teuthology.orchestra.run.smithi115.stdout:1707 trunc from 0xa19c26a to 0x1d2307d 2024-09-17T22:41:58.083 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:58.074+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:58.083 INFO:teuthology.orchestra.run.smithi115.stdout:1708 read 0xe95359 thru 0xe9e1da (0x8e82 bytes) 2024-09-17T22:41:58.083 INFO:teuthology.orchestra.run.smithi115.stdout:1709 punch from 0x13e3e90 to 0x13ef9e9, (0xbb59 bytes) 2024-09-17T22:41:58.094 INFO:teuthology.orchestra.run.smithi115.stdout:1710 writesame 0xb94176c thru 0xb950624 data_size 0xd(0xeeb9 bytes) 2024-09-17T22:41:58.094 INFO:teuthology.orchestra.run.smithi115.stdout:1711 read 0xa079957 thru 0xa0800d0 (0x677a bytes) 2024-09-17T22:41:58.094 INFO:teuthology.orchestra.run.smithi115.stdout:1714 punch from 0x7df8275 to 0x7e07312, (0xf09d bytes) 2024-09-17T22:41:58.094 INFO:teuthology.orchestra.run.smithi115.stdout:1716 write 0x370d17b thru 0x3717086 (0x9f0c bytes) 2024-09-17T22:41:58.097 INFO:teuthology.orchestra.run.smithi115.stdout:1717 read 0x38820b4 thru 0x3883d60 (0x1cad bytes) 2024-09-17T22:41:58.097 INFO:teuthology.orchestra.run.smithi115.stdout:1718 read 0x9b046a6 thru 0x9b0ce9c (0x87f7 bytes) 2024-09-17T22:41:58.097 INFO:teuthology.orchestra.run.smithi115.stdout:1719 writesame 0x562a220 thru 0x562b306 data_size 0x1(0x10e7 bytes) 2024-09-17T22:41:58.101 INFO:teuthology.orchestra.run.smithi115.stdout:1720 writesame 0x3f5754f thru 0x3f66ed1 data_size 0x7d(0xf983 bytes) 2024-09-17T22:41:58.112 INFO:teuthology.orchestra.run.smithi115.stdout:1722 writesame 0xdae2fb1 thru 0xdaec381 data_size 0x1(0x93d1 bytes) 2024-09-17T22:41:58.112 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:58.112 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:58.112 INFO:teuthology.orchestra.run.smithi115.stdout:1725 read 0xd59b3c2 thru 0xd59c1d3 (0xe12 bytes) 2024-09-17T22:41:58.112 INFO:teuthology.orchestra.run.smithi115.stdout:1726 write 0x3bc33f4 thru 0x3bcffde (0xcbeb bytes) 2024-09-17T22:41:58.117 INFO:teuthology.orchestra.run.smithi115.stdout:1727 trunc from 0xdaec382 to 0xaaabd25 2024-09-17T22:41:58.123 INFO:teuthology.orchestra.run.smithi115.stdout:1728 read 0x452fd33 thru 0x453d622 (0xd8f0 bytes) 2024-09-17T22:41:58.123 INFO:teuthology.orchestra.run.smithi115.stdout:1729 punch from 0x49e194e to 0x49e2b5b, (0x120d bytes) 2024-09-17T22:41:58.123 INFO:teuthology.orchestra.run.smithi115.stdout:1730 read 0x1c1d0f3 thru 0x1c24141 (0x704f bytes) 2024-09-17T22:41:58.123 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:58.124 INFO:teuthology.orchestra.run.smithi115.stdout:1733 read 0x231eb1f thru 0x23231ad (0x468f bytes) 2024-09-17T22:41:58.124 INFO:teuthology.orchestra.run.smithi115.stdout:1734 write 0x3adf102 thru 0x3ae52e3 (0x61e2 bytes) 2024-09-17T22:41:58.126 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:58.126 INFO:teuthology.orchestra.run.smithi115.stdout:1736 punch from 0x8b378cc to 0x8b4649c, (0xebd0 bytes) 2024-09-17T22:41:58.126 INFO:teuthology.orchestra.run.smithi115.stdout:1740 write 0x48576e6 thru 0x485ee14 (0x772f bytes) 2024-09-17T22:41:58.129 INFO:teuthology.orchestra.run.smithi115.stdout:1741 write 0x2924caf thru 0x2930f6f (0xc2c1 bytes) 2024-09-17T22:41:58.135 INFO:teuthology.orchestra.run.smithi115.stdout:1742 read 0x9eefda4 thru 0x9ef3d7b (0x3fd8 bytes) 2024-09-17T22:41:58.135 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:58.135 INFO:teuthology.orchestra.run.smithi115.stdout:1744 punch from 0x5780aed to 0x578fa77, (0xef8a bytes) 2024-09-17T22:41:58.135 INFO:teuthology.orchestra.run.smithi115.stdout:1748 read 0x1162eac thru 0x1172c8e (0xfde3 bytes) 2024-09-17T22:41:58.135 INFO:teuthology.orchestra.run.smithi115.stdout:1750 clone 17 order 21 su 262144 sc 14 2024-09-17T22:41:58.136 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is set 2024-09-17T22:41:58.189 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:58.189 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:58.189 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:41:58.189 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:58.189 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:58.187+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:58.198 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:41:58.278 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:58.276+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:58.300 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:58.300 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:58.300 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:58.300 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:58.300 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:58.308 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:41:58.379 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:58.377+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:58.413 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:58.413 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:58.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157229 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4073472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:41:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.2640809770000008, 2024-09-17T22:41:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041957959000000003, 2024-09-17T22:41:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157229 2024-09-17T22:41:58.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:41:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:41:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146207+0000", 2024-09-17T22:41:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0299999999999999e-07 2024-09-17T22:41:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146215+0000", 2024-09-17T22:41:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3059999999999998e-06 2024-09-17T22:41:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146222+0000", 2024-09-17T22:41:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1180000000000003e-06 2024-09-17T22:41:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146227+0000", 2024-09-17T22:41:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1749999999999998e-06 2024-09-17T22:41:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.187985+0000", 2024-09-17T22:41:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041758391999999998 2024-09-17T22:41:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188065+0000", 2024-09-17T22:41:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0290000000000005e-05 2024-09-17T22:41:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188164+0000", 2024-09-17T22:41:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9074999999999996e-05 2024-09-17T22:41:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157231 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4079616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:41:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.2639433530000002, 2024-09-17T22:41:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042034581000000001, 2024-09-17T22:41:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.418 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157231 2024-09-17T22:41:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:41:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:41:58.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146345+0000", 2024-09-17T22:41:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:41:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146347+0000", 2024-09-17T22:41:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.357e-06 2024-09-17T22:41:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146352+0000", 2024-09-17T22:41:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T22:41:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146361+0000", 2024-09-17T22:41:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0219999999999993e-06 2024-09-17T22:41:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188199+0000", 2024-09-17T22:41:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041838130000000001 2024-09-17T22:41:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188285+0000", 2024-09-17T22:41:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5160000000000005e-05 2024-09-17T22:41:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188379+0000", 2024-09-17T22:41:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3949000000000005e-05 2024-09-17T22:41:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.422 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157233 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4085760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:41:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.2637272040000003, 2024-09-17T22:41:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042008960999999997, 2024-09-17T22:41:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157233 2024-09-17T22:41:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:41:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:41:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146561+0000", 2024-09-17T22:41:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6300000000000005e-07 2024-09-17T22:41:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146564+0000", 2024-09-17T22:41:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-17T22:41:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146575+0000", 2024-09-17T22:41:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0891e-05 2024-09-17T22:41:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146579+0000", 2024-09-17T22:41:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.284e-06 2024-09-17T22:41:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188407+0000", 2024-09-17T22:41:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041828390999999999 2024-09-17T22:41:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188476+0000", 2024-09-17T22:41:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9129999999999997e-05 2024-09-17T22:41:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188569+0000", 2024-09-17T22:41:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2669000000000003e-05 2024-09-17T22:41:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.426 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157235 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4091904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:41:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.263533207, 2024-09-17T22:41:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042047517, 2024-09-17T22:41:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.427 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157235 2024-09-17T22:41:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:41:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:41:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146755+0000", 2024-09-17T22:41:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0900000000000001e-07 2024-09-17T22:41:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146758+0000", 2024-09-17T22:41:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.399e-06 2024-09-17T22:41:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146768+0000", 2024-09-17T22:41:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4730000000000006e-06 2024-09-17T22:41:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146774+0000", 2024-09-17T22:41:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6939999999999998e-06 2024-09-17T22:41:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188616+0000", 2024-09-17T22:41:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041841481999999999 2024-09-17T22:41:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188700+0000", 2024-09-17T22:41:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4227999999999994e-05 2024-09-17T22:41:58.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188802+0000", 2024-09-17T22:41:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101532 2024-09-17T22:41:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157237 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4098048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:41:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.2635029360000001, 2024-09-17T22:41:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165452999999999, 2024-09-17T22:41:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157237 2024-09-17T22:41:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:41:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:41:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146785+0000", 2024-09-17T22:41:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6800000000000001e-07 2024-09-17T22:41:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146787+0000", 2024-09-17T22:41:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.046e-06 2024-09-17T22:41:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146794+0000", 2024-09-17T22:41:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9249999999999998e-06 2024-09-17T22:41:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146799+0000", 2024-09-17T22:41:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9289999999999997e-06 2024-09-17T22:41:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188844+0000", 2024-09-17T22:41:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042045386999999997 2024-09-17T22:41:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188888+0000", 2024-09-17T22:41:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3760000000000001e-05 2024-09-17T22:41:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188950+0000", 2024-09-17T22:41:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1938000000000002e-05 2024-09-17T22:41:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.435 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157239 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4104192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:41:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.2634745499999998, 2024-09-17T22:41:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042347193999999998, 2024-09-17T22:41:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157239 2024-09-17T22:41:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:41:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:41:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:41:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:41:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146816+0000", 2024-09-17T22:41:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4700000000000001e-06 2024-09-17T22:41:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146822+0000", 2024-09-17T22:41:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.579e-06 2024-09-17T22:41:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146826+0000", 2024-09-17T22:41:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3780000000000001e-06 2024-09-17T22:41:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188991+0000", 2024-09-17T22:41:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165553000000001 2024-09-17T22:41:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189068+0000", 2024-09-17T22:41:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6548e-05 2024-09-17T22:41:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189160+0000", 2024-09-17T22:41:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2163000000000005e-05 2024-09-17T22:41:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157242 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4110336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:41:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.2634367639999997, 2024-09-17T22:41:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042499783999999999, 2024-09-17T22:41:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.440 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157242 2024-09-17T22:41:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:41:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:41:58.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:41:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:41:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146852+0000", 2024-09-17T22:41:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.37e-06 2024-09-17T22:41:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146856+0000", 2024-09-17T22:41:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1690000000000002e-06 2024-09-17T22:41:58.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146863+0000", 2024-09-17T22:41:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9920000000000001e-06 2024-09-17T22:41:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189193+0000", 2024-09-17T22:41:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042329355999999999 2024-09-17T22:41:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189246+0000", 2024-09-17T22:41:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3643000000000001e-05 2024-09-17T22:41:58.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189350+0000", 2024-09-17T22:41:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010394 2024-09-17T22:41:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157244 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4116480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:41:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.263417553, 2024-09-17T22:41:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042675684999999998, 2024-09-17T22:41:58.444 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157244 2024-09-17T22:41:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:41:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:41:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:41:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1e-07 2024-09-17T22:41:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146871+0000", 2024-09-17T22:41:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.189e-06 2024-09-17T22:41:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146877+0000", 2024-09-17T22:41:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8100000000000003e-06 2024-09-17T22:41:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146880+0000", 2024-09-17T22:41:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.83e-06 2024-09-17T22:41:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189388+0000", 2024-09-17T22:41:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042507637000000001 2024-09-17T22:41:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189443+0000", 2024-09-17T22:41:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5742999999999998e-05 2024-09-17T22:41:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189545+0000", 2024-09-17T22:41:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102166 2024-09-17T22:41:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157246 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4122624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:41:58.448 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.2633885009999997, 2024-09-17T22:41:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042847083000000001, 2024-09-17T22:41:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157246 2024-09-17T22:41:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:41:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:41:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:41:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0100000000000001e-07 2024-09-17T22:41:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146901+0000", 2024-09-17T22:41:58.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.531e-06 2024-09-17T22:41:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146905+0000", 2024-09-17T22:41:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0759999999999996e-06 2024-09-17T22:41:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146908+0000", 2024-09-17T22:41:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.083e-06 2024-09-17T22:41:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189585+0000", 2024-09-17T22:41:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042677009000000002 2024-09-17T22:41:58.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189656+0000", 2024-09-17T22:41:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0712999999999999e-05 2024-09-17T22:41:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189746+0000", 2024-09-17T22:41:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0370000000000004e-05 2024-09-17T22:41:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157247 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4128768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:41:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.2633166649999996, 2024-09-17T22:41:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043016916000000002, 2024-09-17T22:41:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157247 2024-09-17T22:41:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:41:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:41:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147071+0000", 2024-09-17T22:41:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100769 2024-09-17T22:41:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147081+0000", 2024-09-17T22:41:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9229999999999997e-06 2024-09-17T22:41:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147090+0000", 2024-09-17T22:41:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.935e-06 2024-09-17T22:41:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147097+0000", 2024-09-17T22:41:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T22:41:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189787+0000", 2024-09-17T22:41:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042690525 2024-09-17T22:41:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189884+0000", 2024-09-17T22:41:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6858e-05 2024-09-17T22:41:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189988+0000", 2024-09-17T22:41:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103336 2024-09-17T22:41:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.456 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157250 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4134912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:41:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.2631732039999992, 2024-09-17T22:41:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043099446, 2024-09-17T22:41:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157250 2024-09-17T22:41:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:41:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:41:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147235+0000", 2024-09-17T22:41:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120975 2024-09-17T22:41:58.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147238+0000", 2024-09-17T22:41:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5289999999999998e-06 2024-09-17T22:41:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147244+0000", 2024-09-17T22:41:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6050000000000003e-06 2024-09-17T22:41:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147249+0000", 2024-09-17T22:41:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4939999999999997e-06 2024-09-17T22:41:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190020+0000", 2024-09-17T22:41:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042771300999999998 2024-09-17T22:41:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190102+0000", 2024-09-17T22:41:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1600000000000005e-05 2024-09-17T22:41:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190214+0000", 2024-09-17T22:41:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011194200000000001 2024-09-17T22:41:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.460 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157252 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4141056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:41:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.2630305350000004, 2024-09-17T22:41:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043165508999999998, 2024-09-17T22:41:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157252 2024-09-17T22:41:58.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:41:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:41:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147402+0000", 2024-09-17T22:41:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014475600000000001 2024-09-17T22:41:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147411+0000", 2024-09-17T22:41:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4350000000000003e-06 2024-09-17T22:41:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147420+0000", 2024-09-17T22:41:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4790000000000002e-06 2024-09-17T22:41:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147427+0000", 2024-09-17T22:41:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5400000000000001e-06 2024-09-17T22:41:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190254+0000", 2024-09-17T22:41:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042826731 2024-09-17T22:41:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190326+0000", 2024-09-17T22:41:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1798999999999995e-05 2024-09-17T22:41:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190422+0000", 2024-09-17T22:41:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6768999999999995e-05 2024-09-17T22:41:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157253 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4147200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:41:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.2628441689999992, 2024-09-17T22:41:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043157752000000001, 2024-09-17T22:41:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.465 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157253 2024-09-17T22:41:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:41:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:41:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147590+0000", 2024-09-17T22:41:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014709 2024-09-17T22:41:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147593+0000", 2024-09-17T22:41:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5840000000000002e-06 2024-09-17T22:41:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147598+0000", 2024-09-17T22:41:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2469999999999999e-06 2024-09-17T22:41:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147602+0000", 2024-09-17T22:41:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0049999999999998e-06 2024-09-17T22:41:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190475+0000", 2024-09-17T22:41:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872767999999999 2024-09-17T22:41:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190528+0000", 2024-09-17T22:41:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2889999999999997e-05 2024-09-17T22:41:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190601+0000", 2024-09-17T22:41:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3168000000000001e-05 2024-09-17T22:41:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157255 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4153344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:41:58.469 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.2626781880000006, 2024-09-17T22:41:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043132251000000003, 2024-09-17T22:41:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157255 2024-09-17T22:41:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:41:58.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:41:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147820+0000", 2024-09-17T22:41:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021123600000000001 2024-09-17T22:41:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147824+0000", 2024-09-17T22:41:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6780000000000002e-06 2024-09-17T22:41:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147833+0000", 2024-09-17T22:41:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4250000000000004e-06 2024-09-17T22:41:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147842+0000", 2024-09-17T22:41:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0679999999999997e-06 2024-09-17T22:41:58.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190632+0000", 2024-09-17T22:41:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042790863999999998 2024-09-17T22:41:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190674+0000", 2024-09-17T22:41:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1804000000000002e-05 2024-09-17T22:41:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190741+0000", 2024-09-17T22:41:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7175999999999995e-05 2024-09-17T22:41:58.473 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157257 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4159488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:41:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.2624376979999994, 2024-09-17T22:41:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.04307035, 2024-09-17T22:41:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157257 2024-09-17T22:41:58.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:41:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:41:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148844+0000", 2024-09-17T22:41:58.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00099455599999999996 2024-09-17T22:41:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148848+0000", 2024-09-17T22:41:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0180000000000003e-06 2024-09-17T22:41:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148858+0000", 2024-09-17T22:41:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.825e-06 2024-09-17T22:41:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148865+0000", 2024-09-17T22:41:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8290000000000002e-06 2024-09-17T22:41:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190789+0000", 2024-09-17T22:41:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041924630999999997 2024-09-17T22:41:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190860+0000", 2024-09-17T22:41:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0557999999999998e-05 2024-09-17T22:41:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190920+0000", 2024-09-17T22:41:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9932999999999997e-05 2024-09-17T22:41:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157259 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4165632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:41:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.2614116790000001, 2024-09-17T22:41:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042261411999999998, 2024-09-17T22:41:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.478 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.479 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157259 2024-09-17T22:41:58.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.479 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:41:58.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:41:58.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:41:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:41:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148878+0000", 2024-09-17T22:41:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:41:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148885+0000", 2024-09-17T22:41:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7290000000000003e-06 2024-09-17T22:41:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148893+0000", 2024-09-17T22:41:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8110000000000003e-06 2024-09-17T22:41:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190967+0000", 2024-09-17T22:41:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042073948999999999 2024-09-17T22:41:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191069+0000", 2024-09-17T22:41:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102075 2024-09-17T22:41:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191137+0000", 2024-09-17T22:41:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8281000000000006e-05 2024-09-17T22:41:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157261 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4171776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.482 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:41:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.2613620720000007, 2024-09-17T22:41:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042437255, 2024-09-17T22:41:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157261 2024-09-17T22:41:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:41:58.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:41:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148926+0000", 2024-09-17T22:41:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T22:41:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148928+0000", 2024-09-17T22:41:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4329999999999998e-06 2024-09-17T22:41:58.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148936+0000", 2024-09-17T22:41:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6969999999999993e-06 2024-09-17T22:41:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148942+0000", 2024-09-17T22:41:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9479999999999996e-06 2024-09-17T22:41:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191185+0000", 2024-09-17T22:41:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042243147000000002 2024-09-17T22:41:58.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191266+0000", 2024-09-17T22:41:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1186000000000005e-05 2024-09-17T22:41:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191362+0000", 2024-09-17T22:41:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6384000000000004e-05 2024-09-17T22:41:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157263 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4177920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.486 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:41:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.2613376729999999, 2024-09-17T22:41:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042636395000000001, 2024-09-17T22:41:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157263 2024-09-17T22:41:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:41:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:41:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:41:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.37e-07 2024-09-17T22:41:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148952+0000", 2024-09-17T22:41:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.88e-06 2024-09-17T22:41:58.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148959+0000", 2024-09-17T22:41:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5649999999999998e-06 2024-09-17T22:41:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148963+0000", 2024-09-17T22:41:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8570000000000003e-06 2024-09-17T22:41:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191408+0000", 2024-09-17T22:41:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042444707999999998 2024-09-17T22:41:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191489+0000", 2024-09-17T22:41:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0612999999999995e-05 2024-09-17T22:41:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191586+0000", 2024-09-17T22:41:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7235e-05 2024-09-17T22:41:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157265 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4184064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.490 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:41:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.2613161240000004, 2024-09-17T22:41:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042791830000000003, 2024-09-17T22:41:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157265 2024-09-17T22:41:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:41:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:41:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148972+0000", 2024-09-17T22:41:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:41:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148973+0000", 2024-09-17T22:41:58.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.894e-06 2024-09-17T22:41:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148985+0000", 2024-09-17T22:41:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1995e-05 2024-09-17T22:41:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148990+0000", 2024-09-17T22:41:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8269999999999997e-06 2024-09-17T22:41:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191630+0000", 2024-09-17T22:41:58.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042639639 2024-09-17T22:41:58.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191686+0000", 2024-09-17T22:41:58.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6066000000000002e-05 2024-09-17T22:41:58.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191763+0000", 2024-09-17T22:41:58.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7035999999999999e-05 2024-09-17T22:41:58.494 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.495 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.495 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.495 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157267 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4190208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:41:58.495 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:41:58.495 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 8.2612903109999998, 2024-09-17T22:41:58.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043028813999999999, 2024-09-17T22:41:58.495 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.495 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.495 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.495 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157267 2024-09-17T22:41:58.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.496 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:41:58.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:41:58.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:41:58.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:41:58.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148999+0000", 2024-09-17T22:41:58.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.432e-06 2024-09-17T22:41:58.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149002+0000", 2024-09-17T22:41:58.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7850000000000002e-06 2024-09-17T22:41:58.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149005+0000", 2024-09-17T22:41:58.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:41:58.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191878+0000", 2024-09-17T22:41:58.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872312000000003 2024-09-17T22:41:58.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191961+0000", 2024-09-17T22:41:58.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3418000000000007e-05 2024-09-17T22:41:58.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.192026+0000", 2024-09-17T22:41:58.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4689000000000004e-05 2024-09-17T22:41:58.499 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.499 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.499 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.499 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.499 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.499 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:58.501 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.13.0 2024-09-17T22:41:58.501 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:41:58.502 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:58.498+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:58.601 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:58.599+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:58.646 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:58.646 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:58.646 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:41:58.646 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:58.654 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:41:58.702 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:58.700+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:58.797 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:58.797 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:58.797 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:58.797 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:58.797 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:58.802 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:58.800+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:58.805 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:41:58.903 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:58.901+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:58.950 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38083 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.763873+0000", 2024-09-17T22:41:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.183517248000001, 2024-09-17T22:41:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023344747999999999, 2024-09-17T22:41:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.951 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38083 2024-09-17T22:41:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763873+0000", 2024-09-17T22:41:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763873+0000", 2024-09-17T22:41:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763874+0000", 2024-09-17T22:41:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0899999999999997e-07 2024-09-17T22:41:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763875+0000", 2024-09-17T22:41:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8530000000000001e-06 2024-09-17T22:41:58.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763881+0000", 2024-09-17T22:41:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4879999999999998e-06 2024-09-17T22:41:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763886+0000", 2024-09-17T22:41:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.587e-06 2024-09-17T22:41:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787057+0000", 2024-09-17T22:41:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023170611000000001 2024-09-17T22:41:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787127+0000", 2024-09-17T22:41:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0259000000000006e-05 2024-09-17T22:41:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787218+0000", 2024-09-17T22:41:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0641000000000002e-05 2024-09-17T22:41:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38105 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.763938+0000", 2024-09-17T22:41:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.183452441, 2024-09-17T22:41:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023471315999999999, 2024-09-17T22:41:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38105 2024-09-17T22:41:58.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763938+0000", 2024-09-17T22:41:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763938+0000", 2024-09-17T22:41:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763939+0000", 2024-09-17T22:41:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:41:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763941+0000", 2024-09-17T22:41:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9489999999999999e-06 2024-09-17T22:41:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763946+0000", 2024-09-17T22:41:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9139999999999996e-06 2024-09-17T22:41:58.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763951+0000", 2024-09-17T22:41:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6589999999999998e-06 2024-09-17T22:41:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787249+0000", 2024-09-17T22:41:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023297989000000002 2024-09-17T22:41:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787319+0000", 2024-09-17T22:41:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9836e-05 2024-09-17T22:41:58.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787409+0000", 2024-09-17T22:41:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0457000000000005e-05 2024-09-17T22:41:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38107 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.183433652000002, 2024-09-17T22:41:58.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023669550000000001, 2024-09-17T22:41:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38107 2024-09-17T22:41:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:58.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:41:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:41:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763959+0000", 2024-09-17T22:41:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.849e-06 2024-09-17T22:41:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763965+0000", 2024-09-17T22:41:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6110000000000001e-06 2024-09-17T22:41:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763969+0000", 2024-09-17T22:41:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-06 2024-09-17T22:41:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787451+0000", 2024-09-17T22:41:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023482281000000001 2024-09-17T22:41:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787516+0000", 2024-09-17T22:41:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4930000000000003e-05 2024-09-17T22:41:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787626+0000", 2024-09-17T22:41:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011000700000000001 2024-09-17T22:41:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.964 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38141 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764408+0000", 2024-09-17T22:41:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.182982419999998, 2024-09-17T22:41:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023422268, 2024-09-17T22:41:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.965 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38141 2024-09-17T22:41:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764408+0000", 2024-09-17T22:41:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764408+0000", 2024-09-17T22:41:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764409+0000", 2024-09-17T22:41:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:41:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764412+0000", 2024-09-17T22:41:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4470000000000002e-06 2024-09-17T22:41:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764421+0000", 2024-09-17T22:41:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7299999999999994e-06 2024-09-17T22:41:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764427+0000", 2024-09-17T22:41:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.392e-06 2024-09-17T22:41:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787669+0000", 2024-09-17T22:41:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023241385 2024-09-17T22:41:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787743+0000", 2024-09-17T22:41:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4260999999999999e-05 2024-09-17T22:41:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787830+0000", 2024-09-17T22:41:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7398999999999994e-05 2024-09-17T22:41:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38143 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.182952491999998, 2024-09-17T22:41:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023565691, 2024-09-17T22:41:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38143 2024-09-17T22:41:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:41:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6199999999999998e-07 2024-09-17T22:41:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764441+0000", 2024-09-17T22:41:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1179999999999999e-06 2024-09-17T22:41:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764447+0000", 2024-09-17T22:41:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1940000000000003e-06 2024-09-17T22:41:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764452+0000", 2024-09-17T22:41:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9150000000000004e-06 2024-09-17T22:41:58.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787881+0000", 2024-09-17T22:41:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023429141000000001 2024-09-17T22:41:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787947+0000", 2024-09-17T22:41:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5902999999999996e-05 2024-09-17T22:41:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788004+0000", 2024-09-17T22:41:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6957999999999999e-05 2024-09-17T22:41:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38150 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.182932489999999, 2024-09-17T22:41:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023741399, 2024-09-17T22:41:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.974 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38150 2024-09-17T22:41:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:58.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:41:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8299999999999998e-07 2024-09-17T22:41:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764460+0000", 2024-09-17T22:41:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.855e-06 2024-09-17T22:41:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764466+0000", 2024-09-17T22:41:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8200000000000002e-06 2024-09-17T22:41:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764471+0000", 2024-09-17T22:41:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8319999999999996e-06 2024-09-17T22:41:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788032+0000", 2024-09-17T22:41:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023561480999999999 2024-09-17T22:41:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788106+0000", 2024-09-17T22:41:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3406999999999997e-05 2024-09-17T22:41:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788199+0000", 2024-09-17T22:41:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3621000000000004e-05 2024-09-17T22:41:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38152 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 83968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:58.978 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.182913772999999, 2024-09-17T22:41:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023859726000000001, 2024-09-17T22:41:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38152 2024-09-17T22:41:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:41:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3000000000000001e-07 2024-09-17T22:41:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764479+0000", 2024-09-17T22:41:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8920000000000001e-06 2024-09-17T22:41:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764485+0000", 2024-09-17T22:41:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:41:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764490+0000", 2024-09-17T22:41:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6890000000000003e-06 2024-09-17T22:41:58.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788232+0000", 2024-09-17T22:41:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023742452000000001 2024-09-17T22:41:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788276+0000", 2024-09-17T22:41:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3689000000000001e-05 2024-09-17T22:41:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788336+0000", 2024-09-17T22:41:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0581999999999997e-05 2024-09-17T22:41:58.982 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38184 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 161792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764589+0000", 2024-09-17T22:41:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.182801380000001, 2024-09-17T22:41:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023956801, 2024-09-17T22:41:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.983 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38184 2024-09-17T22:41:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764589+0000", 2024-09-17T22:41:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764589+0000", 2024-09-17T22:41:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764590+0000", 2024-09-17T22:41:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7800000000000001e-07 2024-09-17T22:41:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764592+0000", 2024-09-17T22:41:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6929999999999999e-06 2024-09-17T22:41:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764600+0000", 2024-09-17T22:41:58.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1140000000000006e-06 2024-09-17T22:41:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764606+0000", 2024-09-17T22:41:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6200000000000004e-06 2024-09-17T22:41:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788374+0000", 2024-09-17T22:41:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023768083999999998 2024-09-17T22:41:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788447+0000", 2024-09-17T22:41:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2922000000000004e-05 2024-09-17T22:41:58.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788546+0000", 2024-09-17T22:41:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8789999999999994e-05 2024-09-17T22:41:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38186 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.182776430000001, 2024-09-17T22:41:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024156486000000001, 2024-09-17T22:41:58.987 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38186 2024-09-17T22:41:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:41:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:41:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764621+0000", 2024-09-17T22:41:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6409999999999996e-06 2024-09-17T22:41:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764628+0000", 2024-09-17T22:41:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0360000000000001e-06 2024-09-17T22:41:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764633+0000", 2024-09-17T22:41:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0300000000000001e-06 2024-09-17T22:41:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788594+0000", 2024-09-17T22:41:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023960921999999999 2024-09-17T22:41:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788676+0000", 2024-09-17T22:41:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2123e-05 2024-09-17T22:41:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788771+0000", 2024-09-17T22:41:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4284999999999996e-05 2024-09-17T22:41:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38218 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 350208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764706+0000", 2024-09-17T22:41:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.182684176999999, 2024-09-17T22:41:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024324732000000002, 2024-09-17T22:41:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38218 2024-09-17T22:41:58.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764706+0000", 2024-09-17T22:41:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764706+0000", 2024-09-17T22:41:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764707+0000", 2024-09-17T22:41:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7199999999999999e-07 2024-09-17T22:41:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764710+0000", 2024-09-17T22:41:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6819999999999999e-06 2024-09-17T22:41:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764716+0000", 2024-09-17T22:41:58.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7900000000000002e-06 2024-09-17T22:41:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764722+0000", 2024-09-17T22:41:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2560000000000002e-06 2024-09-17T22:41:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788867+0000", 2024-09-17T22:41:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024144982999999998 2024-09-17T22:41:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788941+0000", 2024-09-17T22:41:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4529000000000004e-05 2024-09-17T22:41:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789031+0000", 2024-09-17T22:41:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9920000000000006e-05 2024-09-17T22:41:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38222 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764765+0000", 2024-09-17T22:41:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.182625323, 2024-09-17T22:41:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024443910999999999, 2024-09-17T22:41:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38222 2024-09-17T22:41:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764765+0000", 2024-09-17T22:41:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764765+0000", 2024-09-17T22:41:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764766+0000", 2024-09-17T22:41:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8999999999999996e-07 2024-09-17T22:41:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:58.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764769+0000", 2024-09-17T22:41:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7659999999999999e-06 2024-09-17T22:41:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764776+0000", 2024-09-17T22:41:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4229999999999999e-06 2024-09-17T22:41:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764783+0000", 2024-09-17T22:41:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9600000000000003e-06 2024-09-17T22:41:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789060+0000", 2024-09-17T22:41:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024277343 2024-09-17T22:41:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789129+0000", 2024-09-17T22:41:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8367999999999993e-05 2024-09-17T22:41:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789209+0000", 2024-09-17T22:41:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0461000000000001e-05 2024-09-17T22:41:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.000 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38671 3.12 3:4a5b92ac:::rbd_data.115fe0c30ef6.0000000000000011:5 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778865+0000", 2024-09-17T22:41:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.168525262999999, 2024-09-17T22:41:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023227351, 2024-09-17T22:41:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38671 2024-09-17T22:41:59.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778865+0000", 2024-09-17T22:41:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778865+0000", 2024-09-17T22:41:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778866+0000", 2024-09-17T22:41:59.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0900000000000002e-07 2024-09-17T22:41:59.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778868+0000", 2024-09-17T22:41:59.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4339999999999999e-06 2024-09-17T22:41:59.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778875+0000", 2024-09-17T22:41:59.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4559999999999997e-06 2024-09-17T22:41:59.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778880+0000", 2024-09-17T22:41:59.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2689999999999999e-06 2024-09-17T22:41:59.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.801978+0000", 2024-09-17T22:41:59.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023098225999999999 2024-09-17T22:41:59.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802035+0000", 2024-09-17T22:41:59.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7123999999999997e-05 2024-09-17T22:41:59.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.005 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802093+0000", 2024-09-17T22:41:59.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7432999999999997e-05 2024-09-17T22:41:59.005 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.005 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.005 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.005 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38673 3.12 3:4a5b92ac:::rbd_data.115fe0c30ef6.0000000000000011:5 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.005 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778886+0000", 2024-09-17T22:41:59.005 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.168504115000001, 2024-09-17T22:41:59.005 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023389259999999999, 2024-09-17T22:41:59.005 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.005 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.005 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.006 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.006 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.006 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.006 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38673 2024-09-17T22:41:59.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.006 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778886+0000", 2024-09-17T22:41:59.006 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.006 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.006 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.006 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778886+0000", 2024-09-17T22:41:59.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778887+0000", 2024-09-17T22:41:59.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:41:59.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.007 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.007 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.007 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778889+0000", 2024-09-17T22:41:59.007 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9379999999999999e-06 2024-09-17T22:41:59.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778895+0000", 2024-09-17T22:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0360000000000003e-06 2024-09-17T22:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778900+0000", 2024-09-17T22:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1220000000000003e-06 2024-09-17T22:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802126+0000", 2024-09-17T22:41:59.008 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023226607 2024-09-17T22:41:59.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802208+0000", 2024-09-17T22:41:59.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1157999999999998e-05 2024-09-17T22:41:59.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.009 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.009 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.009 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802276+0000", 2024-09-17T22:41:59.009 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8122999999999998e-05 2024-09-17T22:41:59.009 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.009 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.009 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.010 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38675 3.12 3:4a5b92ac:::rbd_data.115fe0c30ef6.0000000000000011:5 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.010 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:59.010 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.168468910000001, 2024-09-17T22:41:59.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023484221999999999, 2024-09-17T22:41:59.010 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.010 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.010 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.010 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.010 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.010 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.010 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38675 2024-09-17T22:41:59.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.011 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:59.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:59.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:41:59.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9999999999999997e-07 2024-09-17T22:41:59.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778924+0000", 2024-09-17T22:41:59.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3039999999999999e-06 2024-09-17T22:41:59.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778931+0000", 2024-09-17T22:41:59.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9519999999999997e-06 2024-09-17T22:41:59.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778937+0000", 2024-09-17T22:41:59.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3639999999999998e-06 2024-09-17T22:41:59.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802300+0000", 2024-09-17T22:41:59.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023363544 2024-09-17T22:41:59.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802347+0000", 2024-09-17T22:41:59.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6897999999999997e-05 2024-09-17T22:41:59.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802406+0000", 2024-09-17T22:41:59.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8560000000000002e-05 2024-09-17T22:41:59.014 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.014 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.014 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.014 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38679 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.014 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778945+0000", 2024-09-17T22:41:59.014 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.168445294000001, 2024-09-17T22:41:59.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023578015000000001, 2024-09-17T22:41:59.015 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.015 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.015 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.015 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38679 2024-09-17T22:41:59.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.015 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.015 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.015 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.015 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778945+0000", 2024-09-17T22:41:59.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778945+0000", 2024-09-17T22:41:59.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778946+0000", 2024-09-17T22:41:59.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.44e-07 2024-09-17T22:41:59.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778948+0000", 2024-09-17T22:41:59.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-17T22:41:59.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778955+0000", 2024-09-17T22:41:59.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.528e-06 2024-09-17T22:41:59.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778960+0000", 2024-09-17T22:41:59.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1919999999999996e-06 2024-09-17T22:41:59.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802429+0000", 2024-09-17T22:41:59.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023469561999999999 2024-09-17T22:41:59.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802470+0000", 2024-09-17T22:41:59.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1068000000000001e-05 2024-09-17T22:41:59.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802523+0000", 2024-09-17T22:41:59.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2852999999999997e-05 2024-09-17T22:41:59.018 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.018 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.018 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.019 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38680 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.019 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778966+0000", 2024-09-17T22:41:59.019 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.168424336000001, 2024-09-17T22:41:59.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023670668999999998, 2024-09-17T22:41:59.019 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.019 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.019 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.019 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38680 2024-09-17T22:41:59.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.019 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778966+0000", 2024-09-17T22:41:59.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778966+0000", 2024-09-17T22:41:59.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778967+0000", 2024-09-17T22:41:59.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.89e-07 2024-09-17T22:41:59.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778969+0000", 2024-09-17T22:41:59.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0219999999999999e-06 2024-09-17T22:41:59.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778975+0000", 2024-09-17T22:41:59.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3779999999999998e-06 2024-09-17T22:41:59.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778980+0000", 2024-09-17T22:41:59.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0699999999999997e-06 2024-09-17T22:41:59.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802545+0000", 2024-09-17T22:41:59.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023565358000000002 2024-09-17T22:41:59.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802585+0000", 2024-09-17T22:41:59.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0136999999999999e-05 2024-09-17T22:41:59.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802637+0000", 2024-09-17T22:41:59.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1314999999999999e-05 2024-09-17T22:41:59.023 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.023 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.023 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.023 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38697 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 675840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.023 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.779074+0000", 2024-09-17T22:41:59.023 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.168316083000001, 2024-09-17T22:41:59.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023676402999999999, 2024-09-17T22:41:59.023 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.023 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.024 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.024 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38697 2024-09-17T22:41:59.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.024 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779074+0000", 2024-09-17T22:41:59.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779074+0000", 2024-09-17T22:41:59.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779075+0000", 2024-09-17T22:41:59.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:41:59.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779077+0000", 2024-09-17T22:41:59.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8810000000000001e-06 2024-09-17T22:41:59.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779084+0000", 2024-09-17T22:41:59.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.691e-06 2024-09-17T22:41:59.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779088+0000", 2024-09-17T22:41:59.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6970000000000001e-06 2024-09-17T22:41:59.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802659+0000", 2024-09-17T22:41:59.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023571214 2024-09-17T22:41:59.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802699+0000", 2024-09-17T22:41:59.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9762000000000003e-05 2024-09-17T22:41:59.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802751+0000", 2024-09-17T22:41:59.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1572e-05 2024-09-17T22:41:59.028 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.028 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.028 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.028 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38698 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 749568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.028 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.779094+0000", 2024-09-17T22:41:59.028 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.168296145999999, 2024-09-17T22:41:59.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023778845999999999, 2024-09-17T22:41:59.028 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.028 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.028 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.029 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.029 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38698 2024-09-17T22:41:59.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.029 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779094+0000", 2024-09-17T22:41:59.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779094+0000", 2024-09-17T22:41:59.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779095+0000", 2024-09-17T22:41:59.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8799999999999998e-07 2024-09-17T22:41:59.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779097+0000", 2024-09-17T22:41:59.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8530000000000001e-06 2024-09-17T22:41:59.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779103+0000", 2024-09-17T22:41:59.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0410000000000003e-06 2024-09-17T22:41:59.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779107+0000", 2024-09-17T22:41:59.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T22:41:59.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802774+0000", 2024-09-17T22:41:59.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023666348 2024-09-17T22:41:59.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802814+0000", 2024-09-17T22:41:59.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0173000000000003e-05 2024-09-17T22:41:59.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802873+0000", 2024-09-17T22:41:59.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9484999999999997e-05 2024-09-17T22:41:59.032 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.032 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.032 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.032 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38825 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.032 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.781521+0000", 2024-09-17T22:41:59.032 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.165869079, 2024-09-17T22:41:59.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023214104999999999, 2024-09-17T22:41:59.033 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.033 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.033 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.033 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.033 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38825 2024-09-17T22:41:59.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.033 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781521+0000", 2024-09-17T22:41:59.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781521+0000", 2024-09-17T22:41:59.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781522+0000", 2024-09-17T22:41:59.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.53e-07 2024-09-17T22:41:59.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781524+0000", 2024-09-17T22:41:59.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0389999999999999e-06 2024-09-17T22:41:59.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781530+0000", 2024-09-17T22:41:59.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8209999999999998e-06 2024-09-17T22:41:59.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781534+0000", 2024-09-17T22:41:59.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6419999999999998e-06 2024-09-17T22:41:59.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804570+0000", 2024-09-17T22:41:59.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023035951999999998 2024-09-17T22:41:59.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804645+0000", 2024-09-17T22:41:59.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4696000000000004e-05 2024-09-17T22:41:59.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804736+0000", 2024-09-17T22:41:59.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0601999999999998e-05 2024-09-17T22:41:59.037 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.037 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.037 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.037 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38843 3.12 3.c4845672 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.037 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:59.037 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.165746868999999, 2024-09-17T22:41:59.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023236265999999998, 2024-09-17T22:41:59.037 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.037 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.037 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:41:59.037 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.037 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.038 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.038 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38843 2024-09-17T22:41:59.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.038 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:59.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:59.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:41:59.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:41:59.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781646+0000", 2024-09-17T22:41:59.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9290000000000001e-06 2024-09-17T22:41:59.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781652+0000", 2024-09-17T22:41:59.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7389999999999996e-06 2024-09-17T22:41:59.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781656+0000", 2024-09-17T22:41:59.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3830000000000002e-06 2024-09-17T22:41:59.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804871+0000", 2024-09-17T22:41:59.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023215257999999999 2024-09-17T22:41:59.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804880+0000", 2024-09-17T22:41:59.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5769999999999994e-06 2024-09-17T22:41:59.041 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.041 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.041 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.041 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.041 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.041 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:59.043 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:41:59.043 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:59.040+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:59.143 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:41:59.140+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:41:59.189 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:59.189 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:59.190 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:41:59.190 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:59.199 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:41:59.244 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:41:59.241+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:41:59.315 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-17T22:41:59.343 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:59.343 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:41:59.343 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:41:59.343 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:41:59.343 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:59.345 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:59.342+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:59.352 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:41:59.445 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:41:59.443+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:41:59.460 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:41:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:41:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:41:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:41:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.460 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36437 3.b 3:d64eaf12:::rbd_data.115f86d8437d.000000000000001c:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:41:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.741737822000001, 2024-09-17T22:41:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068296390000000002, 2024-09-17T22:41:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36437 2024-09-17T22:41:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:41:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:41:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715035+0000", 2024-09-17T22:41:59.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.288e-06 2024-09-17T22:41:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715041+0000", 2024-09-17T22:41:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.643e-06 2024-09-17T22:41:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715051+0000", 2024-09-17T22:41:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0597e-05 2024-09-17T22:41:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715056+0000", 2024-09-17T22:41:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1379999999999998e-06 2024-09-17T22:41:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715101+0000", 2024-09-17T22:41:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4433e-05 2024-09-17T22:41:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715187+0000", 2024-09-17T22:41:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6148000000000003e-05 2024-09-17T22:41:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.716264+0000", 2024-09-17T22:41:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010770179999999999 2024-09-17T22:41:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721735+0000", 2024-09-17T22:41:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0054709019999999997 2024-09-17T22:41:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:41:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721754+0000", 2024-09-17T22:41:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9133000000000001e-05 2024-09-17T22:41:59.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721807+0000", 2024-09-17T22:41:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2831999999999998e-05 2024-09-17T22:41:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721863+0000", 2024-09-17T22:41:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6507e-05 2024-09-17T22:41:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36743 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 188416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735018+0000", 2024-09-17T22:41:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.721753707000001, 2024-09-17T22:41:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0060955260000000004, 2024-09-17T22:41:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36743 2024-09-17T22:41:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735018+0000", 2024-09-17T22:41:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735018+0000", 2024-09-17T22:41:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735018+0000", 2024-09-17T22:41:59.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7199999999999999e-07 2024-09-17T22:41:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735020+0000", 2024-09-17T22:41:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9889999999999999e-06 2024-09-17T22:41:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735027+0000", 2024-09-17T22:41:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3409999999999999e-06 2024-09-17T22:41:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735032+0000", 2024-09-17T22:41:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.904e-06 2024-09-17T22:41:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.740910+0000", 2024-09-17T22:41:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.005878266 2024-09-17T22:41:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741009+0000", 2024-09-17T22:41:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8754999999999999e-05 2024-09-17T22:41:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741113+0000", 2024-09-17T22:41:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000104799 2024-09-17T22:41:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36744 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:41:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.72173368, 2024-09-17T22:41:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063082559999999999, 2024-09-17T22:41:59.471 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36744 2024-09-17T22:41:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:41:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:41:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:41:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:41:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735040+0000", 2024-09-17T22:41:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9630000000000001e-06 2024-09-17T22:41:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735047+0000", 2024-09-17T22:41:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3439999999999997e-06 2024-09-17T22:41:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735052+0000", 2024-09-17T22:41:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9250000000000003e-06 2024-09-17T22:41:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741147+0000", 2024-09-17T22:41:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0060954210000000002 2024-09-17T22:41:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741243+0000", 2024-09-17T22:41:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5674999999999995e-05 2024-09-17T22:41:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741346+0000", 2024-09-17T22:41:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103522 2024-09-17T22:41:59.475 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36745 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:41:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.721713695999998, 2024-09-17T22:41:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065186740000000003, 2024-09-17T22:41:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.476 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36745 2024-09-17T22:41:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:41:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:41:59.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:41:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2500000000000001e-07 2024-09-17T22:41:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735061+0000", 2024-09-17T22:41:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.824e-06 2024-09-17T22:41:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735068+0000", 2024-09-17T22:41:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8580000000000002e-06 2024-09-17T22:41:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735074+0000", 2024-09-17T22:41:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5829999999999997e-06 2024-09-17T22:41:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741379+0000", 2024-09-17T22:41:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063051909999999999 2024-09-17T22:41:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741472+0000", 2024-09-17T22:41:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3238000000000004e-05 2024-09-17T22:41:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741577+0000", 2024-09-17T22:41:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000104555 2024-09-17T22:41:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36746 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:41:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.721690729999999, 2024-09-17T22:41:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067275240000000004, 2024-09-17T22:41:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36746 2024-09-17T22:41:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.481 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:41:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:41:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:41:59.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.03e-07 2024-09-17T22:41:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735083+0000", 2024-09-17T22:41:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T22:41:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735090+0000", 2024-09-17T22:41:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7800000000000003e-06 2024-09-17T22:41:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735095+0000", 2024-09-17T22:41:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8500000000000002e-06 2024-09-17T22:41:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741608+0000", 2024-09-17T22:41:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065131809999999998 2024-09-17T22:41:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741703+0000", 2024-09-17T22:41:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4914999999999995e-05 2024-09-17T22:41:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741808+0000", 2024-09-17T22:41:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000105448 2024-09-17T22:41:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36747 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:41:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.721670177, 2024-09-17T22:41:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069468790000000004, 2024-09-17T22:41:59.485 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36747 2024-09-17T22:41:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:41:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:41:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735102+0000", 2024-09-17T22:41:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:41:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735104+0000", 2024-09-17T22:41:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7400000000000001e-06 2024-09-17T22:41:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735111+0000", 2024-09-17T22:41:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4189999999999996e-06 2024-09-17T22:41:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735116+0000", 2024-09-17T22:41:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0860000000000001e-06 2024-09-17T22:41:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741847+0000", 2024-09-17T22:41:59.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067306859999999996 2024-09-17T22:41:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741942+0000", 2024-09-17T22:41:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5180999999999996e-05 2024-09-17T22:41:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742048+0000", 2024-09-17T22:41:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106387 2024-09-17T22:41:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36748 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:41:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.721648856000002, 2024-09-17T22:41:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071864119999999997, 2024-09-17T22:41:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36748 2024-09-17T22:41:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.490 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:41:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:41:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:41:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.34e-07 2024-09-17T22:41:59.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735125+0000", 2024-09-17T22:41:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0020000000000001e-06 2024-09-17T22:41:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735132+0000", 2024-09-17T22:41:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3489999999999997e-06 2024-09-17T22:41:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735138+0000", 2024-09-17T22:41:59.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9709999999999998e-06 2024-09-17T22:41:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742087+0000", 2024-09-17T22:41:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069480669999999996 2024-09-17T22:41:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742188+0000", 2024-09-17T22:41:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101279 2024-09-17T22:41:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742309+0000", 2024-09-17T22:41:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012131 2024-09-17T22:41:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36749 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:41:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.721626169, 2024-09-17T22:41:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0074112570000000001, 2024-09-17T22:41:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.494 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36749 2024-09-17T22:41:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:41:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:41:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735146+0000", 2024-09-17T22:41:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9200000000000002e-07 2024-09-17T22:41:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735158+0000", 2024-09-17T22:41:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2508e-05 2024-09-17T22:41:59.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735166+0000", 2024-09-17T22:41:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6290000000000001e-06 2024-09-17T22:41:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735174+0000", 2024-09-17T22:41:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1000000000000004e-06 2024-09-17T22:41:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742354+0000", 2024-09-17T22:41:59.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071795660000000001 2024-09-17T22:41:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742454+0000", 2024-09-17T22:41:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100408 2024-09-17T22:41:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742557+0000", 2024-09-17T22:41:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102654 2024-09-17T22:41:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.498 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36750 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:41:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.721589043000002, 2024-09-17T22:41:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0077413890000000004, 2024-09-17T22:41:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.499 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36750 2024-09-17T22:41:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:41:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:41:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:41:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2399999999999998e-07 2024-09-17T22:41:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735185+0000", 2024-09-17T22:41:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1710000000000001e-06 2024-09-17T22:41:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735192+0000", 2024-09-17T22:41:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.81e-06 2024-09-17T22:41:59.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735197+0000", 2024-09-17T22:41:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9080000000000003e-06 2024-09-17T22:41:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742587+0000", 2024-09-17T22:41:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.007390359 2024-09-17T22:41:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742694+0000", 2024-09-17T22:41:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106901 2024-09-17T22:41:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742924+0000", 2024-09-17T22:41:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022961600000000001 2024-09-17T22:41:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36751 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:41:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.721567068999999, 2024-09-17T22:41:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079273859999999998, 2024-09-17T22:41:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36751 2024-09-17T22:41:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:41:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:41:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:41:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1600000000000001e-07 2024-09-17T22:41:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735207+0000", 2024-09-17T22:41:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1909999999999999e-06 2024-09-17T22:41:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735214+0000", 2024-09-17T22:41:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9269999999999999e-06 2024-09-17T22:41:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735220+0000", 2024-09-17T22:41:59.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3739999999999997e-06 2024-09-17T22:41:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742957+0000", 2024-09-17T22:41:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0077371699999999998 2024-09-17T22:41:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.743037+0000", 2024-09-17T22:41:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0370000000000005e-05 2024-09-17T22:41:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.743132+0000", 2024-09-17T22:41:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4838000000000002e-05 2024-09-17T22:41:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37658 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749751+0000", 2024-09-17T22:41:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.707020989, 2024-09-17T22:41:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061048459999999997, 2024-09-17T22:41:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.508 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37658 2024-09-17T22:41:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749751+0000", 2024-09-17T22:41:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749751+0000", 2024-09-17T22:41:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749751+0000", 2024-09-17T22:41:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8500000000000002e-07 2024-09-17T22:41:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749753+0000", 2024-09-17T22:41:59.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2579999999999998e-06 2024-09-17T22:41:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749760+0000", 2024-09-17T22:41:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9099999999999999e-06 2024-09-17T22:41:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749765+0000", 2024-09-17T22:41:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1170000000000003e-06 2024-09-17T22:41:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755677+0000", 2024-09-17T22:41:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059120140000000002 2024-09-17T22:41:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755756+0000", 2024-09-17T22:41:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9013999999999999e-05 2024-09-17T22:41:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755855+0000", 2024-09-17T22:41:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9048000000000004e-05 2024-09-17T22:41:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.512 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37659 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:41:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.706999425999999, 2024-09-17T22:41:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063144200000000003, 2024-09-17T22:41:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.513 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37659 2024-09-17T22:41:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:41:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:41:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749773+0000", 2024-09-17T22:41:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:41:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749774+0000", 2024-09-17T22:41:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:41:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749781+0000", 2024-09-17T22:41:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5740000000000001e-06 2024-09-17T22:41:59.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749786+0000", 2024-09-17T22:41:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9099999999999996e-06 2024-09-17T22:41:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755889+0000", 2024-09-17T22:41:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061034260000000003 2024-09-17T22:41:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755972+0000", 2024-09-17T22:41:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2795999999999998e-05 2024-09-17T22:41:59.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756087+0000", 2024-09-17T22:41:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000114418 2024-09-17T22:41:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37660 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:41:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.706969588, 2024-09-17T22:41:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065587800000000002, 2024-09-17T22:41:59.517 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37660 2024-09-17T22:41:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:41:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:41:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:41:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:41:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749805+0000", 2024-09-17T22:41:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-17T22:41:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749812+0000", 2024-09-17T22:41:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270000000000002e-06 2024-09-17T22:41:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749819+0000", 2024-09-17T22:41:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.759e-06 2024-09-17T22:41:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756153+0000", 2024-09-17T22:41:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063345809999999997 2024-09-17T22:41:59.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756257+0000", 2024-09-17T22:41:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103583 2024-09-17T22:41:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756361+0000", 2024-09-17T22:41:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103853 2024-09-17T22:41:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37661 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:41:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.706940994, 2024-09-17T22:41:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067442300000000004, 2024-09-17T22:41:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37661 2024-09-17T22:41:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:41:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:41:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:41:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9399999999999995e-07 2024-09-17T22:41:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749835+0000", 2024-09-17T22:41:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1049999999999997e-06 2024-09-17T22:41:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749855+0000", 2024-09-17T22:41:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9343000000000001e-05 2024-09-17T22:41:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749860+0000", 2024-09-17T22:41:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:41:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756395+0000", 2024-09-17T22:41:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065349900000000001 2024-09-17T22:41:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756479+0000", 2024-09-17T22:41:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4030999999999996e-05 2024-09-17T22:41:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756575+0000", 2024-09-17T22:41:59.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5835999999999995e-05 2024-09-17T22:41:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37662 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:41:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.706887188, 2024-09-17T22:41:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006927295, 2024-09-17T22:41:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.526 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37662 2024-09-17T22:41:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:41:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:41:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749885+0000", 2024-09-17T22:41:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2699999999999999e-07 2024-09-17T22:41:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749888+0000", 2024-09-17T22:41:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6599999999999999e-06 2024-09-17T22:41:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749895+0000", 2024-09-17T22:41:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4200000000000001e-06 2024-09-17T22:41:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749901+0000", 2024-09-17T22:41:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5090000000000001e-06 2024-09-17T22:41:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756605+0000", 2024-09-17T22:41:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067040629999999997 2024-09-17T22:41:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756691+0000", 2024-09-17T22:41:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5846000000000004e-05 2024-09-17T22:41:59.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756812+0000", 2024-09-17T22:41:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012116999999999999 2024-09-17T22:41:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37663 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:41:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.706862978, 2024-09-17T22:41:59.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071811440000000004, 2024-09-17T22:41:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37663 2024-09-17T22:41:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:41:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:41:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:41:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:41:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749911+0000", 2024-09-17T22:41:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.035e-06 2024-09-17T22:41:59.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749917+0000", 2024-09-17T22:41:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4370000000000004e-06 2024-09-17T22:41:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749922+0000", 2024-09-17T22:41:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6249999999999998e-06 2024-09-17T22:41:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756845+0000", 2024-09-17T22:41:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069227280000000004 2024-09-17T22:41:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756994+0000", 2024-09-17T22:41:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000149068 2024-09-17T22:41:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757090+0000", 2024-09-17T22:41:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5851000000000001e-05 2024-09-17T22:41:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.534 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37664 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:41:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.706842132999999, 2024-09-17T22:41:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.007384743, 2024-09-17T22:41:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.535 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37664 2024-09-17T22:41:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:41:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:41:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749930+0000", 2024-09-17T22:41:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:41:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749933+0000", 2024-09-17T22:41:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8430000000000001e-06 2024-09-17T22:41:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749939+0000", 2024-09-17T22:41:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:41:59.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749944+0000", 2024-09-17T22:41:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9230000000000001e-06 2024-09-17T22:41:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757124+0000", 2024-09-17T22:41:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071800020000000004 2024-09-17T22:41:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757212+0000", 2024-09-17T22:41:59.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7383e-05 2024-09-17T22:41:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757314+0000", 2024-09-17T22:41:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010247399999999999 2024-09-17T22:41:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:39363 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:41:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:41:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 29.664853793999999, 2024-09-17T22:41:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006226245, 2024-09-17T22:41:59.539 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.540 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.540 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:41:59.540 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.540 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.540 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.540 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 39363 2024-09-17T22:41:59.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.540 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:41:59.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:41:59.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:41:59.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3500000000000002e-07 2024-09-17T22:41:59.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791920+0000", 2024-09-17T22:41:59.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.869e-06 2024-09-17T22:41:59.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791926+0000", 2024-09-17T22:41:59.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8490000000000002e-06 2024-09-17T22:41:59.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791931+0000", 2024-09-17T22:41:59.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4269999999999999e-06 2024-09-17T22:41:59.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.797912+0000", 2024-09-17T22:41:59.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059808819999999999 2024-09-17T22:41:59.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.798020+0000", 2024-09-17T22:41:59.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010783800000000001 2024-09-17T22:41:59.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.798144+0000", 2024-09-17T22:41:59.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012404500000000001 2024-09-17T22:41:59.543 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.543 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.543 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.544 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4447.0:51869 3.6 e35/23 3:6037e35f:::rbd_object_map.115fe9ac8535:head v 35'71, mlcod=35'66)", 2024-09-17T22:41:59.544 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:41:59.544 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 25.679531493999999, 2024-09-17T22:41:59.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0066208669999999999, 2024-09-17T22:41:59.544 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.544 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.544 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:59.544 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:41:59.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:41:59.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777241+0000", 2024-09-17T22:41:59.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4000000000000001e-07 2024-09-17T22:41:59.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777246+0000", 2024-09-17T22:41:59.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2789999999999998e-06 2024-09-17T22:41:59.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777255+0000", 2024-09-17T22:41:59.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6209999999999993e-06 2024-09-17T22:41:59.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777259+0000", 2024-09-17T22:41:59.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7650000000000001e-06 2024-09-17T22:41:59.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777298+0000", 2024-09-17T22:41:59.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8797999999999997e-05 2024-09-17T22:41:59.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777311+0000", 2024-09-17T22:41:59.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2364999999999999e-05 2024-09-17T22:41:59.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:59.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.783814+0000", 2024-09-17T22:41:59.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065039169999999997 2024-09-17T22:41:59.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.783861+0000", 2024-09-17T22:41:59.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6482e-05 2024-09-17T22:41:59.548 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.548 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.548 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.548 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:105336 3.b 3:d3549b8d:::rbd_data.115ff9e8d63c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:41:59.548 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:43.787529+0000", 2024-09-17T22:41:59.548 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 15.669242189, 2024-09-17T22:41:59.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0099864259999999996, 2024-09-17T22:41:59.548 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:41:59.548 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:41:59.548 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:59.548 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:41:59.548 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:41:59.549 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.549 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 105336 2024-09-17T22:41:59.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.549 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:41:59.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:41:59.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.787529+0000", 2024-09-17T22:41:59.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:41:59.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.787529+0000", 2024-09-17T22:41:59.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:41:59.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:41:59.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.787531+0000", 2024-09-17T22:41:59.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9759999999999998e-06 2024-09-17T22:41:59.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:41:59.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788120+0000", 2024-09-17T22:41:59.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00058899799999999997 2024-09-17T22:41:59.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:41:59.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788138+0000", 2024-09-17T22:41:59.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7393000000000001e-05 2024-09-17T22:41:59.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788146+0000", 2024-09-17T22:41:59.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8220000000000007e-06 2024-09-17T22:41:59.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:41:59.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788199+0000", 2024-09-17T22:41:59.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3718000000000003e-05 2024-09-17T22:41:59.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:41:59.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788316+0000", 2024-09-17T22:41:59.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000116999 2024-09-17T22:41:59.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:41:59.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.791608+0000", 2024-09-17T22:41:59.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003291188 2024-09-17T22:41:59.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:59.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797379+0000", 2024-09-17T22:41:59.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057716829999999997 2024-09-17T22:41:59.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:41:59.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797401+0000", 2024-09-17T22:41:59.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1729000000000002e-05 2024-09-17T22:41:59.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:41:59.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797459+0000", 2024-09-17T22:41:59.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8115000000000002e-05 2024-09-17T22:41:59.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:41:59.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:41:59.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:41:59.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797516+0000", 2024-09-17T22:41:59.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6805000000000002e-05 2024-09-17T22:41:59.553 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.553 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.553 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.553 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:41:59.553 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:41:59.554 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:41:59.555 INFO:teuthology.orchestra.run.smithi115.stdout:1751 writesame 0x4c02526 thru 0x4c09158 data_size 0x7(0x6c33 bytes) 2024-09-17T22:41:59.555 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:59.552+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:59.556 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:41:59.570 INFO:teuthology.orchestra.run.smithi115.stdout:1752 trunc from 0xaaabd25 to 0x481687c 2024-09-17T22:41:59.597 INFO:teuthology.orchestra.run.smithi115.stdout:1753 write 0x3f7c595 thru 0x3f8a0c8 (0xdb34 bytes) 2024-09-17T22:41:59.599 INFO:teuthology.orchestra.run.smithi115.stdout:1754 read 0x5b9c0 thru 0x60aea (0x512b bytes) 2024-09-17T22:41:59.599 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:59.599 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:59.599 INFO:teuthology.orchestra.run.smithi115.stdout:1758 read 0x396c6e4 thru 0x3976e13 (0xa730 bytes) 2024-09-17T22:41:59.608 INFO:teuthology.orchestra.run.smithi115.stdout:1760 write 0x76ec5b4 thru 0x76f71a2 (0xabef bytes) 2024-09-17T22:41:59.611 INFO:teuthology.orchestra.run.smithi115.stdout:1761 write 0x48bedcc thru 0x48c198f (0x2bc4 bytes) 2024-09-17T22:41:59.621 INFO:teuthology.orchestra.run.smithi115.stdout:1763 writesame 0x8c45a15 thru 0x8c492f2 data_size 0x1d(0x38de bytes) 2024-09-17T22:41:59.631 INFO:teuthology.orchestra.run.smithi115.stdout:1764 write 0xca77f78 thru 0xca821b6 (0xa23f bytes) 2024-09-17T22:41:59.646 INFO:teuthology.orchestra.run.smithi115.stdout:1765 trunc from 0xca821b7 to 0x2c95fa6 2024-09-17T22:41:59.656 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:41:59.653+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:41:59.680 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:59.680 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:59.680 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:41:59.680 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:59.692 INFO:teuthology.orchestra.run.smithi115.stdout:1766 write 0xaf6e844 thru 0xaf7672c (0x7ee9 bytes) 2024-09-17T22:41:59.694 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:41:59.695 INFO:teuthology.orchestra.run.smithi115.stdout:1767 writesame 0x504b7c9 thru 0x505ae0f data_size 0x1(0xf647 bytes) 2024-09-17T22:41:59.726 INFO:teuthology.orchestra.run.smithi115.stdout:1770 read 0x690e250 thru 0x6917318 (0x90c9 bytes) 2024-09-17T22:41:59.727 INFO:teuthology.orchestra.run.smithi115.stdout:1771 read 0x6287e3f thru 0x628b889 (0x3a4b bytes) 2024-09-17T22:41:59.737 INFO:teuthology.orchestra.run.smithi115.stdout:1772 trunc from 0xaf7672d to 0x2903d23 2024-09-17T22:41:59.752 INFO:teuthology.orchestra.run.smithi115.stdout:1773 trunc from 0x2903d23 to 0x474b37e 2024-09-17T22:41:59.757 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:41:59.754+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:41:59.758 INFO:teuthology.orchestra.run.smithi115.stdout:1774 read 0x416cc27 thru 0x4178757 (0xbb31 bytes) 2024-09-17T22:41:59.758 INFO:teuthology.orchestra.run.smithi115.stdout:1775 punch from 0x280710c to 0x280d185, (0x6079 bytes) 2024-09-17T22:41:59.769 INFO:teuthology.orchestra.run.smithi115.stdout:1776 writesame 0x8de3ac1 thru 0x8de49fb data_size 0x1(0xf3b bytes) 2024-09-17T22:41:59.769 INFO:teuthology.orchestra.run.smithi115.stdout:1777 write 0x7484db0 thru 0x74872e8 (0x2539 bytes) 2024-09-17T22:41:59.784 INFO:teuthology.orchestra.run.smithi115.stdout:1778 writesame 0xe6fda96 thru 0xe70b0a3 data_size 0x3(0xd60e bytes) 2024-09-17T22:41:59.784 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:59.784 INFO:teuthology.orchestra.run.smithi115.stdout:1780 write 0x258bc48 thru 0x259ae22 (0xf1db bytes) 2024-09-17T22:41:59.786 INFO:teuthology.orchestra.run.smithi115.stdout:1781 read 0xce9df79 thru 0xce9dfb5 (0x3d bytes) 2024-09-17T22:41:59.786 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:59.786 INFO:teuthology.orchestra.run.smithi115.stdout:1784 read 0x2351748 thru 0x235f0fc (0xd9b5 bytes) 2024-09-17T22:41:59.786 INFO:teuthology.orchestra.run.smithi115.stdout:1785 writesame 0xb62a3da thru 0xb62cb98 data_size 0x25(0x27bf bytes) 2024-09-17T22:41:59.790 INFO:teuthology.orchestra.run.smithi115.stdout:1786 read 0xc971136 thru 0xc973627 (0x24f2 bytes) 2024-09-17T22:41:59.790 INFO:teuthology.orchestra.run.smithi115.stdout:1787 read 0xc430c66 thru 0xc4373f2 (0x678d bytes) 2024-09-17T22:41:59.795 INFO:teuthology.orchestra.run.smithi115.stdout:1788 trunc from 0xe70b0a4 to 0x9adf281 2024-09-17T22:41:59.805 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:59.808 INFO:teuthology.orchestra.run.smithi115.stdout:1792 trunc from 0x9adf281 to 0xc6ad823 2024-09-17T22:41:59.810 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:59.810 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:41:59.810 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:41:59.810 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:41:59.811 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:41:59.813 INFO:teuthology.orchestra.run.smithi115.stdout:1794 punch from 0x353b2ea to 0x35469aa, (0xb6c0 bytes) 2024-09-17T22:41:59.814 INFO:teuthology.orchestra.run.smithi115.stdout:1795 write 0x9eaed62 thru 0x9eb7296 (0x8535 bytes) 2024-09-17T22:41:59.817 INFO:teuthology.orchestra.run.smithi115.stdout:1796 writesame 0x5fd56a2 thru 0x5fd8141 data_size 0x16(0x2aa0 bytes) 2024-09-17T22:41:59.820 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:41:59.820 INFO:teuthology.orchestra.run.smithi115.stdout:1797 read 0x32ce3 thru 0x3b521 (0x883f bytes) 2024-09-17T22:41:59.821 INFO:teuthology.orchestra.run.smithi115.stdout:1799 read 0x45500b8 thru 0x455aa49 (0xa992 bytes) 2024-09-17T22:41:59.821 INFO:teuthology.orchestra.run.smithi115.stdout:1800 punch from 0xb36b3eb to 0xb3760f5, (0xad0a bytes) 2024-09-17T22:41:59.821 INFO:teuthology.orchestra.run.smithi115.stdout:1801 write 0x14ad38e thru 0x14b3a84 (0x66f7 bytes) 2024-09-17T22:41:59.825 INFO:teuthology.orchestra.run.smithi115.stdout:1803 read 0xa575d68 thru 0xa577d07 (0x1fa0 bytes) 2024-09-17T22:41:59.825 INFO:teuthology.orchestra.run.smithi115.stdout:1805 read 0x1d4ed4b thru 0x1d53f08 (0x51be bytes) 2024-09-17T22:41:59.835 INFO:teuthology.orchestra.run.smithi115.stdout:1806 write 0xe420d12 thru 0xe42bb36 (0xae25 bytes) 2024-09-17T22:41:59.840 INFO:teuthology.orchestra.run.smithi115.stdout:1808 read 0xccc7668 thru 0xccd702b (0xf9c4 bytes) 2024-09-17T22:41:59.841 INFO:teuthology.orchestra.run.smithi115.stdout:1809 read 0x5df7226 thru 0x5e02901 (0xb6dc bytes) 2024-09-17T22:41:59.841 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:59.841 INFO:teuthology.orchestra.run.smithi115.stdout:1811 write 0xc164e1d thru 0xc173f11 (0xf0f5 bytes) 2024-09-17T22:41:59.842 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:59.842 INFO:teuthology.orchestra.run.smithi115.stdout:1815 write 0x84a6cd8 thru 0x84abafd (0x4e26 bytes) 2024-09-17T22:41:59.845 INFO:teuthology.orchestra.run.smithi115.stdout:1816 write 0x82564e9 thru 0x825a4ca (0x3fe2 bytes) 2024-09-17T22:41:59.848 INFO:teuthology.orchestra.run.smithi115.stdout:1819 trunc from 0xe42bb37 to 0xe23ff4a 2024-09-17T22:41:59.857 INFO:teuthology.orchestra.run.smithi115.stdout:1820 trunc from 0xe23ff4a to 0xb381160 2024-09-17T22:41:59.858 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:41:59.855+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:41:59.867 INFO:teuthology.orchestra.run.smithi115.stdout:1822 writesame 0x730e218 thru 0x730e767 data_size 0x14(0x550 bytes) 2024-09-17T22:41:59.868 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:59.880 INFO:teuthology.orchestra.run.smithi115.stdout:1824 trunc from 0xb381160 to 0x162de3b 2024-09-17T22:41:59.907 INFO:teuthology.orchestra.run.smithi115.stdout:1825 punch from 0x3e61c1 to 0x3e768f, (0x14ce bytes) 2024-09-17T22:41:59.907 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:59.914 INFO:teuthology.orchestra.run.smithi115.stdout:1828 writesame 0x2b5b264 thru 0x2b5be3a data_size 0x1(0xbd7 bytes) 2024-09-17T22:41:59.914 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:41:59.914 INFO:teuthology.orchestra.run.smithi115.stdout:1830 read 0xd8f823 thru 0xd98ed4 (0x96b2 bytes) 2024-09-17T22:41:59.928 INFO:teuthology.orchestra.run.smithi115.stdout:1832 write 0xa31e512 thru 0xa327283 (0x8d72 bytes) 2024-09-17T22:41:59.935 INFO:teuthology.orchestra.run.smithi115.stdout:1833 write 0xa4f33aa thru 0xa500c94 (0xd8eb bytes) 2024-09-17T22:41:59.939 INFO:teuthology.orchestra.run.smithi115.stdout:1834 punch from 0x148986c to 0x148ce57, (0x35eb bytes) 2024-09-17T22:41:59.939 INFO:teuthology.orchestra.run.smithi115.stdout:1835 read 0x93b1f50 thru 0x93b9e06 (0x7eb7 bytes) 2024-09-17T22:41:59.940 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:41:59.940 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:41:59.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:41:59.941 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:41:59.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.941 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:41:59.941 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:41:59.941 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.578472797000003, 2024-09-17T22:41:59.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.249158453, 2024-09-17T22:41:59.941 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:59.941 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:59.941 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:41:59.941 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:59.941 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:41:59.942 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:41:59.942 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-17T22:41:59.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.942 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:59.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:59.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:41:59.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:59.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:41:59.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:59.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358680+0000", 2024-09-17T22:41:59.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.592e-06 2024-09-17T22:41:59.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:59.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358688+0000", 2024-09-17T22:41:59.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1610000000000003e-06 2024-09-17T22:41:59.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:59.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358697+0000", 2024-09-17T22:41:59.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2580000000000001e-06 2024-09-17T22:41:59.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358706+0000", 2024-09-17T22:41:59.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1640000000000007e-06 2024-09-17T22:41:59.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358766+0000", 2024-09-17T22:41:59.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9893999999999999e-05 2024-09-17T22:41:59.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:41:59.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358778+0000", 2024-09-17T22:41:59.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1148999999999999e-05 2024-09-17T22:41:59.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606347+0000", 2024-09-17T22:41:59.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.247569971 2024-09-17T22:41:59.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:59.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606533+0000", 2024-09-17T22:41:59.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018599400000000001 2024-09-17T22:41:59.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:41:59.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606752+0000", 2024-09-17T22:41:59.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000218373 2024-09-17T22:41:59.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:41:59.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607455+0000", 2024-09-17T22:41:59.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070311799999999997 2024-09-17T22:41:59.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:41:59.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607735+0000", 2024-09-17T22:41:59.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027972300000000001 2024-09-17T22:41:59.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:41:59.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607793+0000", 2024-09-17T22:41:59.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8529999999999997e-05 2024-09-17T22:41:59.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:59.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607837+0000", 2024-09-17T22:41:59.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3526000000000003e-05 2024-09-17T22:41:59.948 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.948 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:59.948 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.948 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42346 3.1c 3:3b21ac19:::rbd_data.115fe0c30ef6.000000000000003f:5 [read 2048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:41:59.948 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.983654+0000", 2024-09-17T22:41:59.948 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.953496952999998, 2024-09-17T22:41:59.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024578903999999999, 2024-09-17T22:41:59.948 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:59.948 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:59.948 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:59.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:59.949 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:59.949 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.949 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42346 2024-09-17T22:41:59.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.949 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:59.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:59.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983654+0000", 2024-09-17T22:41:59.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:59.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983654+0000", 2024-09-17T22:41:59.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:59.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983654+0000", 2024-09-17T22:41:59.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:41:59.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:59.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983657+0000", 2024-09-17T22:41:59.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-17T22:41:59.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:59.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983662+0000", 2024-09-17T22:41:59.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9170000000000003e-06 2024-09-17T22:41:59.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983667+0000", 2024-09-17T22:41:59.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6310000000000002e-06 2024-09-17T22:41:59.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.987089+0000", 2024-09-17T22:41:59.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0034221709999999999 2024-09-17T22:41:59.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op must wait for map", 2024-09-17T22:41:59.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.987094+0000", 2024-09-17T22:41:59.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0189999999999997e-06 2024-09-17T22:41:59.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008083+0000", 2024-09-17T22:41:59.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020989102999999999 2024-09-17T22:41:59.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:59.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008153+0000", 2024-09-17T22:41:59.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9431000000000007e-05 2024-09-17T22:41:59.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:59.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008233+0000", 2024-09-17T22:41:59.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0275e-05 2024-09-17T22:41:59.953 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.953 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:59.953 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.953 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42347 3.1c 3:3b21ac19:::rbd_data.115fe0c30ef6.000000000000003f:5 [read 10240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:41:59.953 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.983673+0000", 2024-09-17T22:41:59.953 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.953478231999998, 2024-09-17T22:41:59.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024700438000000002, 2024-09-17T22:41:59.953 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:59.953 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:59.953 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:59.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:59.954 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:59.954 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.954 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42347 2024-09-17T22:41:59.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.954 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:59.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:59.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983673+0000", 2024-09-17T22:41:59.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:59.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983673+0000", 2024-09-17T22:41:59.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:59.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983673+0000", 2024-09-17T22:41:59.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-17T22:41:59.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:59.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983675+0000", 2024-09-17T22:41:59.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9259999999999999e-06 2024-09-17T22:41:59.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:59.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983681+0000", 2024-09-17T22:41:59.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.241e-06 2024-09-17T22:41:59.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983687+0000", 2024-09-17T22:41:59.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.507e-06 2024-09-17T22:41:59.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.987136+0000", 2024-09-17T22:41:59.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003448806 2024-09-17T22:41:59.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:41:59.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.987140+0000", 2024-09-17T22:41:59.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6859999999999997e-06 2024-09-17T22:41:59.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008256+0000", 2024-09-17T22:41:59.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021115465999999999 2024-09-17T22:41:59.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:59.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008310+0000", 2024-09-17T22:41:59.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4128000000000001e-05 2024-09-17T22:41:59.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:59.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008373+0000", 2024-09-17T22:41:59.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3398000000000005e-05 2024-09-17T22:41:59.958 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.958 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:59.958 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.958 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42348 3.1c 3:3b21ac19:::rbd_data.115fe0c30ef6.000000000000003f:5 [read 16384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:41:59.958 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.983693+0000", 2024-09-17T22:41:59.958 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.953458454, 2024-09-17T22:41:59.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02481359, 2024-09-17T22:41:59.958 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:59.958 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:59.958 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:59.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:59.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:59.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.959 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42348 2024-09-17T22:41:59.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.959 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:59.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:59.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983693+0000", 2024-09-17T22:41:59.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:59.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983693+0000", 2024-09-17T22:41:59.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:59.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983693+0000", 2024-09-17T22:41:59.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7800000000000002e-07 2024-09-17T22:41:59.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:59.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983695+0000", 2024-09-17T22:41:59.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.069e-06 2024-09-17T22:41:59.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:59.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983701+0000", 2024-09-17T22:41:59.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1970000000000001e-06 2024-09-17T22:41:59.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983706+0000", 2024-09-17T22:41:59.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-09-17T22:41:59.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.987166+0000", 2024-09-17T22:41:59.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0034596430000000001 2024-09-17T22:41:59.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:41:59.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.987170+0000", 2024-09-17T22:41:59.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2919999999999997e-06 2024-09-17T22:41:59.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008391+0000", 2024-09-17T22:41:59.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021221259999999999 2024-09-17T22:41:59.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:59.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008444+0000", 2024-09-17T22:41:59.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2880999999999997e-05 2024-09-17T22:41:59.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:59.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008506+0000", 2024-09-17T22:41:59.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2219999999999997e-05 2024-09-17T22:41:59.963 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.963 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:59.963 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.964 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42349 3.1c 3:3b21ac19:::rbd_data.115fe0c30ef6.000000000000003f:5 [read 24576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:41:59.964 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.983712+0000", 2024-09-17T22:41:59.964 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.953439247999999, 2024-09-17T22:41:59.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024925591, 2024-09-17T22:41:59.964 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:59.964 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:59.964 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:59.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:59.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:59.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.964 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42349 2024-09-17T22:41:59.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.965 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:59.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:59.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983712+0000", 2024-09-17T22:41:59.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:59.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983712+0000", 2024-09-17T22:41:59.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:59.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983712+0000", 2024-09-17T22:41:59.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6199999999999999e-07 2024-09-17T22:41:59.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:59.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983714+0000", 2024-09-17T22:41:59.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9690000000000001e-06 2024-09-17T22:41:59.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:59.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983720+0000", 2024-09-17T22:41:59.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8590000000000001e-06 2024-09-17T22:41:59.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983724+0000", 2024-09-17T22:41:59.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4179999999999999e-06 2024-09-17T22:41:59.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.987463+0000", 2024-09-17T22:41:59.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037386310000000001 2024-09-17T22:41:59.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:41:59.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.987469+0000", 2024-09-17T22:41:59.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.507e-06 2024-09-17T22:41:59.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008524+0000", 2024-09-17T22:41:59.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021055417 2024-09-17T22:41:59.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:59.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008576+0000", 2024-09-17T22:41:59.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1700000000000003e-05 2024-09-17T22:41:59.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:59.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008637+0000", 2024-09-17T22:41:59.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1728000000000003e-05 2024-09-17T22:41:59.969 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.969 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:59.969 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.969 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42735 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 141312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:41:59.969 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.992876+0000", 2024-09-17T22:41:59.969 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.944275333, 2024-09-17T22:41:59.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020079927000000001, 2024-09-17T22:41:59.969 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:59.969 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:59.969 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:59.970 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:59.970 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:59.970 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.970 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42735 2024-09-17T22:41:59.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.970 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:59.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:59.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992876+0000", 2024-09-17T22:41:59.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:59.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992876+0000", 2024-09-17T22:41:59.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:59.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992876+0000", 2024-09-17T22:41:59.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9999999999999999e-07 2024-09-17T22:41:59.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:59.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992878+0000", 2024-09-17T22:41:59.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6279999999999999e-06 2024-09-17T22:41:59.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:59.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992883+0000", 2024-09-17T22:41:59.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3909999999999997e-06 2024-09-17T22:41:59.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992888+0000", 2024-09-17T22:41:59.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4299999999999999e-06 2024-09-17T22:41:59.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994859+0000", 2024-09-17T22:41:59.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001971539 2024-09-17T22:41:59.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:41:59.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994863+0000", 2024-09-17T22:41:59.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.109e-06 2024-09-17T22:41:59.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.012806+0000", 2024-09-17T22:41:59.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017942909999999999 2024-09-17T22:41:59.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:59.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.012863+0000", 2024-09-17T22:41:59.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6818999999999999e-05 2024-09-17T22:41:59.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:59.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.012956+0000", 2024-09-17T22:41:59.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2800999999999995e-05 2024-09-17T22:41:59.974 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.974 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:59.974 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.974 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42736 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:41:59.974 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.992893+0000", 2024-09-17T22:41:59.974 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.944258311999999, 2024-09-17T22:41:59.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020211743000000001, 2024-09-17T22:41:59.975 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:59.975 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:59.975 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:59.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:59.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:59.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.975 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42736 2024-09-17T22:41:59.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.975 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:59.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:59.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992893+0000", 2024-09-17T22:41:59.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:59.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992893+0000", 2024-09-17T22:41:59.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:59.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992893+0000", 2024-09-17T22:41:59.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0699999999999998e-07 2024-09-17T22:41:59.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:59.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992895+0000", 2024-09-17T22:41:59.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.64e-06 2024-09-17T22:41:59.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:59.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992900+0000", 2024-09-17T22:41:59.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4090000000000002e-06 2024-09-17T22:41:59.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992905+0000", 2024-09-17T22:41:59.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6789999999999996e-06 2024-09-17T22:41:59.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994885+0000", 2024-09-17T22:41:59.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019798300000000001 2024-09-17T22:41:59.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:41:59.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994889+0000", 2024-09-17T22:41:59.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1960000000000001e-06 2024-09-17T22:41:59.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.012977+0000", 2024-09-17T22:41:59.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018088160999999998 2024-09-17T22:41:59.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:59.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013029+0000", 2024-09-17T22:41:59.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2315999999999999e-05 2024-09-17T22:41:59.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:59.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013105+0000", 2024-09-17T22:41:59.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5204999999999996e-05 2024-09-17T22:41:59.979 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.979 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:59.979 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.980 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42737 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 163840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:41:59.980 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.992910+0000", 2024-09-17T22:41:59.980 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.944241246000001, 2024-09-17T22:41:59.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020362623, 2024-09-17T22:41:59.980 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:59.980 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:59.980 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:59.980 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:59.980 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:59.980 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.980 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42737 2024-09-17T22:41:59.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.981 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:59.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:59.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992910+0000", 2024-09-17T22:41:59.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:59.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992910+0000", 2024-09-17T22:41:59.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:59.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992910+0000", 2024-09-17T22:41:59.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0800000000000001e-07 2024-09-17T22:41:59.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:59.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992912+0000", 2024-09-17T22:41:59.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.784e-06 2024-09-17T22:41:59.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:59.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992917+0000", 2024-09-17T22:41:59.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0520000000000001e-06 2024-09-17T22:41:59.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992921+0000", 2024-09-17T22:41:59.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2540000000000003e-06 2024-09-17T22:41:59.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994909+0000", 2024-09-17T22:41:59.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001987529 2024-09-17T22:41:59.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:41:59.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994913+0000", 2024-09-17T22:41:59.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1200000000000004e-06 2024-09-17T22:41:59.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013127+0000", 2024-09-17T22:41:59.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018214533000000002 2024-09-17T22:41:59.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:59.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013192+0000", 2024-09-17T22:41:59.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4979000000000002e-05 2024-09-17T22:41:59.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:59.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013272+0000", 2024-09-17T22:41:59.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0063999999999998e-05 2024-09-17T22:41:59.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.984 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:59.985 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.985 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42738 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:41:59.985 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.992927+0000", 2024-09-17T22:41:59.985 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.944223886, 2024-09-17T22:41:59.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020514726, 2024-09-17T22:41:59.985 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:59.985 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:59.985 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:59.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:59.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:59.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.985 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42738 2024-09-17T22:41:59.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.986 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:59.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:59.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992927+0000", 2024-09-17T22:41:59.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:59.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992927+0000", 2024-09-17T22:41:59.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:59.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992928+0000", 2024-09-17T22:41:59.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3000000000000001e-07 2024-09-17T22:41:59.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:59.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992932+0000", 2024-09-17T22:41:59.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:41:59.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:59.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992939+0000", 2024-09-17T22:41:59.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4239999999999998e-06 2024-09-17T22:41:59.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992944+0000", 2024-09-17T22:41:59.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9450000000000001e-06 2024-09-17T22:41:59.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994933+0000", 2024-09-17T22:41:59.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001989326 2024-09-17T22:41:59.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:41:59.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994937+0000", 2024-09-17T22:41:59.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.104e-06 2024-09-17T22:41:59.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013294+0000", 2024-09-17T22:41:59.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018356514000000001 2024-09-17T22:41:59.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:59.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013352+0000", 2024-09-17T22:41:59.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8136e-05 2024-09-17T22:41:59.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:59.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013442+0000", 2024-09-17T22:41:59.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0239000000000001e-05 2024-09-17T22:41:59.989 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.989 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:59.989 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.990 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42739 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:41:59.990 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.992949+0000", 2024-09-17T22:41:59.990 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.944201647, 2024-09-17T22:41:59.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020622234999999999, 2024-09-17T22:41:59.990 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:59.990 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:59.990 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:59.990 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:59.990 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:59.990 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.990 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42739 2024-09-17T22:41:59.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.991 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:59.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:59.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992949+0000", 2024-09-17T22:41:59.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:59.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992949+0000", 2024-09-17T22:41:59.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:59.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992950+0000", 2024-09-17T22:41:59.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.15e-07 2024-09-17T22:41:59.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:59.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992951+0000", 2024-09-17T22:41:59.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7e-06 2024-09-17T22:41:59.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:59.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992957+0000", 2024-09-17T22:41:59.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6890000000000001e-06 2024-09-17T22:41:59.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992962+0000", 2024-09-17T22:41:59.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5789999999999997e-06 2024-09-17T22:41:59.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994958+0000", 2024-09-17T22:41:59.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019959230000000001 2024-09-17T22:41:59.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:41:59.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994962+0000", 2024-09-17T22:41:59.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2490000000000003e-06 2024-09-17T22:41:59.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013462+0000", 2024-09-17T22:41:59.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018500072999999999 2024-09-17T22:41:59.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:41:59.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013512+0000", 2024-09-17T22:41:59.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0306999999999998e-05 2024-09-17T22:41:59.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:41:59.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013572+0000", 2024-09-17T22:41:59.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.94e-05 2024-09-17T22:41:59.995 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.995 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:41:59.995 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:41:59.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.995 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42740 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 186368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:41:59.995 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.992967+0000", 2024-09-17T22:41:59.995 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.944183834, 2024-09-17T22:41:59.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020729171000000001, 2024-09-17T22:41:59.995 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:41:59.995 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:41:59.995 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:41:59.996 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:41:59.996 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:41:59.996 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:41:59.996 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42740 2024-09-17T22:41:59.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.996 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:41:59.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:41:59.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992967+0000", 2024-09-17T22:41:59.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:41:59.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992967+0000", 2024-09-17T22:41:59.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:41:59.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:41:59.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992968+0000", 2024-09-17T22:41:59.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.41e-07 2024-09-17T22:41:59.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:41:59.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992969+0000", 2024-09-17T22:41:59.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.734e-06 2024-09-17T22:41:59.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:41:59.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992975+0000", 2024-09-17T22:41:59.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1909999999999999e-06 2024-09-17T22:41:59.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:41:59.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992979+0000", 2024-09-17T22:41:59.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2490000000000003e-06 2024-09-17T22:41:59.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994982+0000", 2024-09-17T22:41:59.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020027249999999999 2024-09-17T22:41:59.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:41:59.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994986+0000", 2024-09-17T22:41:59.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1400000000000002e-06 2024-09-17T22:41:59.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:41:59.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013591+0000", 2024-09-17T22:41:59.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018605486000000001 2024-09-17T22:41:59.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:41:59.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:41:59.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:00.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013642+0000", 2024-09-17T22:42:00.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0408999999999998e-05 2024-09-17T22:42:00.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:00.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013696+0000", 2024-09-17T22:42:00.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4895999999999998e-05 2024-09-17T22:42:00.000 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.000 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:00.000 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.000 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42741 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 192512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:00.001 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.992984+0000", 2024-09-17T22:42:00.001 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.944166906, 2024-09-17T22:42:00.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020833673, 2024-09-17T22:42:00.001 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:00.001 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:00.001 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:00.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:00.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:00.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:00.001 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42741 2024-09-17T22:42:00.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.001 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:00.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:00.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992984+0000", 2024-09-17T22:42:00.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:00.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:00.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992984+0000", 2024-09-17T22:42:00.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:00.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:00.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992984+0000", 2024-09-17T22:42:00.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9700000000000003e-07 2024-09-17T22:42:00.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:00.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992990+0000", 2024-09-17T22:42:00.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3940000000000004e-06 2024-09-17T22:42:00.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:00.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992996+0000", 2024-09-17T22:42:00.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.455e-06 2024-09-17T22:42:00.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:00.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993002+0000", 2024-09-17T22:42:00.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4620000000000004e-06 2024-09-17T22:42:00.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:00.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995005+0000", 2024-09-17T22:42:00.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002003379 2024-09-17T22:42:00.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:00.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995009+0000", 2024-09-17T22:42:00.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0910000000000003e-06 2024-09-17T22:42:00.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:00.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013715+0000", 2024-09-17T22:42:00.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018705894000000001 2024-09-17T22:42:00.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:00.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013764+0000", 2024-09-17T22:42:00.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9199000000000001e-05 2024-09-17T22:42:00.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:00.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013818+0000", 2024-09-17T22:42:00.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3501999999999997e-05 2024-09-17T22:42:00.005 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.005 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:00.005 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.006 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42742 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:00.006 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.993008+0000", 2024-09-17T22:42:00.006 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.944142807999999, 2024-09-17T22:42:00.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020932441999999999, 2024-09-17T22:42:00.006 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:00.006 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:00.006 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:00.006 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:00.006 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:00.006 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:00.006 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42742 2024-09-17T22:42:00.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.006 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:00.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:00.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993008+0000", 2024-09-17T22:42:00.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:00.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:00.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993008+0000", 2024-09-17T22:42:00.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:00.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:00.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993009+0000", 2024-09-17T22:42:00.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:42:00.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:00.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993011+0000", 2024-09-17T22:42:00.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.148e-06 2024-09-17T22:42:00.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:00.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993017+0000", 2024-09-17T22:42:00.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3210000000000001e-06 2024-09-17T22:42:00.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:00.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993022+0000", 2024-09-17T22:42:00.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8679999999999998e-06 2024-09-17T22:42:00.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:00.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995029+0000", 2024-09-17T22:42:00.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020069770000000001 2024-09-17T22:42:00.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:00.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995033+0000", 2024-09-17T22:42:00.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1960000000000001e-06 2024-09-17T22:42:00.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:00.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013837+0000", 2024-09-17T22:42:00.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018803469 2024-09-17T22:42:00.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:00.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013886+0000", 2024-09-17T22:42:00.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9639999999999999e-05 2024-09-17T22:42:00.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:00.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013941+0000", 2024-09-17T22:42:00.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000002e-05 2024-09-17T22:42:00.010 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.010 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:00.010 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.011 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42743 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:00.011 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.993028+0000", 2024-09-17T22:42:00.011 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.944123443999999, 2024-09-17T22:42:00.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021056887, 2024-09-17T22:42:00.011 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:00.011 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:00.011 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:00.011 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:00.011 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:00.011 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:00.011 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42743 2024-09-17T22:42:00.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.011 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:00.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:00.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993028+0000", 2024-09-17T22:42:00.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:00.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:00.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993028+0000", 2024-09-17T22:42:00.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:00.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:00.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993028+0000", 2024-09-17T22:42:00.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8700000000000001e-07 2024-09-17T22:42:00.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:00.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993030+0000", 2024-09-17T22:42:00.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8470000000000001e-06 2024-09-17T22:42:00.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:00.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993036+0000", 2024-09-17T22:42:00.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0619999999999997e-06 2024-09-17T22:42:00.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:00.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993041+0000", 2024-09-17T22:42:00.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9520000000000002e-06 2024-09-17T22:42:00.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:00.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995066+0000", 2024-09-17T22:42:00.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002025237 2024-09-17T22:42:00.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:00.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995070+0000", 2024-09-17T22:42:00.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.444e-06 2024-09-17T22:42:00.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:00.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013960+0000", 2024-09-17T22:42:00.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018890674999999999 2024-09-17T22:42:00.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:00.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014010+0000", 2024-09-17T22:42:00.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0212e-05 2024-09-17T22:42:00.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:00.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014085+0000", 2024-09-17T22:42:00.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4071000000000003e-05 2024-09-17T22:42:00.015 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.015 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:00.015 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.015 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42744 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:00.015 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.993053+0000", 2024-09-17T22:42:00.016 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.944098056000001, 2024-09-17T22:42:00.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021200131000000001, 2024-09-17T22:42:00.016 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:00.016 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:00.016 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:00.016 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:00.016 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:00.016 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:00.016 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42744 2024-09-17T22:42:00.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.016 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:00.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:00.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993053+0000", 2024-09-17T22:42:00.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:00.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:00.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993053+0000", 2024-09-17T22:42:00.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:00.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:00.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993053+0000", 2024-09-17T22:42:00.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6899999999999998e-07 2024-09-17T22:42:00.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:00.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993056+0000", 2024-09-17T22:42:00.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2400000000000002e-06 2024-09-17T22:42:00.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:00.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993062+0000", 2024-09-17T22:42:00.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3389999999999998e-06 2024-09-17T22:42:00.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:00.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993067+0000", 2024-09-17T22:42:00.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9089999999999999e-06 2024-09-17T22:42:00.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:00.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995088+0000", 2024-09-17T22:42:00.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002021134 2024-09-17T22:42:00.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:00.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995092+0000", 2024-09-17T22:42:00.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:42:00.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:00.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014107+0000", 2024-09-17T22:42:00.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019015018000000002 2024-09-17T22:42:00.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:00.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014170+0000", 2024-09-17T22:42:00.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3156000000000003e-05 2024-09-17T22:42:00.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:00.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014253+0000", 2024-09-17T22:42:00.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3435000000000003e-05 2024-09-17T22:42:00.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.020 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:00.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.021 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42745 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 229376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:00.021 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.993073+0000", 2024-09-17T22:42:00.021 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.944078099999999, 2024-09-17T22:42:00.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02150991, 2024-09-17T22:42:00.021 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:00.021 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:00.021 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:00.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:00.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:00.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:00.021 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42745 2024-09-17T22:42:00.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.021 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:00.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:00.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993073+0000", 2024-09-17T22:42:00.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:00.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:00.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993073+0000", 2024-09-17T22:42:00.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:00.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:00.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993073+0000", 2024-09-17T22:42:00.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2099999999999998e-07 2024-09-17T22:42:00.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:00.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993075+0000", 2024-09-17T22:42:00.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0439999999999998e-06 2024-09-17T22:42:00.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:00.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993082+0000", 2024-09-17T22:42:00.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2210000000000002e-06 2024-09-17T22:42:00.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:00.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993087+0000", 2024-09-17T22:42:00.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9259999999999999e-06 2024-09-17T22:42:00.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:00.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995109+0000", 2024-09-17T22:42:00.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002022505 2024-09-17T22:42:00.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:00.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995112+0000", 2024-09-17T22:42:00.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3739999999999998e-06 2024-09-17T22:42:00.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:00.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014280+0000", 2024-09-17T22:42:00.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019167857 2024-09-17T22:42:00.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:00.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014349+0000", 2024-09-17T22:42:00.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9129999999999997e-05 2024-09-17T22:42:00.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:00.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014583+0000", 2024-09-17T22:42:00.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000233532 2024-09-17T22:42:00.025 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.025 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:00.025 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.026 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42746 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:00.026 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.993092+0000", 2024-09-17T22:42:00.026 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.944058886000001, 2024-09-17T22:42:00.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021651699, 2024-09-17T22:42:00.026 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:00.026 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:00.026 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:00.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:00.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:00.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:00.026 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42746 2024-09-17T22:42:00.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.027 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:00.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:00.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993092+0000", 2024-09-17T22:42:00.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:00.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:00.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993092+0000", 2024-09-17T22:42:00.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:00.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:00.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993093+0000", 2024-09-17T22:42:00.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:42:00.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:00.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993095+0000", 2024-09-17T22:42:00.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1119999999999999e-06 2024-09-17T22:42:00.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:00.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993101+0000", 2024-09-17T22:42:00.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9630000000000003e-06 2024-09-17T22:42:00.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:00.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993105+0000", 2024-09-17T22:42:00.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5079999999999999e-06 2024-09-17T22:42:00.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:00.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995130+0000", 2024-09-17T22:42:00.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020246349999999999 2024-09-17T22:42:00.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:00.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995133+0000", 2024-09-17T22:42:00.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3809999999999999e-06 2024-09-17T22:42:00.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:00.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014611+0000", 2024-09-17T22:42:00.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019477972999999999 2024-09-17T22:42:00.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:00.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014684+0000", 2024-09-17T22:42:00.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3025e-05 2024-09-17T22:42:00.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:00.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014744+0000", 2024-09-17T22:42:00.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9695999999999999e-05 2024-09-17T22:42:00.031 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.031 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:00.031 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.031 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42747 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:00.031 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.993111+0000", 2024-09-17T22:42:00.031 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.944040016999999, 2024-09-17T22:42:00.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021790963999999999, 2024-09-17T22:42:00.031 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:00.031 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:00.031 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:00.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:00.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:00.032 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:00.032 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42747 2024-09-17T22:42:00.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.032 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:00.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:00.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993111+0000", 2024-09-17T22:42:00.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:00.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:00.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993111+0000", 2024-09-17T22:42:00.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:00.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:00.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993111+0000", 2024-09-17T22:42:00.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-09-17T22:42:00.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:00.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993114+0000", 2024-09-17T22:42:00.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.046e-06 2024-09-17T22:42:00.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:00.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993119+0000", 2024-09-17T22:42:00.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9510000000000003e-06 2024-09-17T22:42:00.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:00.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993124+0000", 2024-09-17T22:42:00.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9019999999999998e-06 2024-09-17T22:42:00.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:00.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995151+0000", 2024-09-17T22:42:00.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020268460000000001 2024-09-17T22:42:00.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:00.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995155+0000", 2024-09-17T22:42:00.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7469999999999999e-06 2024-09-17T22:42:00.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:00.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014770+0000", 2024-09-17T22:42:00.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019614691 2024-09-17T22:42:00.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:00.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014844+0000", 2024-09-17T22:42:00.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-09-17T22:42:00.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:00.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014902+0000", 2024-09-17T22:42:00.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8057000000000003e-05 2024-09-17T22:42:00.035 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.036 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:00.036 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.036 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42748 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 251904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:00.036 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.993131+0000", 2024-09-17T22:42:00.036 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.944020184999999, 2024-09-17T22:42:00.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021930076, 2024-09-17T22:42:00.036 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:00.036 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:00.036 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:00.036 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:00.036 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:00.037 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:00.037 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42748 2024-09-17T22:42:00.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.037 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:00.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:00.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993131+0000", 2024-09-17T22:42:00.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:00.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:00.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993131+0000", 2024-09-17T22:42:00.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:00.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:00.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993131+0000", 2024-09-17T22:42:00.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-09-17T22:42:00.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:00.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993133+0000", 2024-09-17T22:42:00.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.069e-06 2024-09-17T22:42:00.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:00.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993139+0000", 2024-09-17T22:42:00.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0440000000000001e-06 2024-09-17T22:42:00.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:00.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993144+0000", 2024-09-17T22:42:00.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-09-17T22:42:00.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:00.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995175+0000", 2024-09-17T22:42:00.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020309199999999999 2024-09-17T22:42:00.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:00.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995179+0000", 2024-09-17T22:42:00.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.233e-06 2024-09-17T22:42:00.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:00.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014927+0000", 2024-09-17T22:42:00.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019747712000000001 2024-09-17T22:42:00.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:00.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014996+0000", 2024-09-17T22:42:00.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9262000000000002e-05 2024-09-17T22:42:00.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:00.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.015061+0000", 2024-09-17T22:42:00.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4673999999999998e-05 2024-09-17T22:42:00.041 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.041 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:00.041 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.041 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42749 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 258048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:00.041 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.993150+0000", 2024-09-17T22:42:00.041 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.944001201999999, 2024-09-17T22:42:00.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022066032999999999, 2024-09-17T22:42:00.044 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:00.044 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:00.045 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:00.045 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:00.045 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:00.045 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:00.045 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42749 2024-09-17T22:42:00.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.045 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:00.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:00.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993150+0000", 2024-09-17T22:42:00.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:00.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:00.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993150+0000", 2024-09-17T22:42:00.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:00.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:00.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993150+0000", 2024-09-17T22:42:00.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-09-17T22:42:00.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:00.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993152+0000", 2024-09-17T22:42:00.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:42:00.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:00.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993159+0000", 2024-09-17T22:42:00.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3740000000000003e-06 2024-09-17T22:42:00.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:00.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993164+0000", 2024-09-17T22:42:00.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1030000000000001e-06 2024-09-17T22:42:00.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:00.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995199+0000", 2024-09-17T22:42:00.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020354190000000001 2024-09-17T22:42:00.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:00.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995203+0000", 2024-09-17T22:42:00.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1459999999999998e-06 2024-09-17T22:42:00.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:00.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.015087+0000", 2024-09-17T22:42:00.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019883483 2024-09-17T22:42:00.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:00.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.015156+0000", 2024-09-17T22:42:00.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9731e-05 2024-09-17T22:42:00.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:00.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:00.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:00.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.015216+0000", 2024-09-17T22:42:00.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9528000000000003e-05 2024-09-17T22:42:00.049 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.049 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:00.049 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.049 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:00.049 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:00.049 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:00.050 INFO:teuthology.orchestra.run.smithi115.stdout:1836 write 0xcb1d46f thru 0xcb1eadb (0x166d bytes) 2024-09-17T22:42:00.050 INFO:teuthology.orchestra.run.smithi115.stdout:1837 read 0x30220b2 thru 0x3030b02 (0xea51 bytes) 2024-09-17T22:42:00.051 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:42:00.051 INFO:teuthology.orchestra.run.smithi115.stdout:1838 write 0xcd4e3d6 thru 0xcd578ce (0x94f9 bytes) 2024-09-17T22:42:00.053 INFO:teuthology.orchestra.run.smithi115.stdout:get_pg_num_history pg_num_history pg_num_history(e50 pg_nums {1={15=1},2={18=8},3={23=32}} deleted_pools ) 2024-09-17T22:42:00.053 INFO:teuthology.orchestra.run.smithi115.stdout:Importing pgid 3.13 2024-09-17T22:42:00.053 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:c828dad6:::rbd_data.115fbc01575.0000000000000073:head# 2024-09-17T22:42:00.053 INFO:teuthology.orchestra.run.smithi115.stdout:snapset b=[]:{} 2024-09-17T22:42:00.053 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:c879d723:::rbd_data.115fe0c30ef6.0000000000000059:head# 2024-09-17T22:42:00.053 INFO:teuthology.orchestra.run.smithi115.stdout:snapset 5=[]:{} 2024-09-17T22:42:00.053 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:c905ebf9:::rbd_data.115fe0c30ef6.0000000000000016:head# 2024-09-17T22:42:00.053 INFO:teuthology.orchestra.run.smithi115.stdout:snapset 5=[]:{} 2024-09-17T22:42:00.053 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:c90a79a2:::rbd_data.115fe0c30ef6.000000000000005c:head# 2024-09-17T22:42:00.053 INFO:teuthology.orchestra.run.smithi115.stdout:snapset 5=[]:{} 2024-09-17T22:42:00.054 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:c94dd37f:::rbd_data.115fe0c30ef6.00000000000001af:5# 2024-09-17T22:42:00.054 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:c94dd37f:::rbd_data.115fe0c30ef6.00000000000001af:head# 2024-09-17T22:42:00.054 INFO:teuthology.orchestra.run.smithi115.stdout:snapset 5=[]:{5=[5]} 2024-09-17T22:42:00.054 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:c9ce73c5:::rbd_data.115fe0c30ef6.000000000000004b:head# 2024-09-17T22:42:00.054 INFO:teuthology.orchestra.run.smithi115.stdout:snapset 5=[]:{} 2024-09-17T22:42:00.054 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:c9ed77b7:::rbd_data.115ff9e8d63c.00000000000000aa:f# 2024-09-17T22:42:00.054 INFO:teuthology.orchestra.run.smithi115.stdout:1840 writesame 0xd9281ee thru 0xd92d7b9 data_size 0x44(0x55cc bytes) 2024-09-17T22:42:00.054 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:c9ed77b7:::rbd_data.115ff9e8d63c.00000000000000aa:head# 2024-09-17T22:42:00.055 INFO:teuthology.orchestra.run.smithi115.stdout:snapset f=[]:{f=[f]} 2024-09-17T22:42:00.055 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:c9f51540:::rbd_data.115f450dd2d3.000000000000023a:d# 2024-09-17T22:42:00.055 INFO:teuthology.orchestra.run.smithi115.stdout:1844 read 0x7d9956d thru 0x7da1c00 (0x8694 bytes) 2024-09-17T22:42:00.055 INFO:teuthology.orchestra.run.smithi115.stdout:1845 writesame 0x336e337 thru 0x336fd3b data_size 0x1(0x1a05 bytes) 2024-09-17T22:42:00.057 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:c9f51540:::rbd_data.115f450dd2d3.000000000000023a:head# 2024-09-17T22:42:00.057 INFO:teuthology.orchestra.run.smithi115.stdout:snapset d=[]:{d=[d]} 2024-09-17T22:42:00.057 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:ca2b7776:::rbd_data.115fbc01575.0000000000000095:b# 2024-09-17T22:42:00.057 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:ca2b7776:::rbd_data.115fbc01575.0000000000000095:head# 2024-09-17T22:42:00.057 INFO:teuthology.orchestra.run.smithi115.stdout:snapset b=[]:{b=[b]} 2024-09-17T22:42:00.057 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:cacd6ade:::rbd_data.115fe0c30ef6.00000000000001ae:5# 2024-09-17T22:42:00.057 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:cacd6ade:::rbd_data.115fe0c30ef6.00000000000001ae:head# 2024-09-17T22:42:00.057 INFO:teuthology.orchestra.run.smithi115.stdout:snapset 5=[]:{5=[5]} 2024-09-17T22:42:00.057 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:cb013b4b:::rbd_data.115fe0c30ef6.000000000000008d:head# 2024-09-17T22:42:00.057 INFO:teuthology.orchestra.run.smithi115.stdout:snapset 5=[]:{} 2024-09-17T22:42:00.057 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:cb9daead:::rbd_object_map.115f86d8437d:head# 2024-09-17T22:42:00.057 INFO:teuthology.orchestra.run.smithi115.stdout:snapset 0=[]:{} 2024-09-17T22:42:00.058 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:cbac0069:::rbd_data.115f450dd2d3.000000000000005b:head# 2024-09-17T22:42:00.058 INFO:teuthology.orchestra.run.smithi115.stdout:snapset d=[]:{} 2024-09-17T22:42:00.058 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:cc5087aa:::rbd_data.115fabba65f5.0000000000000004:head# 2024-09-17T22:42:00.058 INFO:teuthology.orchestra.run.smithi115.stdout:snapset 11=[]:{} 2024-09-17T22:42:00.058 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:cc524d94:::rbd_data.115fc131703f.0000000000000040:head# 2024-09-17T22:42:00.058 INFO:teuthology.orchestra.run.smithi115.stdout:snapset 0=[]:{} 2024-09-17T22:42:00.058 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:ccb197fc:::rbd_data.115f2bd7bdd2.0000000000000005:head# 2024-09-17T22:42:00.058 INFO:teuthology.orchestra.run.smithi115.stdout:snapset 0=[]:{} 2024-09-17T22:42:00.058 INFO:teuthology.orchestra.run.smithi115.stdout:1846 trunc from 0xd92d7ba to 0x6a9186b 2024-09-17T22:42:00.060 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:ccbe4cb7:::rbd_id.image_client.0-clone10:head# 2024-09-17T22:42:00.060 INFO:teuthology.orchestra.run.smithi115.stdout:snapset 0=[]:{} 2024-09-17T22:42:00.060 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:cd38a6e0:::rbd_data.115fe0c30ef6.000000000000008b:head# 2024-09-17T22:42:00.060 INFO:teuthology.orchestra.run.smithi115.stdout:snapset 5=[]:{} 2024-09-17T22:42:00.060 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:cd4faa88:::rbd_object_map.115f2bd7bdd2:head# 2024-09-17T22:42:00.060 INFO:teuthology.orchestra.run.smithi115.stdout:snapset 0=[]:{} 2024-09-17T22:42:00.060 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:cd72b7a5:::rbd_data.115fabba65f5.0000000000000066:head# 2024-09-17T22:42:00.060 INFO:teuthology.orchestra.run.smithi115.stdout:snapset 11=[]:{} 2024-09-17T22:42:00.061 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:cdcab361:::rbd_data.115fe0c30ef6.0000000000000027:head# 2024-09-17T22:42:00.061 INFO:teuthology.orchestra.run.smithi115.stdout:snapset 5=[]:{} 2024-09-17T22:42:00.061 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:cef44135:::rbd_data.115f450dd2d3.0000000000000036:head# 2024-09-17T22:42:00.061 INFO:teuthology.orchestra.run.smithi115.stdout:snapset d=[]:{} 2024-09-17T22:42:00.061 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:cf231ad5:::rbd_data.115fe0c30ef6.00000000000000b9:head# 2024-09-17T22:42:00.061 INFO:teuthology.orchestra.run.smithi115.stdout:snapset 5=[]:{} 2024-09-17T22:42:00.061 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:cf679955:::rbd_data.115f450dd2d3.0000000000000055:head# 2024-09-17T22:42:00.061 INFO:teuthology.orchestra.run.smithi115.stdout:snapset d=[]:{} 2024-09-17T22:42:00.061 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:cf856a34:::rbd_data.115fe0c30ef6.000000000000008c:head# 2024-09-17T22:42:00.061 INFO:teuthology.orchestra.run.smithi115.stdout:snapset 5=[]:{} 2024-09-17T22:42:00.061 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:cfb9cce5:::rbd_data.115fe0c30ef6.000000000000001d:5# 2024-09-17T22:42:00.061 INFO:teuthology.orchestra.run.smithi115.stdout:Write #3:cfb9cce5:::rbd_data.115fe0c30ef6.000000000000001d:head# 2024-09-17T22:42:00.061 INFO:teuthology.orchestra.run.smithi115.stdout:snapset 5=[]:{5=[5]} 2024-09-17T22:42:00.061 INFO:teuthology.orchestra.run.smithi115.stdout:write_pg epoch 50 info 3.13( v 50'249 (0'0,50'249] local-lis/les=23/24 n=32 ec=23/23 lis/c=23/23 les/c/f=24/24/0 sis=23) 2024-09-17T22:42:00.061 INFO:teuthology.orchestra.run.smithi115.stdout:Import successful 2024-09-17T22:42:00.063 INFO:teuthology.orchestra.run.smithi115.stdout:1848 write 0x77ebaca thru 0x77f91b2 (0xd6e9 bytes) 2024-09-17T22:42:00.063 INFO:teuthology.orchestra.run.smithi115.stdout:1849 read 0x34c004a thru 0x34c87d5 (0x878c bytes) 2024-09-17T22:42:00.063 INFO:teuthology.orchestra.run.smithi115.stdout:1850 write 0x4c37d2f thru 0x4c3e4da (0x67ac bytes) 2024-09-17T22:42:00.063 INFO:teuthology.orchestra.run.smithi115.stdout:1852 writesame 0xc9b88ed thru 0xc9c6f85 data_size 0x13(0xe699 bytes) 2024-09-17T22:42:00.063 INFO:teuthology.orchestra.run.smithi115.stdout:1853 punch from 0x9d34996 to 0x9d38501, (0x3b6b bytes) 2024-09-17T22:42:00.063 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:00.063 INFO:teuthology.orchestra.run.smithi115.stdout:1856 writesame 0x7372863 thru 0x737a34e data_size 0x2(0x7aec bytes) 2024-09-17T22:42:00.063 INFO:teuthology.orchestra.run.smithi115.stdout:1857 writesame 0xc09e478 thru 0xc0a79b6 data_size 0x1(0x953f bytes) 2024-09-17T22:42:00.063 INFO:teuthology.orchestra.run.smithi115.stdout:1858 trunc from 0xc9c6f86 to 0xe811684 2024-09-17T22:42:00.063 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:00.063 INFO:teuthology.orchestra.run.smithi115.stdout:1861 read 0xbbcc2ae thru 0xbbcd509 (0x125c bytes) 2024-09-17T22:42:00.063 INFO:teuthology.orchestra.run.smithi115.stdout:1862 write 0x344715d thru 0x3449315 (0x21b9 bytes) 2024-09-17T22:42:00.064 INFO:teuthology.orchestra.run.smithi115.stdout:1863 write 0x9667fd9 thru 0x966c4d9 (0x4501 bytes) 2024-09-17T22:42:00.064 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:00.047+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:00.064 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:00.064 INFO:teuthology.orchestra.run.smithi115.stdout:1865 writesame 0xb61d60c thru 0xb62a294 data_size 0x1(0xcc89 bytes) 2024-09-17T22:42:00.074 INFO:teuthology.orchestra.run.smithi115.stdout:1866 write 0xc91fdd1 thru 0xc92cbe7 (0xce17 bytes) 2024-09-17T22:42:00.077 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:00.078 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:00.078 INFO:teuthology.orchestra.run.smithi115.stdout:1870 read 0x92e65a6 thru 0x92f5d8a (0xf7e5 bytes) 2024-09-17T22:42:00.078 INFO:teuthology.orchestra.run.smithi115.stdout:1871 punch from 0xd620bcb to 0xd623d34, (0x3169 bytes) 2024-09-17T22:42:00.078 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:00.078 INFO:teuthology.orchestra.run.smithi115.stdout:1873 writesame 0xcd52e13 thru 0xcd5c6cb data_size 0x1(0x98b9 bytes) 2024-09-17T22:42:00.082 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:42:00.098 INFO:teuthology.orchestra.run.smithi115.stdout:1874 read 0x1989d7d thru 0x198ecd9 (0x4f5d bytes) 2024-09-17T22:42:00.099 INFO:teuthology.orchestra.run.smithi115.stdout:1875 write 0x588fbbb thru 0x5899dd1 (0xa217 bytes) 2024-09-17T22:42:00.102 INFO:teuthology.orchestra.run.smithi115.stdout:1876 write 0x9c49740 thru 0x9c4a9c5 (0x1286 bytes) 2024-09-17T22:42:00.108 INFO:teuthology.orchestra.run.smithi115.stdout:1879 trunc from 0xe811684 to 0xc7203b3 2024-09-17T22:42:00.115 INFO:teuthology.orchestra.run.smithi115.stdout:1880 read 0x6753ae7 thru 0x675b9e3 (0x7efd bytes) 2024-09-17T22:42:00.115 INFO:teuthology.orchestra.run.smithi115.stdout:1881 read 0x66495da thru 0x664ab40 (0x1567 bytes) 2024-09-17T22:42:00.116 INFO:teuthology.orchestra.run.smithi115.stdout:1884 trunc from 0xc7203b3 to 0xb1474d1 2024-09-17T22:42:00.133 INFO:teuthology.orchestra.run.smithi115.stdout:1885 trunc from 0xb1474d1 to 0x5de32e2 2024-09-17T22:42:00.147 INFO:teuthology.orchestra.run.smithi115.stdout:1886 punch from 0x40abb0c to 0x40b9bf0, (0xe0e4 bytes) 2024-09-17T22:42:00.149 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:42:00.150 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:00.148+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:00.152 INFO:teuthology.orchestra.run.smithi115.stdout:1888 trunc from 0x5de32e2 to 0x25fa263 2024-09-17T22:42:00.165 INFO:teuthology.orchestra.run.smithi115.stdout:1889 punch from 0x793f4d to 0x79b4f5, (0x75a8 bytes) 2024-09-17T22:42:00.166 INFO:teuthology.orchestra.run.smithi115.stdout:1890 trunc from 0x25fa263 to 0x32977ea 2024-09-17T22:42:00.182 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:00.182 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:00.182 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:00.182 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:00.191 INFO:teuthology.orchestra.run.smithi115.stdout:1891 writesame 0xc18b51b thru 0xc18dc59 data_size 0x11(0x273f bytes) 2024-09-17T22:42:00.191 INFO:teuthology.orchestra.run.smithi115.stdout:1892 read 0x35031ec thru 0x350422c (0x1041 bytes) 2024-09-17T22:42:00.191 INFO:teuthology.orchestra.run.smithi115.stdout:1893 read 0xa82c4be thru 0xa830f68 (0x4aab bytes) 2024-09-17T22:42:00.191 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:00.191 INFO:teuthology.orchestra.run.smithi115.stdout:1895 write 0xa2172fc thru 0xa21ded3 (0x6bd8 bytes) 2024-09-17T22:42:00.193 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:42:00.194 INFO:teuthology.orchestra.run.smithi115.stdout:1896 write 0xaa67b39 thru 0xaa6e926 (0x6dee bytes) 2024-09-17T22:42:00.209 INFO:teuthology.orchestra.run.smithi115.stdout:1897 trunc from 0xc18dc5a to 0x2acd20b 2024-09-17T22:42:00.210 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:00.220 INFO:teuthology.orchestra.run.smithi115.stdout:1901 read 0x199a1b thru 0x1a0295 (0x687b bytes) 2024-09-17T22:42:00.230 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:00.231 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:00.233 INFO:teuthology.orchestra.run.smithi115.stdout:1904 write 0x8b9ceb4 thru 0x8ba8d0e (0xbe5b bytes) 2024-09-17T22:42:00.237 INFO:teuthology.orchestra.run.smithi115.stdout:1905 punch from 0x850086e to 0x851007c, (0xf80e bytes) 2024-09-17T22:42:00.248 INFO:teuthology.orchestra.run.smithi115.stdout:1907 write 0xd36786b thru 0xd36d8cd (0x6063 bytes) 2024-09-17T22:42:00.250 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:00.250 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:00.251 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:00.248+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:00.259 INFO:teuthology.orchestra.run.smithi115.stdout:1910 trunc from 0xd36d8ce to 0x63089b2 2024-09-17T22:42:00.270 INFO:teuthology.orchestra.run.smithi115.stdout:1911 punch from 0x45be320 to 0x45c4647, (0x6327 bytes) 2024-09-17T22:42:00.270 INFO:teuthology.orchestra.run.smithi115.stdout:1912 write 0xed7e04 thru 0xee4464 (0xc661 bytes) 2024-09-17T22:42:00.271 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:00.271 INFO:teuthology.orchestra.run.smithi115.stdout:1915 write 0x338ceec thru 0x3393a73 (0x6b88 bytes) 2024-09-17T22:42:00.276 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:00.276 INFO:teuthology.orchestra.run.smithi115.stdout:1918 punch from 0xa78db to 0xb2b01, (0xb226 bytes) 2024-09-17T22:42:00.283 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:00.283 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:00.285 INFO:teuthology.orchestra.run.smithi115.stdout:1919 write 0x87096ce thru 0x8716c62 (0xd595 bytes) 2024-09-17T22:42:00.288 INFO:teuthology.orchestra.run.smithi115.stdout:1923 punch from 0x2da27e0 to 0x2da6e21, (0x4641 bytes) 2024-09-17T22:42:00.288 INFO:teuthology.orchestra.run.smithi115.stdout:1924 punch from 0x6228409 to 0x6232d53, (0xa94a bytes) 2024-09-17T22:42:00.289 INFO:teuthology.orchestra.run.smithi115.stdout:1925 read 0x2bbfa24 thru 0x2bc47a7 (0x4d84 bytes) 2024-09-17T22:42:00.289 INFO:teuthology.orchestra.run.smithi115.stdout:1927 read 0x1a9d62b thru 0x1aa2c95 (0x566b bytes) 2024-09-17T22:42:00.289 INFO:teuthology.orchestra.run.smithi115.stdout:1928 clone 18 order 24 su 8388608 sc 13 2024-09-17T22:42:00.314 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:00.314 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:00.332 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:00.332 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:00.336 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:00.336 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:00.349 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:42:00.351 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:00.349+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:00.356 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:00.357 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:00.373 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:00.373 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:00.376 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:00.376 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:00.452 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:00.449+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:00.460 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:00.460 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:00.469 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:42:00.502 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /var/log/ceph/exp.3.13.0 2024-09-17T22:42:00.554 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:00.551+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:00.647 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:00.716 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:00.718 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:42:00.718 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:00.715+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:00.818 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:00.816+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:00.861 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:00.861 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:00.870 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:42:00.955 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:00.952+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:01.014 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:01.014 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:01.020 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:01.017+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:01.022 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:42:01.121 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:01.119+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:01.130 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:01.130 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:01.139 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:42:01.152 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is unset 2024-09-17T22:42:01.206 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone17 from 0x8716c63 (overlap 0x162de3b) to 0x0 2024-09-17T22:42:01.223 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:01.221+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:01.270 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-17T22:42:01.289 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:01.289 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:01.298 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:42:01.339 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:01.337+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:01.424 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:01.421+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:01.443 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:01.443 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:01.451 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:42:01.459 INFO:teuthology.orchestra.run.smithi115.stdout:1929 read 0x291ddba thru 0x292393a (0x5b81 bytes) 2024-09-17T22:42:01.460 INFO:teuthology.orchestra.run.smithi115.stdout:1930 read 0x2c6f7f5 thru 0x2c7e77f (0xef8b bytes) 2024-09-17T22:42:01.461 INFO:teuthology.orchestra.run.smithi115.stdout:1931 read 0x3ef0bf5 thru 0x3f007f7 (0xfc03 bytes) 2024-09-17T22:42:01.461 INFO:teuthology.orchestra.run.smithi115.stdout:1933 writesame 0x35d6a4 thru 0x367e3e data_size 0x1(0xa79b bytes) 2024-09-17T22:42:01.493 INFO:teuthology.orchestra.run.smithi115.stdout:1934 write 0x55b4a4a thru 0x55b8fcf (0x4586 bytes) 2024-09-17T22:42:01.496 INFO:teuthology.orchestra.run.smithi115.stdout:1937 punch from 0x4411c31 to 0x441bffc, (0xa3cb bytes) 2024-09-17T22:42:01.496 INFO:teuthology.orchestra.run.smithi115.stdout:1938 writesame 0xaa8b72 thru 0xaaa4ab data_size 0x1(0x193a bytes) 2024-09-17T22:42:01.500 INFO:teuthology.orchestra.run.smithi115.stdout:1939 punch from 0x1ee2613 to 0x1eea3ed, (0x7dda bytes) 2024-09-17T22:42:01.503 INFO:teuthology.orchestra.run.smithi115.stdout:1940 trunc from 0x8716c63 to 0x5f2905c 2024-09-17T22:42:01.526 INFO:teuthology.orchestra.run.smithi115.stdout:1941 trunc from 0x5f2905c to 0xbfb6c26 2024-09-17T22:42:01.531 INFO:teuthology.orchestra.run.smithi115.stdout:1942 read 0x9fa59bc thru 0x9fb2410 (0xca55 bytes) 2024-09-17T22:42:01.531 INFO:teuthology.orchestra.run.smithi115.stdout:1943 read 0x10de952 thru 0x10ebed7 (0xd586 bytes) 2024-09-17T22:42:01.532 INFO:teuthology.orchestra.run.smithi115.stdout:1945 read 0x2382230 thru 0x238318f (0xf60 bytes) 2024-09-17T22:42:01.532 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:01.532 INFO:teuthology.orchestra.run.smithi115.stdout:1947 read 0x12a8a2f thru 0x12b673d (0xdd0f bytes) 2024-09-17T22:42:01.533 INFO:teuthology.orchestra.run.smithi115.stdout:1948 punch from 0x1eb7fdb to 0x1ec6564, (0xe589 bytes) 2024-09-17T22:42:01.533 INFO:teuthology.orchestra.run.smithi115.stdout:1949 writesame 0x10c8049 thru 0x10cb22e data_size 0x3(0x31e6 bytes) 2024-09-17T22:42:01.538 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:01.536+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:01.549 INFO:teuthology.orchestra.run.smithi115.stdout:1950 trunc from 0xbfb6c26 to 0x4b9f35 2024-09-17T22:42:01.584 INFO:teuthology.orchestra.run.smithi115.stdout:1951 write 0xeded69d thru 0xedf491c (0x7280 bytes) 2024-09-17T22:42:01.587 INFO:teuthology.orchestra.run.smithi115.stdout:1952 write 0x86113fa thru 0x8620bc2 (0xf7c9 bytes) 2024-09-17T22:42:01.589 INFO:teuthology.orchestra.run.smithi115.stdout:1953 write 0x596e215 thru 0x5971f54 (0x3d40 bytes) 2024-09-17T22:42:01.591 INFO:teuthology.orchestra.run.smithi115.stdout:1954 punch from 0x71fca42 to 0x7200925, (0x3ee3 bytes) 2024-09-17T22:42:01.592 INFO:teuthology.orchestra.run.smithi115.stdout:1955 punch from 0xbe5bab to 0xbef1ff, (0x9654 bytes) 2024-09-17T22:42:01.592 INFO:teuthology.orchestra.run.smithi115.stdout:1956 punch from 0xa1a7f48 to 0xa1b0cbd, (0x8d75 bytes) 2024-09-17T22:42:01.592 INFO:teuthology.orchestra.run.smithi115.stdout:1957 write 0x8880ed2 thru 0x8886dd4 (0x5f03 bytes) 2024-09-17T22:42:01.594 INFO:teuthology.orchestra.run.smithi115.stdout:1958 read 0xa5f1d02 thru 0xa5f2580 (0x87f bytes) 2024-09-17T22:42:01.595 INFO:teuthology.orchestra.run.smithi115.stdout:1960 read 0xd509c11 thru 0xd518cb2 (0xf0a2 bytes) 2024-09-17T22:42:01.595 INFO:teuthology.orchestra.run.smithi115.stdout:1961 read 0x11cde43 thru 0x11cf0a3 (0x1261 bytes) 2024-09-17T22:42:01.595 INFO:teuthology.orchestra.run.smithi115.stdout:truncating to largest ever: 0xecfde90 2024-09-17T22:42:01.595 INFO:teuthology.orchestra.run.smithi115.stdout:1964 trunc from 0xedf491d to 0xecfde90 2024-09-17T22:42:01.600 INFO:teuthology.orchestra.run.smithi115.stdout:1966 punch from 0x74157f3 to 0x741b830, (0x603d bytes) 2024-09-17T22:42:01.600 INFO:teuthology.orchestra.run.smithi115.stdout:1968 punch from 0x4a864f8 to 0x4a8b250, (0x4d58 bytes) 2024-09-17T22:42:01.600 INFO:teuthology.orchestra.run.smithi115.stdout:1969 writesame 0x6151bc4 thru 0x6155745 data_size 0x3(0x3b82 bytes) 2024-09-17T22:42:01.601 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:01.602 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:01.604 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:01.604 INFO:teuthology.orchestra.run.smithi115.stdout:1972 write 0xddc14d2 thru 0xddcebe3 (0xd712 bytes) 2024-09-17T22:42:01.607 INFO:teuthology.orchestra.run.smithi115.stdout:1974 punch from 0x4d0bf7e to 0x4d0d192, (0x1214 bytes) 2024-09-17T22:42:01.611 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:42:01.622 INFO:teuthology.orchestra.run.smithi115.stdout:1976 trunc from 0xecfde90 to 0x17db0e1 2024-09-17T22:42:01.638 INFO:teuthology.orchestra.run.smithi115.stdout:1977 punch from 0x245048 to 0x251719, (0xc6d1 bytes) 2024-09-17T22:42:01.638 INFO:teuthology.orchestra.run.smithi115.stdout:1978 read 0xb37cd4 thru 0xb45d35 (0xe062 bytes) 2024-09-17T22:42:01.651 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:01.649+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:01.657 INFO:teuthology.orchestra.run.smithi115.stdout:1979 write 0xd02cd5d thru 0xd039a93 (0xcd37 bytes) 2024-09-17T22:42:01.660 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:01.660 INFO:teuthology.orchestra.run.smithi115.stdout:1982 write 0x8924839 thru 0x89329ce (0xe196 bytes) 2024-09-17T22:42:01.663 INFO:teuthology.orchestra.run.smithi115.stdout:1984 write 0x9f8019e thru 0x9f83ca2 (0x3b05 bytes) 2024-09-17T22:42:01.665 INFO:teuthology.orchestra.run.smithi115.stdout:1985 write 0x9a0f0d2 thru 0x9a11626 (0x2555 bytes) 2024-09-17T22:42:01.666 INFO:teuthology.orchestra.run.smithi115.stdout:1986 read 0xb5c9012 thru 0xb5cf401 (0x63f0 bytes) 2024-09-17T22:42:01.666 INFO:teuthology.orchestra.run.smithi115.stdout:1987 write 0x8916188 thru 0x89161f9 (0x72 bytes) 2024-09-17T22:42:01.668 INFO:teuthology.orchestra.run.smithi115.stdout:1988 read 0x46752c3 thru 0x4677f93 (0x2cd1 bytes) 2024-09-17T22:42:01.668 INFO:teuthology.orchestra.run.smithi115.stdout:1989 read 0x4df7568 thru 0x4dfdc2c (0x66c5 bytes) 2024-09-17T22:42:01.668 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:01.668 INFO:teuthology.orchestra.run.smithi115.stdout:1991 punch from 0xce2d643 to 0xce34f2e, (0x78eb bytes) 2024-09-17T22:42:01.668 INFO:teuthology.orchestra.run.smithi115.stdout:1993 write 0x5f10b56 thru 0x5f131de (0x2689 bytes) 2024-09-17T22:42:01.670 INFO:teuthology.orchestra.run.smithi115.stdout:1995 writesame 0x65b44c5 thru 0x65c157f data_size 0x1(0xd0bb bytes) 2024-09-17T22:42:01.697 INFO:teuthology.orchestra.run.smithi115.stdout:1996 trunc from 0xd039a94 to 0xae4235f 2024-09-17T22:42:01.708 INFO:teuthology.orchestra.run.smithi115.stdout:1997 read 0x36c7b6a thru 0x36d0410 (0x88a7 bytes) 2024-09-17T22:42:01.709 INFO:teuthology.orchestra.run.smithi115.stdout:2004 read 0xd8e587 thru 0xd9c86d (0xe2e7 bytes) 2024-09-17T22:42:01.720 INFO:teuthology.orchestra.run.smithi115.stdout:2005 trunc from 0xae4235f to 0x115efcc 2024-09-17T22:42:01.726 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:01.724+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:01.737 INFO:teuthology.orchestra.run.smithi115.stdout:2006 read 0xf5179e thru 0xf52edc (0x173f bytes) 2024-09-17T22:42:01.737 INFO:teuthology.orchestra.run.smithi115.stdout:2007 read 0x112005b thru 0x112cd68 (0xcd0e bytes) 2024-09-17T22:42:01.747 INFO:teuthology.orchestra.run.smithi115.stdout:2008 write 0x4175d6d thru 0x41821b5 (0xc449 bytes) 2024-09-17T22:42:01.750 INFO:teuthology.orchestra.run.smithi115.stdout:2009 punch from 0x2651ed to 0x27232d, (0xd140 bytes) 2024-09-17T22:42:01.753 INFO:teuthology.orchestra.run.smithi115.stdout:2010 trunc from 0x41821b6 to 0x1801df2 2024-09-17T22:42:01.755 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:01.755 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:01.765 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:42:01.774 INFO:teuthology.orchestra.run.smithi115.stdout:2011 write 0x6d6d7ca thru 0x6d7a4ec (0xcd23 bytes) 2024-09-17T22:42:01.776 INFO:teuthology.orchestra.run.smithi115.stdout:2012 writesame 0x293a5cd thru 0x293c2e0 data_size 0x2(0x1d14 bytes) 2024-09-17T22:42:01.780 INFO:teuthology.orchestra.run.smithi115.stdout:2013 writesame 0x245e896 thru 0x246697c data_size 0x1(0x80e7 bytes) 2024-09-17T22:42:01.796 INFO:teuthology.orchestra.run.smithi115.stdout:2014 writesame 0x42891f7 thru 0x4290215 data_size 0x1(0x701f bytes) 2024-09-17T22:42:01.810 INFO:teuthology.orchestra.run.smithi115.stdout:2015 read 0xb83bd5 thru 0xb87ba9 (0x3fd5 bytes) 2024-09-17T22:42:01.822 INFO:teuthology.orchestra.run.smithi115.stdout:2016 writesame 0xbd20381 thru 0xbd26a74 data_size 0x4(0x66f4 bytes) 2024-09-17T22:42:01.822 INFO:teuthology.orchestra.run.smithi115.stdout:2017 write 0x744ff7b thru 0x7456002 (0x6088 bytes) 2024-09-17T22:42:01.824 INFO:teuthology.orchestra.run.smithi115.stdout:2018 writesame 0x84553dd thru 0x845c86a data_size 0x3(0x748e bytes) 2024-09-17T22:42:01.827 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:01.825+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:01.831 INFO:teuthology.orchestra.run.smithi115.stdout:2019 punch from 0xb3b03f9 to 0xb3ba9e6, (0xa5ed bytes) 2024-09-17T22:42:01.831 INFO:teuthology.orchestra.run.smithi115.stdout:2020 read 0x60651df thru 0x6074a3a (0xf85c bytes) 2024-09-17T22:42:01.831 INFO:teuthology.orchestra.run.smithi115.stdout:2021 write 0x1e5f3f5 thru 0x1e6127c (0x1e88 bytes) 2024-09-17T22:42:01.832 INFO:teuthology.orchestra.run.smithi115.stdout:2023 punch from 0x2661200 to 0x266b887, (0xa687 bytes) 2024-09-17T22:42:01.832 INFO:teuthology.orchestra.run.smithi115.stdout:2024 read 0x33d7591 thru 0x33df155 (0x7bc5 bytes) 2024-09-17T22:42:01.832 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:01.842 INFO:teuthology.orchestra.run.smithi115.stdout:2026 trunc from 0xbd26a75 to 0x3416113 2024-09-17T22:42:01.852 INFO:teuthology.orchestra.run.smithi115.stdout:2027 punch from 0x311098c to 0x3112520, (0x1b94 bytes) 2024-09-17T22:42:01.852 INFO:teuthology.orchestra.run.smithi115.stdout:2030 punch from 0x206a898 to 0x2073c17, (0x937f bytes) 2024-09-17T22:42:01.852 INFO:teuthology.orchestra.run.smithi115.stdout:2031 punch from 0x1315c85 to 0x131ce64, (0x71df bytes) 2024-09-17T22:42:01.852 INFO:teuthology.orchestra.run.smithi115.stdout:2032 read 0x1d5e1fd thru 0x1d6cbfb (0xe9ff bytes) 2024-09-17T22:42:01.852 INFO:teuthology.orchestra.run.smithi115.stdout:2033 read 0x3405d1 thru 0x3450a8 (0x4ad8 bytes) 2024-09-17T22:42:01.852 INFO:teuthology.orchestra.run.smithi115.stdout:2034 clone 19 order 25 su 262144 sc 2 2024-09-17T22:42:01.872 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:01.872 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:01.881 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:42:01.928 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:01.926+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:02.024 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:02.024 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:02.028 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:02.027+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:02.032 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:42:02.130 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:02.127+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:02.134 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:02.134 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:02.143 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:42:02.222 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone18 from 0x3416113 (overlap 0x4b9f35) to 0x440f49 2024-09-17T22:42:02.231 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:02.229+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:02.286 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-17T22:42:02.287 INFO:teuthology.orchestra.run.smithi115.stdout:2036 read 0x1ceea38 thru 0x1cf0c6c (0x2235 bytes) 2024-09-17T22:42:02.288 INFO:teuthology.orchestra.run.smithi115.stdout:2037 read 0x2e02f49 thru 0x2e10eda (0xdf92 bytes) 2024-09-17T22:42:02.291 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:02.291 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:02.300 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:42:02.301 INFO:teuthology.orchestra.run.smithi115.stdout:2039 write 0x647a765 thru 0x647f86b (0x5107 bytes) 2024-09-17T22:42:02.305 INFO:teuthology.orchestra.run.smithi115.stdout:2040 trunc from 0x647f86c to 0x6b49b18 2024-09-17T22:42:02.311 INFO:teuthology.orchestra.run.smithi115.stdout:2041 punch from 0x3f8e2a2 to 0x3f8e5ae, (0x30c bytes) 2024-09-17T22:42:02.311 INFO:teuthology.orchestra.run.smithi115.stdout:2042 punch from 0x14f71cd to 0x14f7280, (0xb3 bytes) 2024-09-17T22:42:02.322 INFO:teuthology.orchestra.run.smithi115.stdout:2043 write 0xb039b47 thru 0xb03e3ed (0x48a7 bytes) 2024-09-17T22:42:02.326 INFO:teuthology.orchestra.run.smithi115.stdout:2044 read 0xa524e35 thru 0xa52ecec (0x9eb8 bytes) 2024-09-17T22:42:02.327 INFO:teuthology.orchestra.run.smithi115.stdout:2047 writesame 0x5a0079f thru 0x5a082ee data_size 0x1(0x7b50 bytes) 2024-09-17T22:42:02.339 INFO:teuthology.orchestra.run.smithi115.stdout:2048 writesame 0x3a6dd14 thru 0x3a7c16b data_size 0x4(0xe458 bytes) 2024-09-17T22:42:02.344 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:02.342+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:02.356 INFO:teuthology.orchestra.run.smithi115.stdout:2049 write 0x935c4b0 thru 0x9362f5f (0x6ab0 bytes) 2024-09-17T22:42:02.359 INFO:teuthology.orchestra.run.smithi115.stdout:2050 punch from 0x5fadb3d to 0x5fb53c3, (0x7886 bytes) 2024-09-17T22:42:02.359 INFO:teuthology.orchestra.run.smithi115.stdout:2052 write 0x7a033c9 thru 0x7a03f38 (0xb70 bytes) 2024-09-17T22:42:02.360 INFO:teuthology.orchestra.run.smithi115.stdout:2053 write 0x52589bd thru 0x52648af (0xbef3 bytes) 2024-09-17T22:42:02.372 INFO:teuthology.orchestra.run.smithi115.stdout:2054 trunc from 0xb03e3ee to 0x20d8578 2024-09-17T22:42:02.378 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:42:02.413 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:02.414 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:02.416 INFO:teuthology.orchestra.run.smithi115.stdout:2055 writesame 0x9856dc7 thru 0x98580e9 data_size 0x17(0x1323 bytes) 2024-09-17T22:42:02.416 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:02.416 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:02.416 INFO:teuthology.orchestra.run.smithi115.stdout:2059 punch from 0x59a7762 to 0x59a85bc, (0xe5a bytes) 2024-09-17T22:42:02.416 INFO:teuthology.orchestra.run.smithi115.stdout:2060 punch from 0x2e27cf8 to 0x2e2f0f9, (0x7401 bytes) 2024-09-17T22:42:02.416 INFO:teuthology.orchestra.run.smithi115.stdout:2063 write 0x7aebe6f thru 0x7af93a4 (0xd536 bytes) 2024-09-17T22:42:02.418 INFO:teuthology.orchestra.run.smithi115.stdout:2064 writesame 0x81b9408 thru 0x81beb55 data_size 0x4b(0x574e bytes) 2024-09-17T22:42:02.423 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:42:02.433 INFO:teuthology.orchestra.run.smithi115.stdout:2065 write 0xe912979 thru 0xe914ee4 (0x256c bytes) 2024-09-17T22:42:02.437 INFO:teuthology.orchestra.run.smithi115.stdout:2066 read 0x7581160 thru 0x758aa28 (0x98c9 bytes) 2024-09-17T22:42:02.437 INFO:teuthology.orchestra.run.smithi115.stdout:2067 writesame 0x1a01398 thru 0x1a0472f data_size 0x1(0x3398 bytes) 2024-09-17T22:42:02.443 INFO:teuthology.orchestra.run.smithi115.stdout:2069 write 0x4b0c3bb thru 0x4b0d473 (0x10b9 bytes) 2024-09-17T22:42:02.445 INFO:teuthology.orchestra.run.smithi115.stdout:2070 write 0x2285a8b thru 0x228d785 (0x7cfb bytes) 2024-09-17T22:42:02.447 INFO:teuthology.orchestra.run.smithi115.stdout:2071 punch from 0xae4db23 to 0xae526e2, (0x4bbf bytes) 2024-09-17T22:42:02.447 INFO:teuthology.orchestra.run.smithi115.stdout:2072 punch from 0x7544474 to 0x754d910, (0x949c bytes) 2024-09-17T22:42:02.447 INFO:teuthology.orchestra.run.smithi115.stdout:2073 read 0x4a93241 thru 0x4a9b88d (0x864d bytes) 2024-09-17T22:42:02.452 INFO:teuthology.orchestra.run.smithi115.stdout:2074 write 0xed5a234 thru 0xed614a6 (0x7273 bytes) 2024-09-17T22:42:02.455 INFO:teuthology.orchestra.run.smithi115.stdout:2075 write 0x8a54285 thru 0x8a545b0 (0x32c bytes) 2024-09-17T22:42:02.457 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:02.458 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:02.464 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:02.462+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:02.465 INFO:teuthology.orchestra.run.smithi115.stdout:2078 trunc from 0xed614a7 to 0x88dad48 2024-09-17T22:42:02.480 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:02.481 INFO:teuthology.orchestra.run.smithi115.stdout:2082 writesame 0x603203f thru 0x603ec83 data_size 0x1(0xcc45 bytes) 2024-09-17T22:42:02.491 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:02.496 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:02.496 INFO:teuthology.orchestra.run.smithi115.stdout:2085 read 0x9b85a3 thru 0x9b908e (0xaec bytes) 2024-09-17T22:42:02.497 INFO:teuthology.orchestra.run.smithi115.stdout:2086 read 0x31a055e thru 0x31acdc3 (0xc866 bytes) 2024-09-17T22:42:02.497 INFO:teuthology.orchestra.run.smithi115.stdout:2088 write 0x615b173 thru 0x615efb9 (0x3e47 bytes) 2024-09-17T22:42:02.498 INFO:teuthology.orchestra.run.smithi115.stdout:2089 writesame 0x667b2a8 thru 0x66821e9 data_size 0x2f(0x6f42 bytes) 2024-09-17T22:42:02.509 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:02.509 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:02.512 INFO:teuthology.orchestra.run.smithi115.stdout:2090 write 0xe7f9573 thru 0xe804cbb (0xb749 bytes) 2024-09-17T22:42:02.515 INFO:teuthology.orchestra.run.smithi115.stdout:2091 write 0xd697e4b thru 0xd69dbb0 (0x5d66 bytes) 2024-09-17T22:42:02.517 INFO:teuthology.orchestra.run.smithi115.stdout:2092 writesame 0xa67d823 thru 0xa682de5 data_size 0x1(0x55c3 bytes) 2024-09-17T22:42:02.523 INFO:teuthology.orchestra.run.smithi115.stdout:2094 writesame 0x6c47f45 thru 0x6c510aa data_size 0x2(0x9166 bytes) 2024-09-17T22:42:02.527 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:02.527 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:02.534 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:02.532+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:02.551 INFO:teuthology.orchestra.run.smithi115.stdout:2096 trunc from 0xe804cbc to 0x8216cc 2024-09-17T22:42:02.555 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:02.555 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:02.572 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:02.572 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:02.581 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:42:02.582 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:02.582 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:02.583 INFO:teuthology.orchestra.run.smithi115.stdout:2098 writesame 0xc2d8734 thru 0xc2e42d0 data_size 0x1(0xbb9d bytes) 2024-09-17T22:42:02.584 INFO:teuthology.orchestra.run.smithi115.stdout:2099 read 0x2c20307 thru 0x2c20ec5 (0xbbf bytes) 2024-09-17T22:42:02.584 INFO:teuthology.orchestra.run.smithi115.stdout:2100 read 0x7898406 thru 0x78a7dae (0xf9a9 bytes) 2024-09-17T22:42:02.584 INFO:teuthology.orchestra.run.smithi115.stdout:2102 writesame 0x447bebb thru 0x447c3b7 data_size 0x1(0x4fd bytes) 2024-09-17T22:42:02.587 INFO:teuthology.orchestra.run.smithi115.stdout:2103 read 0x3404b93 thru 0x340daf9 (0x8f67 bytes) 2024-09-17T22:42:02.587 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:02.592 INFO:teuthology.orchestra.run.smithi115.stdout:2105 trunc from 0xc2e42d1 to 0x7d81443 2024-09-17T22:42:02.599 INFO:teuthology.orchestra.run.smithi115.stdout:2106 read 0x43c19e1 thru 0x43cda3e (0xc05e bytes) 2024-09-17T22:42:02.599 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:02.599 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:02.600 INFO:teuthology.orchestra.run.smithi115.stdout:2107 punch from 0x210219e to 0x2109886, (0x76e8 bytes) 2024-09-17T22:42:02.600 INFO:teuthology.orchestra.run.smithi115.stdout:2108 read 0x78bef44 thru 0x78c3750 (0x480d bytes) 2024-09-17T22:42:02.600 INFO:teuthology.orchestra.run.smithi115.stdout:2110 read 0x207c548 thru 0x20808ed (0x43a6 bytes) 2024-09-17T22:42:02.607 INFO:teuthology.orchestra.run.smithi115.stdout:2111 trunc from 0x7d81443 to 0xe2da172 2024-09-17T22:42:02.617 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:02.617 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:02.624 INFO:teuthology.orchestra.run.smithi115.stdout:2113 trunc from 0xe2da172 to 0x45da325 2024-09-17T22:42:02.631 INFO:teuthology.orchestra.run.smithi115.stdout:2116 read 0x3890eb3 thru 0x3898aa0 (0x7bee bytes) 2024-09-17T22:42:02.631 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:02.631 INFO:teuthology.orchestra.run.smithi115.stdout:2118 read 0x305e9b0 thru 0x3065cdf (0x7330 bytes) 2024-09-17T22:42:02.632 INFO:teuthology.orchestra.run.smithi115.stdout:2119 read 0x1f27671 thru 0x1f27ceb (0x67b bytes) 2024-09-17T22:42:02.634 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:02.635 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:02.635 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:02.633+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:02.638 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:02.639 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:02.643 INFO:teuthology.orchestra.run.smithi115.stdout:2121 write 0x92a440c thru 0x92ab272 (0x6e67 bytes) 2024-09-17T22:42:02.653 INFO:teuthology.orchestra.run.smithi115.stdout:2122 writesame 0xb14d3db thru 0xb14fe9f data_size 0x1(0x2ac5 bytes) 2024-09-17T22:42:02.653 INFO:teuthology.orchestra.run.smithi115.stdout:2124 read 0x51077df thru 0x51132d2 (0xbaf4 bytes) 2024-09-17T22:42:02.654 INFO:teuthology.orchestra.run.smithi115.stdout:2125 trunc from 0xb14fea0 to 0xa45d6d0 2024-09-17T22:42:02.670 INFO:teuthology.orchestra.run.smithi115.stdout:2126 write 0xd75b0a5 thru 0xd764767 (0x96c3 bytes) 2024-09-17T22:42:02.674 INFO:teuthology.orchestra.run.smithi115.stdout:2127 write 0x937fb89 thru 0x938e663 (0xeadb bytes) 2024-09-17T22:42:02.677 INFO:teuthology.orchestra.run.smithi115.stdout:2128 read 0x6f814be thru 0x6f85989 (0x44cc bytes) 2024-09-17T22:42:02.690 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36437 3.b 3:d64eaf12:::rbd_data.115f86d8437d.000000000000001c:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:42:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.971441818000002, 2024-09-17T22:42:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068296390000000002, 2024-09-17T22:42:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.690 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36437 2024-09-17T22:42:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:42:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:42:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715035+0000", 2024-09-17T22:42:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.288e-06 2024-09-17T22:42:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715041+0000", 2024-09-17T22:42:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.643e-06 2024-09-17T22:42:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715051+0000", 2024-09-17T22:42:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0597e-05 2024-09-17T22:42:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715056+0000", 2024-09-17T22:42:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1379999999999998e-06 2024-09-17T22:42:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715101+0000", 2024-09-17T22:42:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4433e-05 2024-09-17T22:42:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715187+0000", 2024-09-17T22:42:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6148000000000003e-05 2024-09-17T22:42:02.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:42:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.716264+0000", 2024-09-17T22:42:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010770179999999999 2024-09-17T22:42:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721735+0000", 2024-09-17T22:42:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0054709019999999997 2024-09-17T22:42:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:42:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721754+0000", 2024-09-17T22:42:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9133000000000001e-05 2024-09-17T22:42:02.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721807+0000", 2024-09-17T22:42:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2831999999999998e-05 2024-09-17T22:42:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721863+0000", 2024-09-17T22:42:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6507e-05 2024-09-17T22:42:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36744 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:42:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.951437675999998, 2024-09-17T22:42:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063082559999999999, 2024-09-17T22:42:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36744 2024-09-17T22:42:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.697 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:42:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:42:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:42:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:42:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735040+0000", 2024-09-17T22:42:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9630000000000001e-06 2024-09-17T22:42:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735047+0000", 2024-09-17T22:42:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3439999999999997e-06 2024-09-17T22:42:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735052+0000", 2024-09-17T22:42:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9250000000000003e-06 2024-09-17T22:42:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741147+0000", 2024-09-17T22:42:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0060954210000000002 2024-09-17T22:42:02.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741243+0000", 2024-09-17T22:42:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5674999999999995e-05 2024-09-17T22:42:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741346+0000", 2024-09-17T22:42:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103522 2024-09-17T22:42:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.701 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36745 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:42:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.951417692, 2024-09-17T22:42:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065186740000000003, 2024-09-17T22:42:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36745 2024-09-17T22:42:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:42:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:42:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:42:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2500000000000001e-07 2024-09-17T22:42:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735061+0000", 2024-09-17T22:42:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.824e-06 2024-09-17T22:42:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735068+0000", 2024-09-17T22:42:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8580000000000002e-06 2024-09-17T22:42:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735074+0000", 2024-09-17T22:42:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5829999999999997e-06 2024-09-17T22:42:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741379+0000", 2024-09-17T22:42:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063051909999999999 2024-09-17T22:42:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741472+0000", 2024-09-17T22:42:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3238000000000004e-05 2024-09-17T22:42:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741577+0000", 2024-09-17T22:42:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000104555 2024-09-17T22:42:02.705 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36746 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:42:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.951394725999997, 2024-09-17T22:42:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067275240000000004, 2024-09-17T22:42:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:02.706 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36746 2024-09-17T22:42:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:42:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:42:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:42:02.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.03e-07 2024-09-17T22:42:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735083+0000", 2024-09-17T22:42:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T22:42:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735090+0000", 2024-09-17T22:42:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7800000000000003e-06 2024-09-17T22:42:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735095+0000", 2024-09-17T22:42:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8500000000000002e-06 2024-09-17T22:42:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741608+0000", 2024-09-17T22:42:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065131809999999998 2024-09-17T22:42:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741703+0000", 2024-09-17T22:42:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4914999999999995e-05 2024-09-17T22:42:02.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741808+0000", 2024-09-17T22:42:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000105448 2024-09-17T22:42:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36747 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:42:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.951374172999998, 2024-09-17T22:42:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069468790000000004, 2024-09-17T22:42:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.710 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36747 2024-09-17T22:42:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:42:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:42:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735102+0000", 2024-09-17T22:42:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:42:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735104+0000", 2024-09-17T22:42:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7400000000000001e-06 2024-09-17T22:42:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735111+0000", 2024-09-17T22:42:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4189999999999996e-06 2024-09-17T22:42:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735116+0000", 2024-09-17T22:42:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0860000000000001e-06 2024-09-17T22:42:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741847+0000", 2024-09-17T22:42:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067306859999999996 2024-09-17T22:42:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741942+0000", 2024-09-17T22:42:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5180999999999996e-05 2024-09-17T22:42:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742048+0000", 2024-09-17T22:42:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106387 2024-09-17T22:42:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36748 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:42:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.951352851999999, 2024-09-17T22:42:02.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071864119999999997, 2024-09-17T22:42:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36748 2024-09-17T22:42:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:42:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:42:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:42:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.34e-07 2024-09-17T22:42:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735125+0000", 2024-09-17T22:42:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0020000000000001e-06 2024-09-17T22:42:02.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735132+0000", 2024-09-17T22:42:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3489999999999997e-06 2024-09-17T22:42:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735138+0000", 2024-09-17T22:42:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9709999999999998e-06 2024-09-17T22:42:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742087+0000", 2024-09-17T22:42:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069480669999999996 2024-09-17T22:42:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742188+0000", 2024-09-17T22:42:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101279 2024-09-17T22:42:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742309+0000", 2024-09-17T22:42:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012131 2024-09-17T22:42:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36749 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:42:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.951330165000002, 2024-09-17T22:42:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0074112570000000001, 2024-09-17T22:42:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36749 2024-09-17T22:42:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:42:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:42:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735146+0000", 2024-09-17T22:42:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9200000000000002e-07 2024-09-17T22:42:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735158+0000", 2024-09-17T22:42:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2508e-05 2024-09-17T22:42:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735166+0000", 2024-09-17T22:42:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6290000000000001e-06 2024-09-17T22:42:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735174+0000", 2024-09-17T22:42:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1000000000000004e-06 2024-09-17T22:42:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742354+0000", 2024-09-17T22:42:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071795660000000001 2024-09-17T22:42:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742454+0000", 2024-09-17T22:42:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100408 2024-09-17T22:42:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742557+0000", 2024-09-17T22:42:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102654 2024-09-17T22:42:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36750 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:42:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.951293038999999, 2024-09-17T22:42:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0077413890000000004, 2024-09-17T22:42:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.723 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36750 2024-09-17T22:42:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:42:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:42:02.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:42:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2399999999999998e-07 2024-09-17T22:42:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735185+0000", 2024-09-17T22:42:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1710000000000001e-06 2024-09-17T22:42:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735192+0000", 2024-09-17T22:42:02.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.81e-06 2024-09-17T22:42:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735197+0000", 2024-09-17T22:42:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9080000000000003e-06 2024-09-17T22:42:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742587+0000", 2024-09-17T22:42:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.007390359 2024-09-17T22:42:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742694+0000", 2024-09-17T22:42:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106901 2024-09-17T22:42:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742924+0000", 2024-09-17T22:42:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022961600000000001 2024-09-17T22:42:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36751 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:42:02.727 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.951271065, 2024-09-17T22:42:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079273859999999998, 2024-09-17T22:42:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36751 2024-09-17T22:42:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:42:02.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:42:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:42:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1600000000000001e-07 2024-09-17T22:42:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735207+0000", 2024-09-17T22:42:02.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1909999999999999e-06 2024-09-17T22:42:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735214+0000", 2024-09-17T22:42:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9269999999999999e-06 2024-09-17T22:42:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735220+0000", 2024-09-17T22:42:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3739999999999997e-06 2024-09-17T22:42:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742957+0000", 2024-09-17T22:42:02.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0077371699999999998 2024-09-17T22:42:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.743037+0000", 2024-09-17T22:42:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0370000000000005e-05 2024-09-17T22:42:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.743132+0000", 2024-09-17T22:42:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4838000000000002e-05 2024-09-17T22:42:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.731 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37659 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:42:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.936703422000001, 2024-09-17T22:42:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063144200000000003, 2024-09-17T22:42:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:02.732 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37659 2024-09-17T22:42:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:42:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:42:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749773+0000", 2024-09-17T22:42:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:42:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749774+0000", 2024-09-17T22:42:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:42:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749781+0000", 2024-09-17T22:42:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5740000000000001e-06 2024-09-17T22:42:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749786+0000", 2024-09-17T22:42:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9099999999999996e-06 2024-09-17T22:42:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755889+0000", 2024-09-17T22:42:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061034260000000003 2024-09-17T22:42:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755972+0000", 2024-09-17T22:42:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2795999999999998e-05 2024-09-17T22:42:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756087+0000", 2024-09-17T22:42:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000114418 2024-09-17T22:42:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37660 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:42:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.936673583999998, 2024-09-17T22:42:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065587800000000002, 2024-09-17T22:42:02.736 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37660 2024-09-17T22:42:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:42:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:42:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:42:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:42:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749805+0000", 2024-09-17T22:42:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-17T22:42:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749812+0000", 2024-09-17T22:42:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270000000000002e-06 2024-09-17T22:42:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749819+0000", 2024-09-17T22:42:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.759e-06 2024-09-17T22:42:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756153+0000", 2024-09-17T22:42:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063345809999999997 2024-09-17T22:42:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756257+0000", 2024-09-17T22:42:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103583 2024-09-17T22:42:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756361+0000", 2024-09-17T22:42:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103853 2024-09-17T22:42:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.740 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37661 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:42:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.936644989999998, 2024-09-17T22:42:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067442300000000004, 2024-09-17T22:42:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37661 2024-09-17T22:42:02.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:42:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:42:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:42:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9399999999999995e-07 2024-09-17T22:42:02.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749835+0000", 2024-09-17T22:42:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1049999999999997e-06 2024-09-17T22:42:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749855+0000", 2024-09-17T22:42:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9343000000000001e-05 2024-09-17T22:42:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749860+0000", 2024-09-17T22:42:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:42:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756395+0000", 2024-09-17T22:42:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065349900000000001 2024-09-17T22:42:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756479+0000", 2024-09-17T22:42:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4030999999999996e-05 2024-09-17T22:42:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756575+0000", 2024-09-17T22:42:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5835999999999995e-05 2024-09-17T22:42:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37662 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:42:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.936591184000001, 2024-09-17T22:42:02.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006927295, 2024-09-17T22:42:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37662 2024-09-17T22:42:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:42:02.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:42:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749885+0000", 2024-09-17T22:42:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2699999999999999e-07 2024-09-17T22:42:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749888+0000", 2024-09-17T22:42:02.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6599999999999999e-06 2024-09-17T22:42:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749895+0000", 2024-09-17T22:42:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4200000000000001e-06 2024-09-17T22:42:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749901+0000", 2024-09-17T22:42:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5090000000000001e-06 2024-09-17T22:42:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756605+0000", 2024-09-17T22:42:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067040629999999997 2024-09-17T22:42:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756691+0000", 2024-09-17T22:42:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5846000000000004e-05 2024-09-17T22:42:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756812+0000", 2024-09-17T22:42:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012116999999999999 2024-09-17T22:42:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.749 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37663 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:42:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.936566974000002, 2024-09-17T22:42:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071811440000000004, 2024-09-17T22:42:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37663 2024-09-17T22:42:02.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:42:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:42:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:42:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:42:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749911+0000", 2024-09-17T22:42:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.035e-06 2024-09-17T22:42:02.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749917+0000", 2024-09-17T22:42:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4370000000000004e-06 2024-09-17T22:42:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749922+0000", 2024-09-17T22:42:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6249999999999998e-06 2024-09-17T22:42:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756845+0000", 2024-09-17T22:42:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069227280000000004 2024-09-17T22:42:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756994+0000", 2024-09-17T22:42:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000149068 2024-09-17T22:42:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757090+0000", 2024-09-17T22:42:02.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5851000000000001e-05 2024-09-17T22:42:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37664 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:42:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.936546129, 2024-09-17T22:42:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.007384743, 2024-09-17T22:42:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:02.755 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37664 2024-09-17T22:42:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:42:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:42:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749930+0000", 2024-09-17T22:42:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:42:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749933+0000", 2024-09-17T22:42:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8430000000000001e-06 2024-09-17T22:42:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749939+0000", 2024-09-17T22:42:02.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:42:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749944+0000", 2024-09-17T22:42:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9230000000000001e-06 2024-09-17T22:42:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757124+0000", 2024-09-17T22:42:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071800020000000004 2024-09-17T22:42:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757212+0000", 2024-09-17T22:42:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7383e-05 2024-09-17T22:42:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757314+0000", 2024-09-17T22:42:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010247399999999999 2024-09-17T22:42:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:39363 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:42:02.759 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.89455779, 2024-09-17T22:42:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006226245, 2024-09-17T22:42:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 39363 2024-09-17T22:42:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:42:02.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:42:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:42:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3500000000000002e-07 2024-09-17T22:42:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791920+0000", 2024-09-17T22:42:02.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.869e-06 2024-09-17T22:42:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791926+0000", 2024-09-17T22:42:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8490000000000002e-06 2024-09-17T22:42:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791931+0000", 2024-09-17T22:42:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4269999999999999e-06 2024-09-17T22:42:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.797912+0000", 2024-09-17T22:42:02.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059808819999999999 2024-09-17T22:42:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.798020+0000", 2024-09-17T22:42:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010783800000000001 2024-09-17T22:42:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.763 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.798144+0000", 2024-09-17T22:42:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012404500000000001 2024-09-17T22:42:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4447.0:51869 3.6 e35/23 3:6037e35f:::rbd_object_map.115fe9ac8535:head v 35'71, mlcod=35'66)", 2024-09-17T22:42:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:42:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.90923549, 2024-09-17T22:42:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0066208669999999999, 2024-09-17T22:42:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:42:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:42:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777241+0000", 2024-09-17T22:42:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4000000000000001e-07 2024-09-17T22:42:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777246+0000", 2024-09-17T22:42:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2789999999999998e-06 2024-09-17T22:42:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777255+0000", 2024-09-17T22:42:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6209999999999993e-06 2024-09-17T22:42:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777259+0000", 2024-09-17T22:42:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7650000000000001e-06 2024-09-17T22:42:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777298+0000", 2024-09-17T22:42:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8797999999999997e-05 2024-09-17T22:42:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777311+0000", 2024-09-17T22:42:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2364999999999999e-05 2024-09-17T22:42:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.783814+0000", 2024-09-17T22:42:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065039169999999997 2024-09-17T22:42:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.783861+0000", 2024-09-17T22:42:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6482e-05 2024-09-17T22:42:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.768 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:105336 3.b 3:d3549b8d:::rbd_data.115ff9e8d63c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:42:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:43.787529+0000", 2024-09-17T22:42:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 18.898946185, 2024-09-17T22:42:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0099864259999999996, 2024-09-17T22:42:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:02.769 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 105336 2024-09-17T22:42:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.787529+0000", 2024-09-17T22:42:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.787529+0000", 2024-09-17T22:42:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.787531+0000", 2024-09-17T22:42:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9759999999999998e-06 2024-09-17T22:42:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788120+0000", 2024-09-17T22:42:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00058899799999999997 2024-09-17T22:42:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788138+0000", 2024-09-17T22:42:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7393000000000001e-05 2024-09-17T22:42:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788146+0000", 2024-09-17T22:42:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8220000000000007e-06 2024-09-17T22:42:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788199+0000", 2024-09-17T22:42:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3718000000000003e-05 2024-09-17T22:42:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788316+0000", 2024-09-17T22:42:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000116999 2024-09-17T22:42:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:42:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.791608+0000", 2024-09-17T22:42:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003291188 2024-09-17T22:42:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797379+0000", 2024-09-17T22:42:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057716829999999997 2024-09-17T22:42:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:42:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797401+0000", 2024-09-17T22:42:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1729000000000002e-05 2024-09-17T22:42:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797459+0000", 2024-09-17T22:42:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8115000000000002e-05 2024-09-17T22:42:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797516+0000", 2024-09-17T22:42:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6805000000000002e-05 2024-09-17T22:42:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.774 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4447.0:177615 3.0 e57/23 3:036f4184:::rbd_data.115f31bbb4b1.000000000000001c:head v 57'151, mlcod=55'148)", 2024-09-17T22:42:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:59.581018+0000", 2024-09-17T22:42:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.1054577820000002, 2024-09-17T22:42:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067345199999999999, 2024-09-17T22:42:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581018+0000", 2024-09-17T22:42:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581018+0000", 2024-09-17T22:42:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581018+0000", 2024-09-17T22:42:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4499999999999997e-07 2024-09-17T22:42:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581024+0000", 2024-09-17T22:42:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-17T22:42:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581036+0000", 2024-09-17T22:42:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1287999999999999e-05 2024-09-17T22:42:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581044+0000", 2024-09-17T22:42:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8220000000000007e-06 2024-09-17T22:42:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581094+0000", 2024-09-17T22:42:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0475e-05 2024-09-17T22:42:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581112+0000", 2024-09-17T22:42:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7932e-05 2024-09-17T22:42:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.587716+0000", 2024-09-17T22:42:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0066042879999999998 2024-09-17T22:42:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.587752+0000", 2024-09-17T22:42:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6106999999999998e-05 2024-09-17T22:42:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:178091 3.b 3:d1597d64:::rbd_data.115f31bbb4b1.000000000000006b:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T22:42:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:00.078810+0000", 2024-09-17T22:42:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 2.6076656759999999, 2024-09-17T22:42:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017828758, 2024-09-17T22:42:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 178091 2024-09-17T22:42:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078810+0000", 2024-09-17T22:42:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078810+0000", 2024-09-17T22:42:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078811+0000", 2024-09-17T22:42:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.176e-06 2024-09-17T22:42:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078817+0000", 2024-09-17T22:42:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6540000000000004e-06 2024-09-17T22:42:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078828+0000", 2024-09-17T22:42:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0875e-05 2024-09-17T22:42:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078833+0000", 2024-09-17T22:42:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6099999999999997e-06 2024-09-17T22:42:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078884+0000", 2024-09-17T22:42:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1013000000000001e-05 2024-09-17T22:42:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078933+0000", 2024-09-17T22:42:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8380000000000001e-05 2024-09-17T22:42:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:42:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.081828+0000", 2024-09-17T22:42:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0028952219999999998 2024-09-17T22:42:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.096563+0000", 2024-09-17T22:42:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.014734860000000001 2024-09-17T22:42:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:42:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.096575+0000", 2024-09-17T22:42:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2126e-05 2024-09-17T22:42:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.096608+0000", 2024-09-17T22:42:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2836000000000002e-05 2024-09-17T22:42:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.096639+0000", 2024-09-17T22:42:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1006000000000001e-05 2024-09-17T22:42:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:02.786 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:02.788 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:42:02.789 INFO:teuthology.orchestra.run.smithi115.stdout:2130 trunc from 0xd764768 to 0x6b89c5 2024-09-17T22:42:02.789 INFO:teuthology.orchestra.run.smithi115.stdout:2131 read 0x478508 thru 0x484e43 (0xc93c bytes) 2024-09-17T22:42:02.789 INFO:teuthology.orchestra.run.smithi115.stdout:2132 trunc from 0x6b89c5 to 0xf9360a 2024-09-17T22:42:02.790 INFO:teuthology.orchestra.run.smithi115.stdout:2134 writesame 0x75f7e42 thru 0x76019a3 data_size 0x1(0x9b62 bytes) 2024-09-17T22:42:02.790 INFO:teuthology.orchestra.run.smithi115.stdout:2135 trunc from 0x76019a4 to 0xb9049d6 2024-09-17T22:42:02.790 INFO:teuthology.orchestra.run.smithi115.stdout:2136 read 0x8f8b9a4 thru 0x8f8d7a6 (0x1e03 bytes) 2024-09-17T22:42:02.790 INFO:teuthology.orchestra.run.smithi115.stdout:2137 writesame 0xac351eb thru 0xac3b5ba data_size 0x8(0x63d0 bytes) 2024-09-17T22:42:02.790 INFO:teuthology.orchestra.run.smithi115.stdout:2138 writesame 0x92a46ac thru 0x92a8ac1 data_size 0x1e(0x4416 bytes) 2024-09-17T22:42:02.790 INFO:teuthology.orchestra.run.smithi115.stdout:2139 read 0xad2f340 thru 0xad31539 (0x21fa bytes) 2024-09-17T22:42:02.790 INFO:teuthology.orchestra.run.smithi115.stdout:2140 punch from 0x2cb3b77 to 0x2cb6bfa, (0x3083 bytes) 2024-09-17T22:42:02.790 INFO:teuthology.orchestra.run.smithi115.stdout:2141 punch from 0x6354b2b to 0x635782a, (0x2cff bytes) 2024-09-17T22:42:02.790 INFO:teuthology.orchestra.run.smithi115.stdout:2142 trunc from 0xb9049d6 to 0x70beb73 2024-09-17T22:42:02.790 INFO:teuthology.orchestra.run.smithi115.stdout:2143 read 0x4ac36e1 thru 0x4ad23c5 (0xece5 bytes) 2024-09-17T22:42:02.790 INFO:teuthology.orchestra.run.smithi115.stdout:2144 write 0x6c92eb thru 0x6d5e3e (0xcb54 bytes) 2024-09-17T22:42:02.790 INFO:teuthology.orchestra.run.smithi115.stdout:2146 write 0x839de89 thru 0x839e69f (0x817 bytes) 2024-09-17T22:42:02.790 INFO:teuthology.orchestra.run.smithi115.stdout:2147 trunc from 0x839e6a0 to 0x71f0b0a 2024-09-17T22:42:02.791 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:02.785+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:02.791 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:02.799 INFO:teuthology.orchestra.run.smithi115.stdout:2152 write 0xc86ba2d thru 0xc871370 (0x5944 bytes) 2024-09-17T22:42:02.807 INFO:teuthology.orchestra.run.smithi115.stdout:2154 trunc from 0xc871371 to 0x945818b 2024-09-17T22:42:02.816 INFO:teuthology.orchestra.run.smithi115.stdout:2155 read 0x4f69b32 thru 0x4f6d56c (0x3a3b bytes) 2024-09-17T22:42:02.816 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:02.817 INFO:teuthology.orchestra.run.smithi115.stdout:2160 read 0x3e9f7df thru 0x3ea9217 (0x9a39 bytes) 2024-09-17T22:42:02.817 INFO:teuthology.orchestra.run.smithi115.stdout:2162 read 0x74ac99e thru 0x74b8d27 (0xc38a bytes) 2024-09-17T22:42:02.817 INFO:teuthology.orchestra.run.smithi115.stdout:2163 read 0x33bb433 thru 0x33c78da (0xc4a8 bytes) 2024-09-17T22:42:02.822 INFO:teuthology.orchestra.run.smithi115.stdout:2164 trunc from 0x945818b to 0xc46b047 2024-09-17T22:42:02.827 INFO:teuthology.orchestra.run.smithi115.stdout:2165 read 0x544194d thru 0x544b56c (0x9c20 bytes) 2024-09-17T22:42:02.827 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:02.835 INFO:teuthology.orchestra.run.smithi115.stdout:2168 trunc from 0xc46b047 to 0x5928ac8 2024-09-17T22:42:02.841 INFO:teuthology.orchestra.run.smithi115.stdout:2169 trunc from 0x5928ac8 to 0x4c93e3d 2024-09-17T22:42:02.849 INFO:teuthology.orchestra.run.smithi115.stdout:2170 punch from 0x1e66910 to 0x1e6c878, (0x5f68 bytes) 2024-09-17T22:42:02.849 INFO:teuthology.orchestra.run.smithi115.stdout:2171 read 0x2595940 thru 0x25a2b54 (0xd215 bytes) 2024-09-17T22:42:02.852 INFO:teuthology.orchestra.run.smithi115.stdout:2172 trunc from 0x4c93e3d to 0x99b3534 2024-09-17T22:42:02.857 INFO:teuthology.orchestra.run.smithi115.stdout:2173 write 0x1c787e4 thru 0x1c864dd (0xdcfa bytes) 2024-09-17T22:42:02.859 INFO:teuthology.orchestra.run.smithi115.stdout:2174 write 0xef8ad7 thru 0xefa135 (0x165f bytes) 2024-09-17T22:42:02.872 INFO:teuthology.orchestra.run.smithi115.stdout:2176 write 0xea2e1ef thru 0xea363b7 (0x81c9 bytes) 2024-09-17T22:42:02.881 INFO:teuthology.orchestra.run.smithi115.stdout:2177 trunc from 0xea363b8 to 0x93152cb 2024-09-17T22:42:02.888 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:02.886+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:02.891 INFO:teuthology.orchestra.run.smithi115.stdout:2179 writesame 0x3e9868d thru 0x3e99322 data_size 0x9(0xc96 bytes) 2024-09-17T22:42:02.892 INFO:teuthology.orchestra.run.smithi115.stdout:2180 read 0x20f3d71 thru 0x20fa308 (0x6598 bytes) 2024-09-17T22:42:02.893 INFO:teuthology.orchestra.run.smithi115.stdout:2182 read 0x3579d92 thru 0x357e4cb (0x473a bytes) 2024-09-17T22:42:02.893 INFO:teuthology.orchestra.run.smithi115.stdout:2183 write 0x3640255 thru 0x3642c99 (0x2a45 bytes) 2024-09-17T22:42:02.894 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:02.894 INFO:teuthology.orchestra.run.smithi115.stdout:2186 punch from 0x17f53eb to 0x17fc859, (0x746e bytes) 2024-09-17T22:42:02.906 INFO:teuthology.orchestra.run.smithi115.stdout:2187 write 0xe20f6c2 thru 0xe2149f7 (0x5336 bytes) 2024-09-17T22:42:02.907 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:02.907 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:02.907 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:02.907 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:02.909 INFO:teuthology.orchestra.run.smithi115.stdout:2189 read 0x7697721 thru 0x769d9fe (0x62de bytes) 2024-09-17T22:42:02.909 INFO:teuthology.orchestra.run.smithi115.stdout:2190 writesame 0x9af6727 thru 0x9b02b7a data_size 0x23(0xc454 bytes) 2024-09-17T22:42:02.913 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:02.913 INFO:teuthology.orchestra.run.smithi115.stdout:2194 write 0x7e4253d thru 0x7e451e2 (0x2ca6 bytes) 2024-09-17T22:42:02.915 INFO:teuthology.orchestra.run.smithi115.stdout:2195 punch from 0xd75388d to 0xd75d327, (0x9a9a bytes) 2024-09-17T22:42:02.916 INFO:teuthology.orchestra.run.smithi115.stdout:2196 punch from 0xc7e938 to 0xc8ac1d, (0xc2e5 bytes) 2024-09-17T22:42:02.916 INFO:teuthology.orchestra.run.smithi115.stdout:2197 write 0x17576b0 thru 0x1758d73 (0x16c4 bytes) 2024-09-17T22:42:02.916 INFO:teuthology.orchestra.run.smithi115.stdout:2199 punch from 0x91d7ac1 to 0x91e043a, (0x8979 bytes) 2024-09-17T22:42:02.916 INFO:teuthology.orchestra.run.smithi115.stdout:2200 write 0xc8d78d1 thru 0xc8e02f4 (0x8a24 bytes) 2024-09-17T22:42:02.918 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:42:02.919 INFO:teuthology.orchestra.run.smithi115.stdout:2201 write 0x6259ba7 thru 0x625a158 (0x5b2 bytes) 2024-09-17T22:42:02.921 INFO:teuthology.orchestra.run.smithi115.stdout:2202 read 0xa13a18c thru 0xa148508 (0xe37d bytes) 2024-09-17T22:42:02.921 INFO:teuthology.orchestra.run.smithi115.stdout:2203 read 0xf46aa0 thru 0xf4d0d8 (0x6639 bytes) 2024-09-17T22:42:02.921 INFO:teuthology.orchestra.run.smithi115.stdout:2204 writesame 0x85f2277 thru 0x8601889 data_size 0x2b(0xf613 bytes) 2024-09-17T22:42:02.923 INFO:teuthology.orchestra.run.smithi115.stdout:2205 clone 20 order 25 su 2097152 sc 13 2024-09-17T22:42:02.989 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:02.987+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:03.021 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:03.021 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:03.022 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:03.022 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:03.022 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:03.030 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:42:03.114 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:03.113+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:03.167 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:42:03.183 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:03.183 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:03.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:42:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.821267009000003, 2024-09-17T22:42:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.249158453, 2024-09-17T22:42:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:42:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:42:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-17T22:42:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358680+0000", 2024-09-17T22:42:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.592e-06 2024-09-17T22:42:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358688+0000", 2024-09-17T22:42:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1610000000000003e-06 2024-09-17T22:42:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358697+0000", 2024-09-17T22:42:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2580000000000001e-06 2024-09-17T22:42:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358706+0000", 2024-09-17T22:42:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1640000000000007e-06 2024-09-17T22:42:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358766+0000", 2024-09-17T22:42:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9893999999999999e-05 2024-09-17T22:42:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:42:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358778+0000", 2024-09-17T22:42:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1148999999999999e-05 2024-09-17T22:42:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606347+0000", 2024-09-17T22:42:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.247569971 2024-09-17T22:42:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606533+0000", 2024-09-17T22:42:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018599400000000001 2024-09-17T22:42:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:42:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606752+0000", 2024-09-17T22:42:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000218373 2024-09-17T22:42:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:42:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607455+0000", 2024-09-17T22:42:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070311799999999997 2024-09-17T22:42:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607735+0000", 2024-09-17T22:42:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027972300000000001 2024-09-17T22:42:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:42:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607793+0000", 2024-09-17T22:42:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8529999999999997e-05 2024-09-17T22:42:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607837+0000", 2024-09-17T22:42:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3526000000000003e-05 2024-09-17T22:42:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42346 3.1c 3:3b21ac19:::rbd_data.115fe0c30ef6.000000000000003f:5 [read 2048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.983654+0000", 2024-09-17T22:42:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.196291164999998, 2024-09-17T22:42:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024578903999999999, 2024-09-17T22:42:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42346 2024-09-17T22:42:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983654+0000", 2024-09-17T22:42:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983654+0000", 2024-09-17T22:42:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983654+0000", 2024-09-17T22:42:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:42:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983657+0000", 2024-09-17T22:42:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-09-17T22:42:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983662+0000", 2024-09-17T22:42:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9170000000000003e-06 2024-09-17T22:42:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983667+0000", 2024-09-17T22:42:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6310000000000002e-06 2024-09-17T22:42:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.987089+0000", 2024-09-17T22:42:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0034221709999999999 2024-09-17T22:42:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op must wait for map", 2024-09-17T22:42:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.987094+0000", 2024-09-17T22:42:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0189999999999997e-06 2024-09-17T22:42:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008083+0000", 2024-09-17T22:42:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020989102999999999 2024-09-17T22:42:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008153+0000", 2024-09-17T22:42:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9431000000000007e-05 2024-09-17T22:42:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008233+0000", 2024-09-17T22:42:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0275e-05 2024-09-17T22:42:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42347 3.1c 3:3b21ac19:::rbd_data.115fe0c30ef6.000000000000003f:5 [read 10240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.983673+0000", 2024-09-17T22:42:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.196272444000002, 2024-09-17T22:42:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024700438000000002, 2024-09-17T22:42:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42347 2024-09-17T22:42:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983673+0000", 2024-09-17T22:42:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983673+0000", 2024-09-17T22:42:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983673+0000", 2024-09-17T22:42:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8000000000000002e-07 2024-09-17T22:42:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983675+0000", 2024-09-17T22:42:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9259999999999999e-06 2024-09-17T22:42:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983681+0000", 2024-09-17T22:42:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.241e-06 2024-09-17T22:42:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983687+0000", 2024-09-17T22:42:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.507e-06 2024-09-17T22:42:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.987136+0000", 2024-09-17T22:42:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003448806 2024-09-17T22:42:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.987140+0000", 2024-09-17T22:42:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6859999999999997e-06 2024-09-17T22:42:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008256+0000", 2024-09-17T22:42:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021115465999999999 2024-09-17T22:42:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008310+0000", 2024-09-17T22:42:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4128000000000001e-05 2024-09-17T22:42:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008373+0000", 2024-09-17T22:42:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3398000000000005e-05 2024-09-17T22:42:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42348 3.1c 3:3b21ac19:::rbd_data.115fe0c30ef6.000000000000003f:5 [read 16384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.983693+0000", 2024-09-17T22:42:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.196252665999999, 2024-09-17T22:42:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02481359, 2024-09-17T22:42:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42348 2024-09-17T22:42:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983693+0000", 2024-09-17T22:42:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983693+0000", 2024-09-17T22:42:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983693+0000", 2024-09-17T22:42:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7800000000000002e-07 2024-09-17T22:42:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983695+0000", 2024-09-17T22:42:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.069e-06 2024-09-17T22:42:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983701+0000", 2024-09-17T22:42:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1970000000000001e-06 2024-09-17T22:42:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983706+0000", 2024-09-17T22:42:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-09-17T22:42:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.987166+0000", 2024-09-17T22:42:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0034596430000000001 2024-09-17T22:42:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.987170+0000", 2024-09-17T22:42:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2919999999999997e-06 2024-09-17T22:42:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008391+0000", 2024-09-17T22:42:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021221259999999999 2024-09-17T22:42:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008444+0000", 2024-09-17T22:42:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2880999999999997e-05 2024-09-17T22:42:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008506+0000", 2024-09-17T22:42:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2219999999999997e-05 2024-09-17T22:42:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42349 3.1c 3:3b21ac19:::rbd_data.115fe0c30ef6.000000000000003f:5 [read 24576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.983712+0000", 2024-09-17T22:42:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.196233460000002, 2024-09-17T22:42:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024925591, 2024-09-17T22:42:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42349 2024-09-17T22:42:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983712+0000", 2024-09-17T22:42:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983712+0000", 2024-09-17T22:42:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983712+0000", 2024-09-17T22:42:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6199999999999999e-07 2024-09-17T22:42:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983714+0000", 2024-09-17T22:42:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9690000000000001e-06 2024-09-17T22:42:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983720+0000", 2024-09-17T22:42:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8590000000000001e-06 2024-09-17T22:42:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.983724+0000", 2024-09-17T22:42:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4179999999999999e-06 2024-09-17T22:42:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.987463+0000", 2024-09-17T22:42:03.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037386310000000001 2024-09-17T22:42:03.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:03.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.987469+0000", 2024-09-17T22:42:03.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.507e-06 2024-09-17T22:42:03.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008524+0000", 2024-09-17T22:42:03.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021055417 2024-09-17T22:42:03.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008576+0000", 2024-09-17T22:42:03.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1700000000000003e-05 2024-09-17T22:42:03.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.008637+0000", 2024-09-17T22:42:03.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1728000000000003e-05 2024-09-17T22:42:03.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.213 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.214 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42735 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 141312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:03.214 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.992876+0000", 2024-09-17T22:42:03.214 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.187069545, 2024-09-17T22:42:03.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020079927000000001, 2024-09-17T22:42:03.214 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.214 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.214 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.214 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42735 2024-09-17T22:42:03.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.214 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992876+0000", 2024-09-17T22:42:03.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992876+0000", 2024-09-17T22:42:03.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992876+0000", 2024-09-17T22:42:03.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9999999999999999e-07 2024-09-17T22:42:03.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992878+0000", 2024-09-17T22:42:03.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6279999999999999e-06 2024-09-17T22:42:03.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992883+0000", 2024-09-17T22:42:03.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3909999999999997e-06 2024-09-17T22:42:03.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992888+0000", 2024-09-17T22:42:03.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4299999999999999e-06 2024-09-17T22:42:03.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994859+0000", 2024-09-17T22:42:03.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001971539 2024-09-17T22:42:03.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:03.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994863+0000", 2024-09-17T22:42:03.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.109e-06 2024-09-17T22:42:03.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.012806+0000", 2024-09-17T22:42:03.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.017942909999999999 2024-09-17T22:42:03.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.012863+0000", 2024-09-17T22:42:03.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6818999999999999e-05 2024-09-17T22:42:03.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.012956+0000", 2024-09-17T22:42:03.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2800999999999995e-05 2024-09-17T22:42:03.219 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.219 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.219 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.220 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42736 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:03.220 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.992893+0000", 2024-09-17T22:42:03.220 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.187052524000002, 2024-09-17T22:42:03.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020211743000000001, 2024-09-17T22:42:03.220 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.220 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.221 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.221 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42736 2024-09-17T22:42:03.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.221 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992893+0000", 2024-09-17T22:42:03.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992893+0000", 2024-09-17T22:42:03.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992893+0000", 2024-09-17T22:42:03.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0699999999999998e-07 2024-09-17T22:42:03.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992895+0000", 2024-09-17T22:42:03.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.64e-06 2024-09-17T22:42:03.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992900+0000", 2024-09-17T22:42:03.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4090000000000002e-06 2024-09-17T22:42:03.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992905+0000", 2024-09-17T22:42:03.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6789999999999996e-06 2024-09-17T22:42:03.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994885+0000", 2024-09-17T22:42:03.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019798300000000001 2024-09-17T22:42:03.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:03.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994889+0000", 2024-09-17T22:42:03.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1960000000000001e-06 2024-09-17T22:42:03.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.012977+0000", 2024-09-17T22:42:03.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018088160999999998 2024-09-17T22:42:03.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013029+0000", 2024-09-17T22:42:03.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2315999999999999e-05 2024-09-17T22:42:03.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013105+0000", 2024-09-17T22:42:03.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5204999999999996e-05 2024-09-17T22:42:03.226 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.226 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.226 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.226 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42737 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 163840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:03.226 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.992910+0000", 2024-09-17T22:42:03.226 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.187035457999997, 2024-09-17T22:42:03.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020362623, 2024-09-17T22:42:03.226 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.226 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.226 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.227 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.227 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.227 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42737 2024-09-17T22:42:03.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.227 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992910+0000", 2024-09-17T22:42:03.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992910+0000", 2024-09-17T22:42:03.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992910+0000", 2024-09-17T22:42:03.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0800000000000001e-07 2024-09-17T22:42:03.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992912+0000", 2024-09-17T22:42:03.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.784e-06 2024-09-17T22:42:03.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992917+0000", 2024-09-17T22:42:03.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0520000000000001e-06 2024-09-17T22:42:03.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992921+0000", 2024-09-17T22:42:03.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2540000000000003e-06 2024-09-17T22:42:03.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994909+0000", 2024-09-17T22:42:03.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001987529 2024-09-17T22:42:03.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:03.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994913+0000", 2024-09-17T22:42:03.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1200000000000004e-06 2024-09-17T22:42:03.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013127+0000", 2024-09-17T22:42:03.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018214533000000002 2024-09-17T22:42:03.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013192+0000", 2024-09-17T22:42:03.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4979000000000002e-05 2024-09-17T22:42:03.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013272+0000", 2024-09-17T22:42:03.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0063999999999998e-05 2024-09-17T22:42:03.232 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.232 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.232 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.232 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42738 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:03.232 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.992927+0000", 2024-09-17T22:42:03.232 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.187018098000003, 2024-09-17T22:42:03.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020514726, 2024-09-17T22:42:03.232 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.232 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.233 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.233 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42738 2024-09-17T22:42:03.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.234 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992927+0000", 2024-09-17T22:42:03.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992927+0000", 2024-09-17T22:42:03.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992928+0000", 2024-09-17T22:42:03.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3000000000000001e-07 2024-09-17T22:42:03.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992932+0000", 2024-09-17T22:42:03.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:42:03.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992939+0000", 2024-09-17T22:42:03.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4239999999999998e-06 2024-09-17T22:42:03.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992944+0000", 2024-09-17T22:42:03.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9450000000000001e-06 2024-09-17T22:42:03.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994933+0000", 2024-09-17T22:42:03.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001989326 2024-09-17T22:42:03.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:03.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994937+0000", 2024-09-17T22:42:03.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.104e-06 2024-09-17T22:42:03.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013294+0000", 2024-09-17T22:42:03.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018356514000000001 2024-09-17T22:42:03.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013352+0000", 2024-09-17T22:42:03.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8136e-05 2024-09-17T22:42:03.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013442+0000", 2024-09-17T22:42:03.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0239000000000001e-05 2024-09-17T22:42:03.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.239 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.239 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42739 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:03.239 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.992949+0000", 2024-09-17T22:42:03.239 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.186995859, 2024-09-17T22:42:03.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020622234999999999, 2024-09-17T22:42:03.239 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.239 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.239 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.239 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.239 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.240 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42739 2024-09-17T22:42:03.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.240 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992949+0000", 2024-09-17T22:42:03.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992949+0000", 2024-09-17T22:42:03.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992950+0000", 2024-09-17T22:42:03.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.15e-07 2024-09-17T22:42:03.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992951+0000", 2024-09-17T22:42:03.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7e-06 2024-09-17T22:42:03.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992957+0000", 2024-09-17T22:42:03.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6890000000000001e-06 2024-09-17T22:42:03.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992962+0000", 2024-09-17T22:42:03.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5789999999999997e-06 2024-09-17T22:42:03.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994958+0000", 2024-09-17T22:42:03.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019959230000000001 2024-09-17T22:42:03.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:03.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994962+0000", 2024-09-17T22:42:03.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2490000000000003e-06 2024-09-17T22:42:03.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013462+0000", 2024-09-17T22:42:03.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018500072999999999 2024-09-17T22:42:03.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013512+0000", 2024-09-17T22:42:03.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0306999999999998e-05 2024-09-17T22:42:03.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013572+0000", 2024-09-17T22:42:03.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.94e-05 2024-09-17T22:42:03.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.244 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.244 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42740 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 186368~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:03.244 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.992967+0000", 2024-09-17T22:42:03.244 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.186978046, 2024-09-17T22:42:03.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020729171000000001, 2024-09-17T22:42:03.245 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.245 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.245 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.245 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42740 2024-09-17T22:42:03.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.245 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992967+0000", 2024-09-17T22:42:03.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992967+0000", 2024-09-17T22:42:03.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992968+0000", 2024-09-17T22:42:03.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.41e-07 2024-09-17T22:42:03.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992969+0000", 2024-09-17T22:42:03.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.734e-06 2024-09-17T22:42:03.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992975+0000", 2024-09-17T22:42:03.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1909999999999999e-06 2024-09-17T22:42:03.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992979+0000", 2024-09-17T22:42:03.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2490000000000003e-06 2024-09-17T22:42:03.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994982+0000", 2024-09-17T22:42:03.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020027249999999999 2024-09-17T22:42:03.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:03.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.994986+0000", 2024-09-17T22:42:03.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1400000000000002e-06 2024-09-17T22:42:03.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013591+0000", 2024-09-17T22:42:03.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018605486000000001 2024-09-17T22:42:03.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013642+0000", 2024-09-17T22:42:03.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0408999999999998e-05 2024-09-17T22:42:03.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013696+0000", 2024-09-17T22:42:03.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4895999999999998e-05 2024-09-17T22:42:03.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.250 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.250 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.250 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42741 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 192512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:03.250 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.992984+0000", 2024-09-17T22:42:03.250 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.186961117999999, 2024-09-17T22:42:03.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020833673, 2024-09-17T22:42:03.250 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.250 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.250 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.251 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.251 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42741 2024-09-17T22:42:03.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.251 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992984+0000", 2024-09-17T22:42:03.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992984+0000", 2024-09-17T22:42:03.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992984+0000", 2024-09-17T22:42:03.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9700000000000003e-07 2024-09-17T22:42:03.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992990+0000", 2024-09-17T22:42:03.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3940000000000004e-06 2024-09-17T22:42:03.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.992996+0000", 2024-09-17T22:42:03.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.455e-06 2024-09-17T22:42:03.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993002+0000", 2024-09-17T22:42:03.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4620000000000004e-06 2024-09-17T22:42:03.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995005+0000", 2024-09-17T22:42:03.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002003379 2024-09-17T22:42:03.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:03.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995009+0000", 2024-09-17T22:42:03.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0910000000000003e-06 2024-09-17T22:42:03.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013715+0000", 2024-09-17T22:42:03.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018705894000000001 2024-09-17T22:42:03.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013764+0000", 2024-09-17T22:42:03.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9199000000000001e-05 2024-09-17T22:42:03.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013818+0000", 2024-09-17T22:42:03.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3501999999999997e-05 2024-09-17T22:42:03.255 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.255 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.255 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.255 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42742 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:03.255 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.993008+0000", 2024-09-17T22:42:03.255 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.186937020000002, 2024-09-17T22:42:03.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.020932441999999999, 2024-09-17T22:42:03.256 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.256 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.256 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.256 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42742 2024-09-17T22:42:03.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.257 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993008+0000", 2024-09-17T22:42:03.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993008+0000", 2024-09-17T22:42:03.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993009+0000", 2024-09-17T22:42:03.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:42:03.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993011+0000", 2024-09-17T22:42:03.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.148e-06 2024-09-17T22:42:03.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993017+0000", 2024-09-17T22:42:03.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3210000000000001e-06 2024-09-17T22:42:03.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993022+0000", 2024-09-17T22:42:03.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8679999999999998e-06 2024-09-17T22:42:03.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995029+0000", 2024-09-17T22:42:03.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020069770000000001 2024-09-17T22:42:03.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:03.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995033+0000", 2024-09-17T22:42:03.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1960000000000001e-06 2024-09-17T22:42:03.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013837+0000", 2024-09-17T22:42:03.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018803469 2024-09-17T22:42:03.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013886+0000", 2024-09-17T22:42:03.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9639999999999999e-05 2024-09-17T22:42:03.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013941+0000", 2024-09-17T22:42:03.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000002e-05 2024-09-17T22:42:03.261 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.261 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.261 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.261 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42743 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:03.261 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.993028+0000", 2024-09-17T22:42:03.261 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.186917655999999, 2024-09-17T22:42:03.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021056887, 2024-09-17T22:42:03.262 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.262 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.262 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.262 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.262 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.262 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.262 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42743 2024-09-17T22:42:03.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.262 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993028+0000", 2024-09-17T22:42:03.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993028+0000", 2024-09-17T22:42:03.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993028+0000", 2024-09-17T22:42:03.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8700000000000001e-07 2024-09-17T22:42:03.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993030+0000", 2024-09-17T22:42:03.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8470000000000001e-06 2024-09-17T22:42:03.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993036+0000", 2024-09-17T22:42:03.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0619999999999997e-06 2024-09-17T22:42:03.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993041+0000", 2024-09-17T22:42:03.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9520000000000002e-06 2024-09-17T22:42:03.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995066+0000", 2024-09-17T22:42:03.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002025237 2024-09-17T22:42:03.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:03.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995070+0000", 2024-09-17T22:42:03.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.444e-06 2024-09-17T22:42:03.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.013960+0000", 2024-09-17T22:42:03.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018890674999999999 2024-09-17T22:42:03.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014010+0000", 2024-09-17T22:42:03.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0212e-05 2024-09-17T22:42:03.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014085+0000", 2024-09-17T22:42:03.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4071000000000003e-05 2024-09-17T22:42:03.266 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.266 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.266 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.267 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42744 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:03.267 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.993053+0000", 2024-09-17T22:42:03.267 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.186892268000001, 2024-09-17T22:42:03.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021200131000000001, 2024-09-17T22:42:03.267 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.267 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.267 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.267 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.267 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.267 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.267 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42744 2024-09-17T22:42:03.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.267 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993053+0000", 2024-09-17T22:42:03.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993053+0000", 2024-09-17T22:42:03.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993053+0000", 2024-09-17T22:42:03.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6899999999999998e-07 2024-09-17T22:42:03.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993056+0000", 2024-09-17T22:42:03.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2400000000000002e-06 2024-09-17T22:42:03.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993062+0000", 2024-09-17T22:42:03.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3389999999999998e-06 2024-09-17T22:42:03.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993067+0000", 2024-09-17T22:42:03.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9089999999999999e-06 2024-09-17T22:42:03.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995088+0000", 2024-09-17T22:42:03.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002021134 2024-09-17T22:42:03.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:03.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995092+0000", 2024-09-17T22:42:03.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5310000000000002e-06 2024-09-17T22:42:03.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014107+0000", 2024-09-17T22:42:03.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019015018000000002 2024-09-17T22:42:03.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014170+0000", 2024-09-17T22:42:03.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3156000000000003e-05 2024-09-17T22:42:03.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014253+0000", 2024-09-17T22:42:03.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3435000000000003e-05 2024-09-17T22:42:03.272 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.272 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.272 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.272 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42745 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 229376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:03.272 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.993073+0000", 2024-09-17T22:42:03.272 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.186872311999998, 2024-09-17T22:42:03.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02150991, 2024-09-17T22:42:03.272 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.272 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.273 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.273 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.273 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.273 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.273 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42745 2024-09-17T22:42:03.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.273 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993073+0000", 2024-09-17T22:42:03.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993073+0000", 2024-09-17T22:42:03.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993073+0000", 2024-09-17T22:42:03.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2099999999999998e-07 2024-09-17T22:42:03.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993075+0000", 2024-09-17T22:42:03.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0439999999999998e-06 2024-09-17T22:42:03.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993082+0000", 2024-09-17T22:42:03.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2210000000000002e-06 2024-09-17T22:42:03.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993087+0000", 2024-09-17T22:42:03.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9259999999999999e-06 2024-09-17T22:42:03.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995109+0000", 2024-09-17T22:42:03.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002022505 2024-09-17T22:42:03.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:03.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995112+0000", 2024-09-17T22:42:03.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3739999999999998e-06 2024-09-17T22:42:03.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014280+0000", 2024-09-17T22:42:03.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019167857 2024-09-17T22:42:03.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014349+0000", 2024-09-17T22:42:03.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9129999999999997e-05 2024-09-17T22:42:03.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014583+0000", 2024-09-17T22:42:03.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000233532 2024-09-17T22:42:03.277 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.277 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.277 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.277 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42746 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:03.277 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.993092+0000", 2024-09-17T22:42:03.277 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.186853098, 2024-09-17T22:42:03.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021651699, 2024-09-17T22:42:03.278 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.278 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.278 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.278 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42746 2024-09-17T22:42:03.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.278 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993092+0000", 2024-09-17T22:42:03.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993092+0000", 2024-09-17T22:42:03.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993093+0000", 2024-09-17T22:42:03.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:42:03.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993095+0000", 2024-09-17T22:42:03.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1119999999999999e-06 2024-09-17T22:42:03.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993101+0000", 2024-09-17T22:42:03.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9630000000000003e-06 2024-09-17T22:42:03.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993105+0000", 2024-09-17T22:42:03.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5079999999999999e-06 2024-09-17T22:42:03.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995130+0000", 2024-09-17T22:42:03.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020246349999999999 2024-09-17T22:42:03.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:03.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995133+0000", 2024-09-17T22:42:03.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3809999999999999e-06 2024-09-17T22:42:03.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014611+0000", 2024-09-17T22:42:03.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019477972999999999 2024-09-17T22:42:03.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014684+0000", 2024-09-17T22:42:03.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3025e-05 2024-09-17T22:42:03.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014744+0000", 2024-09-17T22:42:03.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9695999999999999e-05 2024-09-17T22:42:03.282 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.282 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.283 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.283 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42747 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:03.283 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.993111+0000", 2024-09-17T22:42:03.283 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.186834228999999, 2024-09-17T22:42:03.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021790963999999999, 2024-09-17T22:42:03.283 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.283 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.283 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.283 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42747 2024-09-17T22:42:03.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.284 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993111+0000", 2024-09-17T22:42:03.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993111+0000", 2024-09-17T22:42:03.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993111+0000", 2024-09-17T22:42:03.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-09-17T22:42:03.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993114+0000", 2024-09-17T22:42:03.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.046e-06 2024-09-17T22:42:03.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993119+0000", 2024-09-17T22:42:03.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9510000000000003e-06 2024-09-17T22:42:03.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993124+0000", 2024-09-17T22:42:03.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9019999999999998e-06 2024-09-17T22:42:03.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995151+0000", 2024-09-17T22:42:03.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020268460000000001 2024-09-17T22:42:03.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:03.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995155+0000", 2024-09-17T22:42:03.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7469999999999999e-06 2024-09-17T22:42:03.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014770+0000", 2024-09-17T22:42:03.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019614691 2024-09-17T22:42:03.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014844+0000", 2024-09-17T22:42:03.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-09-17T22:42:03.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014902+0000", 2024-09-17T22:42:03.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8057000000000003e-05 2024-09-17T22:42:03.288 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.288 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.288 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.289 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42748 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 251904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:03.289 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.993131+0000", 2024-09-17T22:42:03.289 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.186814396999999, 2024-09-17T22:42:03.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021930076, 2024-09-17T22:42:03.289 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.289 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.289 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.289 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42748 2024-09-17T22:42:03.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.290 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993131+0000", 2024-09-17T22:42:03.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993131+0000", 2024-09-17T22:42:03.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993131+0000", 2024-09-17T22:42:03.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-09-17T22:42:03.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993133+0000", 2024-09-17T22:42:03.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.069e-06 2024-09-17T22:42:03.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993139+0000", 2024-09-17T22:42:03.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0440000000000001e-06 2024-09-17T22:42:03.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993144+0000", 2024-09-17T22:42:03.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-09-17T22:42:03.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995175+0000", 2024-09-17T22:42:03.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020309199999999999 2024-09-17T22:42:03.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:03.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995179+0000", 2024-09-17T22:42:03.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.233e-06 2024-09-17T22:42:03.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014927+0000", 2024-09-17T22:42:03.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019747712000000001 2024-09-17T22:42:03.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.014996+0000", 2024-09-17T22:42:03.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9262000000000002e-05 2024-09-17T22:42:03.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.015061+0000", 2024-09-17T22:42:03.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4673999999999998e-05 2024-09-17T22:42:03.294 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.294 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.294 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.295 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:42749 3.1c 3:3f3f1072:::rbd_data.115fe0c30ef6.0000000000000030:5 [read 258048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-17T22:42:03.295 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.993150+0000", 2024-09-17T22:42:03.295 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.186795414000002, 2024-09-17T22:42:03.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022066032999999999, 2024-09-17T22:42:03.295 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.295 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.295 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.295 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.295 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.295 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.295 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 42749 2024-09-17T22:42:03.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.296 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993150+0000", 2024-09-17T22:42:03.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993150+0000", 2024-09-17T22:42:03.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993150+0000", 2024-09-17T22:42:03.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-09-17T22:42:03.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993152+0000", 2024-09-17T22:42:03.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8729999999999999e-06 2024-09-17T22:42:03.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993159+0000", 2024-09-17T22:42:03.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3740000000000003e-06 2024-09-17T22:42:03.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.993164+0000", 2024-09-17T22:42:03.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1030000000000001e-06 2024-09-17T22:42:03.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995199+0000", 2024-09-17T22:42:03.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020354190000000001 2024-09-17T22:42:03.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting_for_map not empty", 2024-09-17T22:42:03.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.995203+0000", 2024-09-17T22:42:03.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1459999999999998e-06 2024-09-17T22:42:03.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.015087+0000", 2024-09-17T22:42:03.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.019883483 2024-09-17T22:42:03.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.015156+0000", 2024-09-17T22:42:03.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9731e-05 2024-09-17T22:42:03.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:30.015216+0000", 2024-09-17T22:42:03.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9528000000000003e-05 2024-09-17T22:42:03.300 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.300 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.300 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.300 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.300 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.301 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:03.302 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:42:03.303 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone19 from 0xe2149f8 (overlap 0x6b89c5) to 0x2b03e8 2024-09-17T22:42:03.303 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:03.299+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:03.311 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-17T22:42:03.322 INFO:teuthology.orchestra.run.smithi115.stdout:2206 read 0xbf759c1 thru 0xbf85318 (0xf958 bytes) 2024-09-17T22:42:03.324 INFO:teuthology.orchestra.run.smithi115.stdout:2207 read 0xabd37e2 thru 0xabdf5ef (0xbe0e bytes) 2024-09-17T22:42:03.324 INFO:teuthology.orchestra.run.smithi115.stdout:2208 write 0x1ad46b thru 0x1b5a85 (0x861b bytes) 2024-09-17T22:42:03.336 INFO:teuthology.orchestra.run.smithi115.stdout:2209 trunc from 0xe2149f8 to 0xdc5dbca 2024-09-17T22:42:03.349 INFO:teuthology.orchestra.run.smithi115.stdout:2210 write 0x640fb37 thru 0x641b24e (0xb718 bytes) 2024-09-17T22:42:03.354 INFO:teuthology.orchestra.run.smithi115.stdout:2211 write 0xa2e95ee thru 0xa2ea57a (0xf8d bytes) 2024-09-17T22:42:03.358 INFO:teuthology.orchestra.run.smithi115.stdout:2212 write 0xd0732db thru 0xd0784a3 (0x51c9 bytes) 2024-09-17T22:42:03.359 INFO:teuthology.orchestra.run.smithi115.stdout:2213 writesame 0x8f6bc10 thru 0x8f75d84 data_size 0x1(0xa175 bytes) 2024-09-17T22:42:03.378 INFO:teuthology.orchestra.run.smithi115.stdout:2214 write 0x195ed1e thru 0x196d669 (0xe94c bytes) 2024-09-17T22:42:03.384 INFO:teuthology.orchestra.run.smithi115.stdout:2215 read 0x439d33c thru 0x43a0f88 (0x3c4d bytes) 2024-09-17T22:42:03.385 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:03.385 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:03.385 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:03.385 INFO:teuthology.orchestra.run.smithi115.stdout:2220 read 0xa43d23b thru 0xa44b7c0 (0xe586 bytes) 2024-09-17T22:42:03.385 INFO:teuthology.orchestra.run.smithi115.stdout:2221 write 0x4aec667 thru 0x4af7f11 (0xb8ab bytes) 2024-09-17T22:42:03.387 INFO:teuthology.orchestra.run.smithi115.stdout:2222 write 0x23fffea thru 0x240b3ed (0xb404 bytes) 2024-09-17T22:42:03.393 INFO:teuthology.orchestra.run.smithi115.stdout:2223 read 0x267a3d5 thru 0x26860dc (0xbd08 bytes) 2024-09-17T22:42:03.393 INFO:teuthology.orchestra.run.smithi115.stdout:2224 write 0xad58683 thru 0xad5db70 (0x54ee bytes) 2024-09-17T22:42:03.395 INFO:teuthology.orchestra.run.smithi115.stdout:2225 writesame 0x2b70ff thru 0x2c13f0 data_size 0x1(0xa2f2 bytes) 2024-09-17T22:42:03.403 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:03.400+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:03.424 INFO:teuthology.orchestra.run.smithi115.stdout:2227 write 0xd7a4ca0 thru 0xd7ab5d3 (0x6934 bytes) 2024-09-17T22:42:03.425 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:03.425 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:03.425 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:03.425 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:03.425 INFO:teuthology.orchestra.run.smithi115.stdout:2230 clone 21 order 23 su 4194304 sc 8 2024-09-17T22:42:03.434 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:42:03.504 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:03.502+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:03.578 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:03.578 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:03.578 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:03.578 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:03.578 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:03.587 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:42:03.632 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:03.630+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:03.691 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:03.691 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:03.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:03.691 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:03.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.691 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145246 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3850240~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.692 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:03.692 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.358349796000001, 2024-09-17T22:42:03.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052644991000000002, 2024-09-17T22:42:03.692 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.692 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.692 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.692 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145246 2024-09-17T22:42:03.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.692 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:03.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:03.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329899+0000", 2024-09-17T22:42:03.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-09-17T22:42:03.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329901+0000", 2024-09-17T22:42:03.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1619999999999998e-06 2024-09-17T22:42:03.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329907+0000", 2024-09-17T22:42:03.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9059999999999999e-06 2024-09-17T22:42:03.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329911+0000", 2024-09-17T22:42:03.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7419999999999997e-06 2024-09-17T22:42:03.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382383+0000", 2024-09-17T22:42:03.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052471583000000002 2024-09-17T22:42:03.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382457+0000", 2024-09-17T22:42:03.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4309999999999998e-05 2024-09-17T22:42:03.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382543+0000", 2024-09-17T22:42:03.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5918e-05 2024-09-17T22:42:03.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.696 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.696 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145247 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3946496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.696 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:03.696 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.358331145999999, 2024-09-17T22:42:03.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052805863000000001, 2024-09-17T22:42:03.696 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.696 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.696 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.696 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145247 2024-09-17T22:42:03.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.697 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:03.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:03.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:03.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3200000000000001e-07 2024-09-17T22:42:03.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329919+0000", 2024-09-17T22:42:03.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.888e-06 2024-09-17T22:42:03.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329925+0000", 2024-09-17T22:42:03.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5369999999999997e-06 2024-09-17T22:42:03.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329929+0000", 2024-09-17T22:42:03.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.639e-06 2024-09-17T22:42:03.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382570+0000", 2024-09-17T22:42:03.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052641213999999999 2024-09-17T22:42:03.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382636+0000", 2024-09-17T22:42:03.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6008999999999998e-05 2024-09-17T22:42:03.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382723+0000", 2024-09-17T22:42:03.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6243999999999996e-05 2024-09-17T22:42:03.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.700 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.700 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145248 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3856384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.700 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:03.700 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.358313098, 2024-09-17T22:42:03.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053002858, 2024-09-17T22:42:03.700 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.700 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.701 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.701 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145248 2024-09-17T22:42:03.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.701 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:03.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:03.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:03.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.13e-07 2024-09-17T22:42:03.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329937+0000", 2024-09-17T22:42:03.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9300000000000002e-06 2024-09-17T22:42:03.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329943+0000", 2024-09-17T22:42:03.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-09-17T22:42:03.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329948+0000", 2024-09-17T22:42:03.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8579999999999999e-06 2024-09-17T22:42:03.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382763+0000", 2024-09-17T22:42:03.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052814537000000002 2024-09-17T22:42:03.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382840+0000", 2024-09-17T22:42:03.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6879999999999996e-05 2024-09-17T22:42:03.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382938+0000", 2024-09-17T22:42:03.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8159999999999995e-05 2024-09-17T22:42:03.704 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.704 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.704 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.704 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145249 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3952640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.704 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:03.704 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.358294196999999, 2024-09-17T22:42:03.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053188602000000001, 2024-09-17T22:42:03.705 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.705 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.705 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.705 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145249 2024-09-17T22:42:03.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.705 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:03.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:03.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:03.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3700000000000001e-07 2024-09-17T22:42:03.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329956+0000", 2024-09-17T22:42:03.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1430000000000001e-06 2024-09-17T22:42:03.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329962+0000", 2024-09-17T22:42:03.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8270000000000003e-06 2024-09-17T22:42:03.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329967+0000", 2024-09-17T22:42:03.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7249999999999997e-06 2024-09-17T22:42:03.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382969+0000", 2024-09-17T22:42:03.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053001952999999997 2024-09-17T22:42:03.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383040+0000", 2024-09-17T22:42:03.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1272999999999993e-05 2024-09-17T22:42:03.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383142+0000", 2024-09-17T22:42:03.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010234399999999999 2024-09-17T22:42:03.708 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.708 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.709 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.709 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145250 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3862528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.709 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:03.709 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.358275398, 2024-09-17T22:42:03.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053359537999999998, 2024-09-17T22:42:03.709 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.709 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.709 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.709 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.709 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.709 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.709 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145250 2024-09-17T22:42:03.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.710 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:03.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:03.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:03.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:42:03.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329975+0000", 2024-09-17T22:42:03.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2199999999999999e-06 2024-09-17T22:42:03.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329981+0000", 2024-09-17T22:42:03.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1020000000000002e-06 2024-09-17T22:42:03.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329991+0000", 2024-09-17T22:42:03.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6800000000000005e-06 2024-09-17T22:42:03.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383171+0000", 2024-09-17T22:42:03.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053180142 2024-09-17T22:42:03.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383242+0000", 2024-09-17T22:42:03.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1310000000000007e-05 2024-09-17T22:42:03.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383332+0000", 2024-09-17T22:42:03.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9703999999999994e-05 2024-09-17T22:42:03.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.713 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.716 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145251 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3958784~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.716 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:03.716 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.358249698, 2024-09-17T22:42:03.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053438537000000001, 2024-09-17T22:42:03.716 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.716 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.716 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.717 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145251 2024-09-17T22:42:03.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.717 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:03.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:03.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329999+0000", 2024-09-17T22:42:03.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-07 2024-09-17T22:42:03.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330001+0000", 2024-09-17T22:42:03.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:42:03.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330006+0000", 2024-09-17T22:42:03.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000004e-06 2024-09-17T22:42:03.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330011+0000", 2024-09-17T22:42:03.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4039999999999997e-06 2024-09-17T22:42:03.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383350+0000", 2024-09-17T22:42:03.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053339011999999998 2024-09-17T22:42:03.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383387+0000", 2024-09-17T22:42:03.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7085999999999997e-05 2024-09-17T22:42:03.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383437+0000", 2024-09-17T22:42:03.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9988000000000003e-05 2024-09-17T22:42:03.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.720 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.720 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145252 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3868672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.720 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:03.720 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.358231250999999, 2024-09-17T22:42:03.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053526724999999997, 2024-09-17T22:42:03.720 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.720 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.720 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.721 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145252 2024-09-17T22:42:03.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.721 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:03.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:03.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331054+0000", 2024-09-17T22:42:03.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010371670000000001 2024-09-17T22:42:03.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331059+0000", 2024-09-17T22:42:03.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1680000000000003e-06 2024-09-17T22:42:03.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331075+0000", 2024-09-17T22:42:03.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6028e-05 2024-09-17T22:42:03.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331084+0000", 2024-09-17T22:42:03.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3980000000000005e-06 2024-09-17T22:42:03.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383464+0000", 2024-09-17T22:42:03.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052379851999999998 2024-09-17T22:42:03.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383499+0000", 2024-09-17T22:42:03.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4610000000000002e-05 2024-09-17T22:42:03.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383543+0000", 2024-09-17T22:42:03.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4502000000000002e-05 2024-09-17T22:42:03.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.724 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.725 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145255 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3971072~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.725 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:03.725 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.357101159000001, 2024-09-17T22:42:03.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052677527000000002, 2024-09-17T22:42:03.725 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.725 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.725 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.725 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145255 2024-09-17T22:42:03.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.725 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:03.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:03.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:03.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8800000000000002e-07 2024-09-17T22:42:03.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331150+0000", 2024-09-17T22:42:03.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1160000000000002e-06 2024-09-17T22:42:03.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331156+0000", 2024-09-17T22:42:03.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7469999999999999e-06 2024-09-17T22:42:03.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331162+0000", 2024-09-17T22:42:03.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-17T22:42:03.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383750+0000", 2024-09-17T22:42:03.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052588441999999999 2024-09-17T22:42:03.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383782+0000", 2024-09-17T22:42:03.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1610999999999999e-05 2024-09-17T22:42:03.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383824+0000", 2024-09-17T22:42:03.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2512000000000002e-05 2024-09-17T22:42:03.728 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.729 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.729 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145256 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3880960~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.729 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:03.729 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.357079737999999, 2024-09-17T22:42:03.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052744775000000001, 2024-09-17T22:42:03.729 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.729 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.729 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.729 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.729 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.729 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.729 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145256 2024-09-17T22:42:03.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.730 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:03.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:03.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331169+0000", 2024-09-17T22:42:03.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:42:03.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331175+0000", 2024-09-17T22:42:03.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5660000000000003e-06 2024-09-17T22:42:03.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331182+0000", 2024-09-17T22:42:03.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0169999999999999e-06 2024-09-17T22:42:03.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331188+0000", 2024-09-17T22:42:03.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2750000000000003e-06 2024-09-17T22:42:03.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383839+0000", 2024-09-17T22:42:03.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052651728000000002 2024-09-17T22:42:03.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383872+0000", 2024-09-17T22:42:03.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2347e-05 2024-09-17T22:42:03.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383913+0000", 2024-09-17T22:42:03.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1362000000000001e-05 2024-09-17T22:42:03.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.733 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.733 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145257 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3977216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.733 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:03.733 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.357053690000001, 2024-09-17T22:42:03.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052809598999999999, 2024-09-17T22:42:03.733 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.733 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.733 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.733 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.733 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.734 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145257 2024-09-17T22:42:03.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.734 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:03.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:03.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333051+0000", 2024-09-17T22:42:03.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018567189999999999 2024-09-17T22:42:03.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333056+0000", 2024-09-17T22:42:03.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-17T22:42:03.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333067+0000", 2024-09-17T22:42:03.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1704e-05 2024-09-17T22:42:03.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333074+0000", 2024-09-17T22:42:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1280000000000002e-06 2024-09-17T22:42:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383927+0000", 2024-09-17T22:42:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050853094000000001 2024-09-17T22:42:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383960+0000", 2024-09-17T22:42:03.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2891000000000001e-05 2024-09-17T22:42:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.384004+0000", 2024-09-17T22:42:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3565e-05 2024-09-17T22:42:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146168 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2467840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.293565069, 2024-09-17T22:42:03.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052652501999999997, 2024-09-17T22:42:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146168 2024-09-17T22:42:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394989+0000", 2024-09-17T22:42:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030618300000000002 2024-09-17T22:42:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394994+0000", 2024-09-17T22:42:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.566e-06 2024-09-17T22:42:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395003+0000", 2024-09-17T22:42:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3619999999999995e-06 2024-09-17T22:42:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395009+0000", 2024-09-17T22:42:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3589999999999996e-06 2024-09-17T22:42:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447180+0000", 2024-09-17T22:42:03.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052170643000000003 2024-09-17T22:42:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447251+0000", 2024-09-17T22:42:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1382000000000002e-05 2024-09-17T22:42:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447335+0000", 2024-09-17T22:42:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4006999999999997e-05 2024-09-17T22:42:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146172 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2480128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.293210183999999, 2024-09-17T22:42:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052714629999999998, 2024-09-17T22:42:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146172 2024-09-17T22:42:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395486+0000", 2024-09-17T22:42:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044787000000000001 2024-09-17T22:42:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395491+0000", 2024-09-17T22:42:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8550000000000001e-06 2024-09-17T22:42:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395503+0000", 2024-09-17T22:42:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2271e-05 2024-09-17T22:42:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395510+0000", 2024-09-17T22:42:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6320000000000008e-06 2024-09-17T22:42:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447565+0000", 2024-09-17T22:42:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052055016000000003 2024-09-17T22:42:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447644+0000", 2024-09-17T22:42:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8306000000000005e-05 2024-09-17T22:42:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447752+0000", 2024-09-17T22:42:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010868 2024-09-17T22:42:03.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146181 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2504704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.292342794, 2024-09-17T22:42:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052850394000000002, 2024-09-17T22:42:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.746 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146181 2024-09-17T22:42:03.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.747 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:03.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:03.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396210+0000", 2024-09-17T22:42:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000305214 2024-09-17T22:42:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396214+0000", 2024-09-17T22:42:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9389999999999999e-06 2024-09-17T22:42:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396228+0000", 2024-09-17T22:42:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3365e-05 2024-09-17T22:42:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396235+0000", 2024-09-17T22:42:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2559999999999996e-06 2024-09-17T22:42:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448568+0000", 2024-09-17T22:42:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052333084000000002 2024-09-17T22:42:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448646+0000", 2024-09-17T22:42:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8287000000000004e-05 2024-09-17T22:42:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448756+0000", 2024-09-17T22:42:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109249 2024-09-17T22:42:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146183 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2510848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.292002246999999, 2024-09-17T22:42:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052725453999999998, 2024-09-17T22:42:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.750 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146183 2024-09-17T22:42:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396508+0000", 2024-09-17T22:42:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026271299999999998 2024-09-17T22:42:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396512+0000", 2024-09-17T22:42:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8039999999999999e-06 2024-09-17T22:42:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396522+0000", 2024-09-17T22:42:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9289999999999993e-06 2024-09-17T22:42:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396528+0000", 2024-09-17T22:42:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2990000000000002e-06 2024-09-17T22:42:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448798+0000", 2024-09-17T22:42:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052269985999999997 2024-09-17T22:42:03.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448871+0000", 2024-09-17T22:42:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2581000000000002e-05 2024-09-17T22:42:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448971+0000", 2024-09-17T22:42:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100142 2024-09-17T22:42:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146391 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.284274755, 2024-09-17T22:42:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052666719000000001, 2024-09-17T22:42:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.755 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146391 2024-09-17T22:42:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:03.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404098+0000", 2024-09-17T22:42:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000124509 2024-09-17T22:42:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404107+0000", 2024-09-17T22:42:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4220000000000006e-06 2024-09-17T22:42:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404117+0000", 2024-09-17T22:42:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011e-05 2024-09-17T22:42:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404123+0000", 2024-09-17T22:42:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1530000000000002e-06 2024-09-17T22:42:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456481+0000", 2024-09-17T22:42:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052357923000000001 2024-09-17T22:42:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456551+0000", 2024-09-17T22:42:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9869000000000005e-05 2024-09-17T22:42:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456640+0000", 2024-09-17T22:42:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8733000000000006e-05 2024-09-17T22:42:03.759 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146393 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2748416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.284114691999999, 2024-09-17T22:42:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052739239, 2024-09-17T22:42:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.760 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146393 2024-09-17T22:42:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404443+0000", 2024-09-17T22:42:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030977199999999999 2024-09-17T22:42:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404447+0000", 2024-09-17T22:42:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6370000000000001e-06 2024-09-17T22:42:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404469+0000", 2024-09-17T22:42:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2739999999999999e-05 2024-09-17T22:42:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404477+0000", 2024-09-17T22:42:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6559999999999992e-06 2024-09-17T22:42:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456734+0000", 2024-09-17T22:42:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052256889000000001 2024-09-17T22:42:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456796+0000", 2024-09-17T22:42:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2485999999999998e-05 2024-09-17T22:42:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456873+0000", 2024-09-17T22:42:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6058999999999998e-05 2024-09-17T22:42:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146397 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2760704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.283738928, 2024-09-17T22:42:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052788928999999998, 2024-09-17T22:42:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.765 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146397 2024-09-17T22:42:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404763+0000", 2024-09-17T22:42:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025351399999999999 2024-09-17T22:42:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404767+0000", 2024-09-17T22:42:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:42:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404777+0000", 2024-09-17T22:42:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9310000000000003e-06 2024-09-17T22:42:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404783+0000", 2024-09-17T22:42:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.336e-06 2024-09-17T22:42:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457126+0000", 2024-09-17T22:42:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052342975999999999 2024-09-17T22:42:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457200+0000", 2024-09-17T22:42:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4146000000000005e-05 2024-09-17T22:42:03.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457298+0000", 2024-09-17T22:42:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7807999999999996e-05 2024-09-17T22:42:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146399 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2766848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.283448786999999, 2024-09-17T22:42:03.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052829779, 2024-09-17T22:42:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146399 2024-09-17T22:42:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:03.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404800+0000", 2024-09-17T22:42:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8500000000000001e-07 2024-09-17T22:42:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404802+0000", 2024-09-17T22:42:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5349999999999999e-06 2024-09-17T22:42:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404811+0000", 2024-09-17T22:42:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9390000000000003e-06 2024-09-17T22:42:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404817+0000", 2024-09-17T22:42:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.524e-06 2024-09-17T22:42:03.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457461+0000", 2024-09-17T22:42:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052643784999999998 2024-09-17T22:42:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457539+0000", 2024-09-17T22:42:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8589000000000003e-05 2024-09-17T22:42:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457629+0000", 2024-09-17T22:42:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9821999999999995e-05 2024-09-17T22:42:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146401 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2772992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.283423539999999, 2024-09-17T22:42:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052910077, 2024-09-17T22:42:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.774 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146401 2024-09-17T22:42:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:03.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405106+0000", 2024-09-17T22:42:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000281286 2024-09-17T22:42:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405110+0000", 2024-09-17T22:42:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1459999999999998e-06 2024-09-17T22:42:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405120+0000", 2024-09-17T22:42:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0546e-05 2024-09-17T22:42:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405127+0000", 2024-09-17T22:42:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4300000000000003e-06 2024-09-17T22:42:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457647+0000", 2024-09-17T22:42:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052520229000000002 2024-09-17T22:42:03.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457684+0000", 2024-09-17T22:42:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6609999999999997e-05 2024-09-17T22:42:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457735+0000", 2024-09-17T22:42:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.083e-05 2024-09-17T22:42:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146402 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2779136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.283110128000001, 2024-09-17T22:42:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052689812000000003, 2024-09-17T22:42:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146402 2024-09-17T22:42:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:03.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405365+0000", 2024-09-17T22:42:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022744599999999999 2024-09-17T22:42:03.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405370+0000", 2024-09-17T22:42:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2769999999999999e-06 2024-09-17T22:42:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405380+0000", 2024-09-17T22:42:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0349e-05 2024-09-17T22:42:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:03.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405386+0000", 2024-09-17T22:42:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5169999999999996e-06 2024-09-17T22:42:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457750+0000", 2024-09-17T22:42:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052363448999999999 2024-09-17T22:42:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457784+0000", 2024-09-17T22:42:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4363999999999998e-05 2024-09-17T22:42:03.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457828+0000", 2024-09-17T22:42:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999999e-05 2024-09-17T22:42:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:03.783 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:03.783 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:03.785 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:42:03.785 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:03.782+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:03.886 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:03.883+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:03.929 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:03.929 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:03.929 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:03.929 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:03.937 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:42:03.987 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:03.984+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:04.039 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:04.039 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:04.039 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:04.039 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:04.039 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:04.048 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:42:04.089 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:04.086+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:04.170 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is unset 2024-09-17T22:42:04.190 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:04.187+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:04.199 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:04.199 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:04.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:04.199 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:04.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.199 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4229.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-17T22:42:04.199 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:04.200 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.801470197, 2024-09-17T22:42:04.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25708877200000002, 2024-09-17T22:42:04.200 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.200 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.200 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.200 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.200 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4229", 2024-09-17T22:42:04.200 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/3080427038", 2024-09-17T22:42:04.200 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2 2024-09-17T22:42:04.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.200 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:04.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:04.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394611+0000", 2024-09-17T22:42:04.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3149999999999999e-06 2024-09-17T22:42:04.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394619+0000", 2024-09-17T22:42:04.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240000000000003e-06 2024-09-17T22:42:04.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394632+0000", 2024-09-17T22:42:04.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2921e-05 2024-09-17T22:42:04.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394639+0000", 2024-09-17T22:42:04.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-09-17T22:42:04.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394703+0000", 2024-09-17T22:42:04.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4407999999999997e-05 2024-09-17T22:42:04.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:42:04.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394711+0000", 2024-09-17T22:42:04.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5939999999999996e-06 2024-09-17T22:42:04.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651545+0000", 2024-09-17T22:42:04.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25683409200000001 2024-09-17T22:42:04.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651698+0000", 2024-09-17T22:42:04.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000153678 2024-09-17T22:42:04.204 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.204 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.204 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.204 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38736 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:04.204 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:04.205 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.418874191999997, 2024-09-17T22:42:04.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051395418999999998, 2024-09-17T22:42:04.205 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.205 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.205 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:04.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:04.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.205 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38736 2024-09-17T22:42:04.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.205 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:04.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:04.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:04.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6899999999999998e-07 2024-09-17T22:42:04.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777212+0000", 2024-09-17T22:42:04.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1240000000000002e-06 2024-09-17T22:42:04.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777219+0000", 2024-09-17T22:42:04.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5409999999999997e-06 2024-09-17T22:42:04.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777224+0000", 2024-09-17T22:42:04.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:42:04.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.826131+0000", 2024-09-17T22:42:04.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048907369999999999 2024-09-17T22:42:04.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:04.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828457+0000", 2024-09-17T22:42:04.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023256209999999999 2024-09-17T22:42:04.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828601+0000", 2024-09-17T22:42:04.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014438799999999999 2024-09-17T22:42:04.208 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.209 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.209 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38752 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 673792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:04.209 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:04.209 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.418850120999998, 2024-09-17T22:42:04.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051582301999999997, 2024-09-17T22:42:04.209 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.209 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.209 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:04.209 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:04.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.210 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38752 2024-09-17T22:42:04.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.210 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:04.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:04.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:04.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3000000000000001e-07 2024-09-17T22:42:04.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777232+0000", 2024-09-17T22:42:04.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9709999999999998e-06 2024-09-17T22:42:04.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777238+0000", 2024-09-17T22:42:04.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7209999999999999e-06 2024-09-17T22:42:04.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777243+0000", 2024-09-17T22:42:04.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5210000000000004e-06 2024-09-17T22:42:04.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828633+0000", 2024-09-17T22:42:04.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051390915000000002 2024-09-17T22:42:04.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:04.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828711+0000", 2024-09-17T22:42:04.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7474999999999999e-05 2024-09-17T22:42:04.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828812+0000", 2024-09-17T22:42:04.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010116900000000001 2024-09-17T22:42:04.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.213 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.213 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38753 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:04.213 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:04.213 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.418830847999999, 2024-09-17T22:42:04.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051770444999999998, 2024-09-17T22:42:04.214 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.214 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.214 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:04.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:04.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.214 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38753 2024-09-17T22:42:04.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.214 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:04.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:04.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:04.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:42:04.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777251+0000", 2024-09-17T22:42:04.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8059999999999999e-06 2024-09-17T22:42:04.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777257+0000", 2024-09-17T22:42:04.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2480000000000001e-06 2024-09-17T22:42:04.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777262+0000", 2024-09-17T22:42:04.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000004e-06 2024-09-17T22:42:04.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828842+0000", 2024-09-17T22:42:04.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051580203999999998 2024-09-17T22:42:04.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:04.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828917+0000", 2024-09-17T22:42:04.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4980000000000004e-05 2024-09-17T22:42:04.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829019+0000", 2024-09-17T22:42:04.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102248 2024-09-17T22:42:04.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.218 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.218 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38766 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:04.218 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:04.218 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.418811863000002, 2024-09-17T22:42:04.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051958839999999999, 2024-09-17T22:42:04.218 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.218 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.218 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:04.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:04.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.219 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38766 2024-09-17T22:42:04.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.219 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:04.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:04.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777269+0000", 2024-09-17T22:42:04.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:42:04.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777271+0000", 2024-09-17T22:42:04.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-17T22:42:04.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777276+0000", 2024-09-17T22:42:04.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:42:04.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777281+0000", 2024-09-17T22:42:04.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:42:04.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829055+0000", 2024-09-17T22:42:04.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051773570999999997 2024-09-17T22:42:04.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:04.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829128+0000", 2024-09-17T22:42:04.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3131000000000002e-05 2024-09-17T22:42:04.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829227+0000", 2024-09-17T22:42:04.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8800000000000003e-05 2024-09-17T22:42:04.222 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.222 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.222 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.222 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38767 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:04.222 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:04.223 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.418791880000001, 2024-09-17T22:42:04.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137018, 2024-09-17T22:42:04.223 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.223 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.223 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:04.223 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.223 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:04.223 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.223 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38767 2024-09-17T22:42:04.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.223 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:04.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:04.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:04.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:42:04.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777290+0000", 2024-09-17T22:42:04.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.88e-06 2024-09-17T22:42:04.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777296+0000", 2024-09-17T22:42:04.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-09-17T22:42:04.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777300+0000", 2024-09-17T22:42:04.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4510000000000002e-06 2024-09-17T22:42:04.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829257+0000", 2024-09-17T22:42:04.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051956975000000002 2024-09-17T22:42:04.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:04.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829328+0000", 2024-09-17T22:42:04.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1026999999999996e-05 2024-09-17T22:42:04.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829425+0000", 2024-09-17T22:42:04.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.679e-05 2024-09-17T22:42:04.227 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.227 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.227 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.227 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38784 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 675840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:04.227 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:04.227 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.418773426000001, 2024-09-17T22:42:04.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052344522999999997, 2024-09-17T22:42:04.227 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.227 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.227 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:04.228 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.228 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:04.228 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.228 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38784 2024-09-17T22:42:04.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.228 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:04.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:04.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777307+0000", 2024-09-17T22:42:04.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:42:04.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777309+0000", 2024-09-17T22:42:04.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.063e-06 2024-09-17T22:42:04.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777314+0000", 2024-09-17T22:42:04.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-17T22:42:04.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777319+0000", 2024-09-17T22:42:04.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999996e-06 2024-09-17T22:42:04.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829456+0000", 2024-09-17T22:42:04.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137151 2024-09-17T22:42:04.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:04.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829527+0000", 2024-09-17T22:42:04.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1588000000000006e-05 2024-09-17T22:42:04.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829651+0000", 2024-09-17T22:42:04.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012365 2024-09-17T22:42:04.231 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.231 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.231 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.231 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38785 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 749568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:04.232 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:04.232 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.418755310999998, 2024-09-17T22:42:04.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052546572999999999, 2024-09-17T22:42:04.232 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.232 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.232 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:04.232 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.232 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:04.232 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.232 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38785 2024-09-17T22:42:04.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.232 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:04.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:04.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:04.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:42:04.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777327+0000", 2024-09-17T22:42:04.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:42:04.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777333+0000", 2024-09-17T22:42:04.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7309999999999998e-06 2024-09-17T22:42:04.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777337+0000", 2024-09-17T22:42:04.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:42:04.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829686+0000", 2024-09-17T22:42:04.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052349159999999999 2024-09-17T22:42:04.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:04.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829766+0000", 2024-09-17T22:42:04.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9326999999999995e-05 2024-09-17T22:42:04.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829871+0000", 2024-09-17T22:42:04.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000105466 2024-09-17T22:42:04.236 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.236 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.236 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.236 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38798 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:04.236 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:04.236 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.418121702999997, 2024-09-17T22:42:04.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052117564999999998, 2024-09-17T22:42:04.236 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.236 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.236 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:04.236 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:04.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.237 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38798 2024-09-17T22:42:04.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.237 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:04.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:04.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777959+0000", 2024-09-17T22:42:04.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9599999999999999e-07 2024-09-17T22:42:04.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777963+0000", 2024-09-17T22:42:04.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2660000000000003e-06 2024-09-17T22:42:04.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777971+0000", 2024-09-17T22:42:04.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3489999999999992e-06 2024-09-17T22:42:04.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777978+0000", 2024-09-17T22:42:04.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:42:04.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829906+0000", 2024-09-17T22:42:04.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051927635999999999 2024-09-17T22:42:04.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:04.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829986+0000", 2024-09-17T22:42:04.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0444000000000004e-05 2024-09-17T22:42:04.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830076+0000", 2024-09-17T22:42:04.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9473000000000002e-05 2024-09-17T22:42:04.240 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.240 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.240 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.241 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38799 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 882688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:04.241 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:04.241 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.418090264, 2024-09-17T22:42:04.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052359606000000003, 2024-09-17T22:42:04.241 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.241 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.241 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:04.241 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.241 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:04.241 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.241 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38799 2024-09-17T22:42:04.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.241 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:04.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:04.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:04.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8299999999999997e-07 2024-09-17T22:42:04.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777992+0000", 2024-09-17T22:42:04.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.176e-06 2024-09-17T22:42:04.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777998+0000", 2024-09-17T22:42:04.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.173e-06 2024-09-17T22:42:04.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778003+0000", 2024-09-17T22:42:04.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6349999999999997e-06 2024-09-17T22:42:04.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830112+0000", 2024-09-17T22:42:04.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052109366999999997 2024-09-17T22:42:04.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:04.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830192+0000", 2024-09-17T22:42:04.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9814000000000005e-05 2024-09-17T22:42:04.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830349+0000", 2024-09-17T22:42:04.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015695800000000001 2024-09-17T22:42:04.245 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.245 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.245 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.245 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38814 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:04.245 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:04.245 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.417806859999999, 2024-09-17T22:42:04.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052274339000000003, 2024-09-17T22:42:04.245 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.245 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.245 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:04.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.246 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:04.246 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.246 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38814 2024-09-17T22:42:04.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.246 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:04.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:04.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:04.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:42:04.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778276+0000", 2024-09-17T22:42:04.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2029999999999999e-06 2024-09-17T22:42:04.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778282+0000", 2024-09-17T22:42:04.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0059999999999998e-06 2024-09-17T22:42:04.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778287+0000", 2024-09-17T22:42:04.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0039999999999999e-06 2024-09-17T22:42:04.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830382+0000", 2024-09-17T22:42:04.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052095379999999997 2024-09-17T22:42:04.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:04.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830455+0000", 2024-09-17T22:42:04.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3127000000000006e-05 2024-09-17T22:42:04.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830547+0000", 2024-09-17T22:42:04.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2207999999999995e-05 2024-09-17T22:42:04.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.249 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.249 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38828 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 163840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:04.249 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:04.249 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.417624451999998, 2024-09-17T22:42:04.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052466039999999999, 2024-09-17T22:42:04.249 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.250 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.250 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:04.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:04.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.250 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38828 2024-09-17T22:42:04.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.250 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:04.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:04.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778456+0000", 2024-09-17T22:42:04.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7599999999999996e-07 2024-09-17T22:42:04.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778460+0000", 2024-09-17T22:42:04.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8229999999999997e-06 2024-09-17T22:42:04.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778470+0000", 2024-09-17T22:42:04.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8199999999999992e-06 2024-09-17T22:42:04.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778476+0000", 2024-09-17T22:42:04.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2589999999999997e-06 2024-09-17T22:42:04.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830578+0000", 2024-09-17T22:42:04.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052102273999999997 2024-09-17T22:42:04.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:04.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830667+0000", 2024-09-17T22:42:04.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8701000000000003e-05 2024-09-17T22:42:04.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830921+0000", 2024-09-17T22:42:04.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025438700000000002 2024-09-17T22:42:04.253 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.253 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.254 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.254 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38829 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:04.254 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:04.254 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.417593054999998, 2024-09-17T22:42:04.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052648882000000001, 2024-09-17T22:42:04.254 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.254 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.254 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:04.254 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.254 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:04.254 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.254 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38829 2024-09-17T22:42:04.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.255 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:04.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:04.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:04.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7e-07 2024-09-17T22:42:04.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778490+0000", 2024-09-17T22:42:04.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:42:04.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778495+0000", 2024-09-17T22:42:04.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9329999999999998e-06 2024-09-17T22:42:04.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778500+0000", 2024-09-17T22:42:04.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9509999999999997e-06 2024-09-17T22:42:04.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830958+0000", 2024-09-17T22:42:04.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05245785 2024-09-17T22:42:04.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:04.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831051+0000", 2024-09-17T22:42:04.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2989e-05 2024-09-17T22:42:04.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831136+0000", 2024-09-17T22:42:04.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4448000000000002e-05 2024-09-17T22:42:04.258 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.258 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.258 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.258 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38891 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:04.258 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:04.258 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.414280996999999, 2024-09-17T22:42:04.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050847934999999997, 2024-09-17T22:42:04.259 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.259 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.259 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:04.259 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.259 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:04.259 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.259 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38891 2024-09-17T22:42:04.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.259 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:04.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:04.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:04.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-09-17T22:42:04.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781802+0000", 2024-09-17T22:42:04.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:42:04.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781810+0000", 2024-09-17T22:42:04.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8019999999999992e-06 2024-09-17T22:42:04.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781816+0000", 2024-09-17T22:42:04.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.285e-06 2024-09-17T22:42:04.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832485+0000", 2024-09-17T22:42:04.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050668462999999997 2024-09-17T22:42:04.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:04.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832565+0000", 2024-09-17T22:42:04.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9692999999999997e-05 2024-09-17T22:42:04.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832647+0000", 2024-09-17T22:42:04.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2224000000000004e-05 2024-09-17T22:42:04.263 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.263 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.263 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.264 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38892 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:04.264 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:04.264 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.414252748000003, 2024-09-17T22:42:04.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050946120999999997, 2024-09-17T22:42:04.264 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.264 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.264 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:04.264 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.264 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:04.264 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.264 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38892 2024-09-17T22:42:04.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.265 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:04.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:04.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781828+0000", 2024-09-17T22:42:04.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5400000000000002e-07 2024-09-17T22:42:04.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781830+0000", 2024-09-17T22:42:04.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1959999999999998e-06 2024-09-17T22:42:04.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781837+0000", 2024-09-17T22:42:04.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8310000000000003e-06 2024-09-17T22:42:04.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781843+0000", 2024-09-17T22:42:04.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1e-06 2024-09-17T22:42:04.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832674+0000", 2024-09-17T22:42:04.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050831182000000003 2024-09-17T22:42:04.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:04.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832725+0000", 2024-09-17T22:42:04.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1588000000000001e-05 2024-09-17T22:42:04.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832773+0000", 2024-09-17T22:42:04.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7769999999999998e-05 2024-09-17T22:42:04.268 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.268 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.268 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.268 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38909 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 36864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:04.268 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:04.268 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.414229978000002, 2024-09-17T22:42:04.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051046518999999999, 2024-09-17T22:42:04.269 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.269 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.269 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:04.269 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.269 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:04.269 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.269 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38909 2024-09-17T22:42:04.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.269 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:04.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:04.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:04.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:42:04.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781852+0000", 2024-09-17T22:42:04.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0590000000000001e-06 2024-09-17T22:42:04.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781859+0000", 2024-09-17T22:42:04.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3380000000000001e-06 2024-09-17T22:42:04.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781864+0000", 2024-09-17T22:42:04.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:42:04.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832804+0000", 2024-09-17T22:42:04.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050939614000000001 2024-09-17T22:42:04.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:04.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832845+0000", 2024-09-17T22:42:04.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1014999999999999e-05 2024-09-17T22:42:04.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832896+0000", 2024-09-17T22:42:04.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1836999999999999e-05 2024-09-17T22:42:04.272 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.272 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.273 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.273 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38910 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:04.273 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:04.273 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.414209667999998, 2024-09-17T22:42:04.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051216818999999997, 2024-09-17T22:42:04.273 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.273 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.273 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:04.273 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.273 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:04.273 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.273 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38910 2024-09-17T22:42:04.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.274 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:04.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:04.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781871+0000", 2024-09-17T22:42:04.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:42:04.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781873+0000", 2024-09-17T22:42:04.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:42:04.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781879+0000", 2024-09-17T22:42:04.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0700000000000003e-06 2024-09-17T22:42:04.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781884+0000", 2024-09-17T22:42:04.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-09-17T22:42:04.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832932+0000", 2024-09-17T22:42:04.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051048219999999998 2024-09-17T22:42:04.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:04.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833001+0000", 2024-09-17T22:42:04.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8504999999999995e-05 2024-09-17T22:42:04.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833087+0000", 2024-09-17T22:42:04.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6464000000000004e-05 2024-09-17T22:42:04.277 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.277 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.277 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.277 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38923 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:04.277 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:04.277 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.414189348000001, 2024-09-17T22:42:04.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051379573999999997, 2024-09-17T22:42:04.277 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.277 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.278 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:04.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:04.278 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.278 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38923 2024-09-17T22:42:04.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.278 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:04.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:04.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:04.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:42:04.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781893+0000", 2024-09-17T22:42:04.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-09-17T22:42:04.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781900+0000", 2024-09-17T22:42:04.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3860000000000004e-06 2024-09-17T22:42:04.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781904+0000", 2024-09-17T22:42:04.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9300000000000002e-06 2024-09-17T22:42:04.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833115+0000", 2024-09-17T22:42:04.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051210845999999997 2024-09-17T22:42:04.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:04.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833188+0000", 2024-09-17T22:42:04.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2276999999999999e-05 2024-09-17T22:42:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833270+0000", 2024-09-17T22:42:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2501000000000002e-05 2024-09-17T22:42:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38941 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 34.413396812999999, 2024-09-17T22:42:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050969459000000002, 2024-09-17T22:42:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38941 2024-09-17T22:42:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:04.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782684+0000", 2024-09-17T22:42:04.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:42:04.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782686+0000", 2024-09-17T22:42:04.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.396e-06 2024-09-17T22:42:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782693+0000", 2024-09-17T22:42:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1999999999999997e-06 2024-09-17T22:42:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782699+0000", 2024-09-17T22:42:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7910000000000001e-06 2024-09-17T22:42:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833471+0000", 2024-09-17T22:42:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050772333000000003 2024-09-17T22:42:04.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:04.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833536+0000", 2024-09-17T22:42:04.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4380000000000004e-05 2024-09-17T22:42:04.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833653+0000", 2024-09-17T22:42:04.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011691799999999999 2024-09-17T22:42:04.285 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.285 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.285 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.286 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:151189 3.14 3:2b696b13:::rbd_data.115f14f2d1d5.0000000000000009:e [read 3127296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.286 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:04.286 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.483176068000001, 2024-09-17T22:42:04.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.054254700000000003, 2024-09-17T22:42:04.286 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:04.286 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:04.286 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:04.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:04.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:04.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.286 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 151189 2024-09-17T22:42:04.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.286 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:04.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:04.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:04.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:04.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:04.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:04.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:04.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713051+0000", 2024-09-17T22:42:04.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014722100000000001 2024-09-17T22:42:04.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:04.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713055+0000", 2024-09-17T22:42:04.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.16e-06 2024-09-17T22:42:04.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:04.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713065+0000", 2024-09-17T22:42:04.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8390000000000002e-06 2024-09-17T22:42:04.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713073+0000", 2024-09-17T22:42:04.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0390000000000005e-06 2024-09-17T22:42:04.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:04.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762644+0000", 2024-09-17T22:42:04.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049571160000000003 2024-09-17T22:42:04.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:04.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762739+0000", 2024-09-17T22:42:04.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4565e-05 2024-09-17T22:42:04.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:04.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:04.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:04.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.767159+0000", 2024-09-17T22:42:04.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044197159999999997 2024-09-17T22:42:04.289 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.290 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.290 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.290 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:04.290 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:04.290 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:04.291 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:42:04.292 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone20 from 0xdc5dbca (overlap 0xdc5dbca) to 0xb04afd4 2024-09-17T22:42:04.292 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:04.288+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:04.392 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:04.390+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:04.400 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:04.401 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:04.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.401 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:180627 3.2 3:427eee3b:::rbd_header.115f31bbb4b1:head [call rbd.parent_get in=13b,call rbd.parent_overlap_get in=29b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-17T22:42:04.401 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:04.398876+0000", 2024-09-17T22:42:04.401 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.00017574100000000001, 2024-09-17T22:42:04.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00019685699999999999, 2024-09-17T22:42:04.401 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.401 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.401 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:04.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.402 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 180627 2024-09-17T22:42:04.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.402 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:04.398876+0000", 2024-09-17T22:42:04.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:04.398876+0000", 2024-09-17T22:42:04.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:04.398876+0000", 2024-09-17T22:42:04.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2399999999999998e-07 2024-09-17T22:42:04.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:04.398879+0000", 2024-09-17T22:42:04.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7489999999999999e-06 2024-09-17T22:42:04.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:04.398885+0000", 2024-09-17T22:42:04.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9889999999999997e-06 2024-09-17T22:42:04.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:04.398888+0000", 2024-09-17T22:42:04.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5020000000000001e-06 2024-09-17T22:42:04.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:04.398925+0000", 2024-09-17T22:42:04.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6661e-05 2024-09-17T22:42:04.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:04.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:04.398994+0000", 2024-09-17T22:42:04.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9397e-05 2024-09-17T22:42:04.404 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.404 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.404 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.404 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.405 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T22:42:04.405 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 1 2024-09-17T22:42:04.405 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:04.410 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:42:04.440 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-17T22:42:04.449 INFO:teuthology.orchestra.run.smithi115.stdout:2231 punch from 0x4b0ebac to 0x4b14a0f, (0x5e63 bytes) 2024-09-17T22:42:04.452 INFO:teuthology.orchestra.run.smithi115.stdout:2233 writesame 0x8cf4d0f thru 0x8cf7e6f data_size 0x1(0x3161 bytes) 2024-09-17T22:42:04.461 INFO:teuthology.orchestra.run.smithi115.stdout:2234 writesame 0x2a37e4 thru 0x2a853d data_size 0x1(0x4d5a bytes) 2024-09-17T22:42:04.482 INFO:teuthology.orchestra.run.smithi115.stdout:2235 write 0x8efa60a thru 0x8effd9e (0x5795 bytes) 2024-09-17T22:42:04.485 INFO:teuthology.orchestra.run.smithi115.stdout:2236 read 0x9fe6f2e thru 0x9ff2ccb (0xbd9e bytes) 2024-09-17T22:42:04.485 INFO:teuthology.orchestra.run.smithi115.stdout:2237 read 0xc57fe39 thru 0xc580d12 (0xeda bytes) 2024-09-17T22:42:04.485 INFO:teuthology.orchestra.run.smithi115.stdout:2238 read 0x8f8420b thru 0x8f930e4 (0xeeda bytes) 2024-09-17T22:42:04.486 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:04.486 INFO:teuthology.orchestra.run.smithi115.stdout:2241 trunc from 0xdc5dbca to 0xd7ed545 2024-09-17T22:42:04.493 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:04.491+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:04.495 INFO:teuthology.orchestra.run.smithi115.stdout:2243 read 0x7714595 thru 0x7715f93 (0x19ff bytes) 2024-09-17T22:42:04.496 INFO:teuthology.orchestra.run.smithi115.stdout:2244 punch from 0x880aac4 to 0x8819486, (0xe9c2 bytes) 2024-09-17T22:42:04.496 INFO:teuthology.orchestra.run.smithi115.stdout:2246 punch from 0xb6d7f6d to 0xb6e341d, (0xb4b0 bytes) 2024-09-17T22:42:04.496 INFO:teuthology.orchestra.run.smithi115.stdout:2248 punch from 0x7d43368 to 0x7d4942b, (0x60c3 bytes) 2024-09-17T22:42:04.496 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:04.496 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:04.496 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:04.496 INFO:teuthology.orchestra.run.smithi115.stdout:2258 writesame 0x2722bae thru 0x2731f79 data_size 0x2a(0xf3cc bytes) 2024-09-17T22:42:04.500 INFO:teuthology.orchestra.run.smithi115.stdout:2259 read 0x4e09464 thru 0x4e14313 (0xaeb0 bytes) 2024-09-17T22:42:04.506 INFO:teuthology.orchestra.run.smithi115.stdout:2261 trunc from 0xd7ed545 to 0x927e0b3 2024-09-17T22:42:04.518 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:04.518 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:04.518 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:04.518 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:04.519 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:04.520 INFO:teuthology.orchestra.run.smithi115.stdout:2262 punch from 0x65def20 to 0x65e4e5e, (0x5f3e bytes) 2024-09-17T22:42:04.527 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:42:04.528 INFO:teuthology.orchestra.run.smithi115.stdout:2263 trunc from 0x927e0b3 to 0x3c62ba0 2024-09-17T22:42:04.543 INFO:teuthology.orchestra.run.smithi115.stdout:2264 trunc from 0x3c62ba0 to 0x1f031fe 2024-09-17T22:42:04.555 INFO:teuthology.orchestra.run.smithi115.stdout:2266 read 0x1cf7642 thru 0x1d069cb (0xf38a bytes) 2024-09-17T22:42:04.556 INFO:teuthology.orchestra.run.smithi115.stdout:2267 trunc from 0x1f031fe to 0x15e7866 2024-09-17T22:42:04.565 INFO:teuthology.orchestra.run.smithi115.stdout:2268 punch from 0x1188f6d to 0x11983b4, (0xf447 bytes) 2024-09-17T22:42:04.565 INFO:teuthology.orchestra.run.smithi115.stdout:2269 punch from 0x6b8451 to 0x6c6cd8, (0xe887 bytes) 2024-09-17T22:42:04.565 INFO:teuthology.orchestra.run.smithi115.stdout:2270 punch from 0x1570684 to 0x1576dee, (0x676a bytes) 2024-09-17T22:42:04.566 INFO:teuthology.orchestra.run.smithi115.stdout:2271 read 0xfc5ccd thru 0xfd2f86 (0xd2ba bytes) 2024-09-17T22:42:04.580 INFO:teuthology.orchestra.run.smithi115.stdout:2272 writesame 0x97952b1 thru 0x9798f2c data_size 0x4f(0x3c7c bytes) 2024-09-17T22:42:04.586 INFO:teuthology.orchestra.run.smithi115.stdout:2278 writesame 0xabc4b2b thru 0xabcceda data_size 0x1c(0x83b0 bytes) 2024-09-17T22:42:04.586 INFO:teuthology.orchestra.run.smithi115.stdout:2279 writesame 0x786b13e thru 0x7871a8b data_size 0x3(0x694e bytes) 2024-09-17T22:42:04.594 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:04.591+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:04.601 INFO:teuthology.orchestra.run.smithi115.stdout:2281 writesame 0xd68b2f3 thru 0xd698530 data_size 0x11(0xd23e bytes) 2024-09-17T22:42:04.611 INFO:teuthology.orchestra.run.smithi115.stdout:2282 trunc from 0xd698531 to 0x4c78b72 2024-09-17T22:42:04.631 INFO:teuthology.orchestra.run.smithi115.stdout:2283 write 0x994d736 thru 0x995c214 (0xeadf bytes) 2024-09-17T22:42:04.634 INFO:teuthology.orchestra.run.smithi115.stdout:2284 clone 22 order 20 su 32768 sc 11 2024-09-17T22:42:04.639 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:42:04.674 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:04.674 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:04.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:04.674 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:04.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.674 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158673 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.674 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:04.675 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429457428999999, 2024-09-17T22:42:04.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039400614, 2024-09-17T22:42:04.675 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.675 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.675 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.675 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.675 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.675 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.675 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158673 2024-09-17T22:42:04.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.675 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:04.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:04.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:04.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.04e-07 2024-09-17T22:42:04.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241563+0000", 2024-09-17T22:42:04.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5799999999999998e-07 2024-09-17T22:42:04.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241566+0000", 2024-09-17T22:42:04.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8279999999999999e-06 2024-09-17T22:42:04.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241568+0000", 2024-09-17T22:42:04.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2709999999999999e-06 2024-09-17T22:42:04.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280954+0000", 2024-09-17T22:42:04.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039386051999999998 2024-09-17T22:42:04.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280962+0000", 2024-09-17T22:42:04.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3010000000000007e-06 2024-09-17T22:42:04.678 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.678 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.678 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.678 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158675 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.679 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:04.679 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429448138, 2024-09-17T22:42:04.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039408033000000002, 2024-09-17T22:42:04.679 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.679 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.679 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.679 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.679 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.679 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.679 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158675 2024-09-17T22:42:04.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.679 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:04.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:04.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:04.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.36e-07 2024-09-17T22:42:04.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241572+0000", 2024-09-17T22:42:04.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6599999999999994e-07 2024-09-17T22:42:04.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241575+0000", 2024-09-17T22:42:04.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8389999999999998e-06 2024-09-17T22:42:04.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241577+0000", 2024-09-17T22:42:04.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4119999999999999e-06 2024-09-17T22:42:04.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280975+0000", 2024-09-17T22:42:04.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039397205999999997 2024-09-17T22:42:04.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280979+0000", 2024-09-17T22:42:04.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.374e-06 2024-09-17T22:42:04.682 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.682 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.682 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.683 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158677 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.683 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:04.683 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429425695000001, 2024-09-17T22:42:04.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039409307999999997, 2024-09-17T22:42:04.683 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.683 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.683 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.683 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.683 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.683 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.683 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158677 2024-09-17T22:42:04.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.683 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:04.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:04.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241594+0000", 2024-09-17T22:42:04.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:42:04.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241595+0000", 2024-09-17T22:42:04.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2729999999999999e-06 2024-09-17T22:42:04.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241602+0000", 2024-09-17T22:42:04.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:42:04.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241606+0000", 2024-09-17T22:42:04.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3019999999999999e-06 2024-09-17T22:42:04.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280994+0000", 2024-09-17T22:42:04.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039388484000000001 2024-09-17T22:42:04.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281003+0000", 2024-09-17T22:42:04.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5459999999999992e-06 2024-09-17T22:42:04.686 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.687 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.687 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.687 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158679 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.687 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:04.687 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429410017, 2024-09-17T22:42:04.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039413403999999999, 2024-09-17T22:42:04.687 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.687 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.687 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.687 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.687 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.687 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.688 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158679 2024-09-17T22:42:04.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.688 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:04.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:04.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:04.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4999999999999999e-07 2024-09-17T22:42:04.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241613+0000", 2024-09-17T22:42:04.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.207e-06 2024-09-17T22:42:04.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241616+0000", 2024-09-17T22:42:04.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.275e-06 2024-09-17T22:42:04.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241618+0000", 2024-09-17T22:42:04.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:42:04.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281017+0000", 2024-09-17T22:42:04.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039398385000000001 2024-09-17T22:42:04.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281022+0000", 2024-09-17T22:42:04.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.896e-06 2024-09-17T22:42:04.691 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.691 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.691 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.691 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158681 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.691 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:04.691 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429397839, 2024-09-17T22:42:04.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039425292000000001, 2024-09-17T22:42:04.692 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.692 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.692 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.692 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.692 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.692 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.692 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158681 2024-09-17T22:42:04.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.692 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:04.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:04.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:04.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:42:04.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241622+0000", 2024-09-17T22:42:04.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:42:04.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241625+0000", 2024-09-17T22:42:04.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9629999999999998e-06 2024-09-17T22:42:04.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241628+0000", 2024-09-17T22:42:04.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4339999999999999e-06 2024-09-17T22:42:04.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281039+0000", 2024-09-17T22:42:04.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039410672000000001 2024-09-17T22:42:04.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281047+0000", 2024-09-17T22:42:04.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9969999999999998e-06 2024-09-17T22:42:04.695 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.695 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.695 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.696 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158682 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.696 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:04.696 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429388554999999, 2024-09-17T22:42:04.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039436455000000002, 2024-09-17T22:42:04.696 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.696 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.696 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.696 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.696 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.696 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.696 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158682 2024-09-17T22:42:04.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.697 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:04.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:04.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:04.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:42:04.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241632+0000", 2024-09-17T22:42:04.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:42:04.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241635+0000", 2024-09-17T22:42:04.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8940000000000002e-06 2024-09-17T22:42:04.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241637+0000", 2024-09-17T22:42:04.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3659999999999999e-06 2024-09-17T22:42:04.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281061+0000", 2024-09-17T22:42:04.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039423664999999997 2024-09-17T22:42:04.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281067+0000", 2024-09-17T22:42:04.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.285e-06 2024-09-17T22:42:04.700 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.700 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.700 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.700 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158684 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.700 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:04.700 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429379209, 2024-09-17T22:42:04.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462980000000002, 2024-09-17T22:42:04.700 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.700 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.700 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.700 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.701 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.701 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.701 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158684 2024-09-17T22:42:04.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.701 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:04.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:04.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:04.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:04.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241641+0000", 2024-09-17T22:42:04.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:42:04.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241644+0000", 2024-09-17T22:42:04.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8870000000000001e-06 2024-09-17T22:42:04.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241646+0000", 2024-09-17T22:42:04.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3530000000000002e-06 2024-09-17T22:42:04.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281093+0000", 2024-09-17T22:42:04.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039446947000000003 2024-09-17T22:42:04.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281103+0000", 2024-09-17T22:42:04.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6299999999999993e-06 2024-09-17T22:42:04.704 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.704 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.704 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.704 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158686 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.704 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:04.704 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429370039, 2024-09-17T22:42:04.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039474214000000001, 2024-09-17T22:42:04.704 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.704 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.704 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.705 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.705 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.705 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.705 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158686 2024-09-17T22:42:04.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.705 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:04.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:04.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:04.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:42:04.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241650+0000", 2024-09-17T22:42:04.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2999999999999999e-07 2024-09-17T22:42:04.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241653+0000", 2024-09-17T22:42:04.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9689999999999999e-06 2024-09-17T22:42:04.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241655+0000", 2024-09-17T22:42:04.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3460000000000001e-06 2024-09-17T22:42:04.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281118+0000", 2024-09-17T22:42:04.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462876000000001 2024-09-17T22:42:04.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281123+0000", 2024-09-17T22:42:04.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8840000000000002e-06 2024-09-17T22:42:04.708 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.708 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.708 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.708 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158688 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.708 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:04.708 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429360808, 2024-09-17T22:42:04.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039490458999999999, 2024-09-17T22:42:04.709 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.709 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.709 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.709 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.709 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.709 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.709 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158688 2024-09-17T22:42:04.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.709 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:04.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:04.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:04.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:42:04.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241659+0000", 2024-09-17T22:42:04.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3699999999999999e-07 2024-09-17T22:42:04.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241662+0000", 2024-09-17T22:42:04.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8499999999999998e-06 2024-09-17T22:42:04.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241665+0000", 2024-09-17T22:42:04.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.356e-06 2024-09-17T22:42:04.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281141+0000", 2024-09-17T22:42:04.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039476031000000002 2024-09-17T22:42:04.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281149+0000", 2024-09-17T22:42:04.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0709999999999995e-06 2024-09-17T22:42:04.712 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.712 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.712 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.713 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158690 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.713 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:04.713 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429351749, 2024-09-17T22:42:04.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039502378999999997, 2024-09-17T22:42:04.713 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.713 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.713 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.713 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.713 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.713 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.713 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158690 2024-09-17T22:42:04.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.713 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241668+0000", 2024-09-17T22:42:04.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241669+0000", 2024-09-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1009999999999999e-06 2024-09-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241672+0000", 2024-09-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9059999999999998e-06 2024-09-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241674+0000", 2024-09-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4439999999999998e-06 2024-09-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281162+0000", 2024-09-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039488189 2024-09-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281170+0000", 2024-09-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5310000000000004e-06 2024-09-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.716 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.717 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.717 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158692 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.717 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:04.717 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429342316, 2024-09-17T22:42:04.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039531064999999997, 2024-09-17T22:42:04.717 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.717 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.717 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.717 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.717 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.717 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.717 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158692 2024-09-17T22:42:04.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.718 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:04.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:04.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:04.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:04.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241678+0000", 2024-09-17T22:42:04.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0800000000000003e-07 2024-09-17T22:42:04.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241681+0000", 2024-09-17T22:42:04.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8729999999999999e-06 2024-09-17T22:42:04.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241683+0000", 2024-09-17T22:42:04.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.447e-06 2024-09-17T22:42:04.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281199+0000", 2024-09-17T22:42:04.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039515388999999998 2024-09-17T22:42:04.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281208+0000", 2024-09-17T22:42:04.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2399999999999996e-06 2024-09-17T22:42:04.721 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.721 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.721 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.721 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158694 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.721 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:04.721 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429333144999999, 2024-09-17T22:42:04.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039543986000000003, 2024-09-17T22:42:04.721 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.721 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.722 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.722 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.722 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.722 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.722 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158694 2024-09-17T22:42:04.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.722 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:04.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:04.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:04.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:42:04.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241687+0000", 2024-09-17T22:42:04.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:42:04.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241691+0000", 2024-09-17T22:42:04.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.811e-06 2024-09-17T22:42:04.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241693+0000", 2024-09-17T22:42:04.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.385e-06 2024-09-17T22:42:04.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281223+0000", 2024-09-17T22:42:04.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039529332 2024-09-17T22:42:04.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281230+0000", 2024-09-17T22:42:04.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2139999999999999e-06 2024-09-17T22:42:04.725 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.725 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.725 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.725 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158696 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.725 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:04.725 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429322757, 2024-09-17T22:42:04.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039570458000000003, 2024-09-17T22:42:04.726 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.726 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.726 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.726 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.726 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.726 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.726 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158696 2024-09-17T22:42:04.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.726 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:04.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:04.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:42:04.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:42:04.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:42:04.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.33e-07 2024-09-17T22:42:04.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241700+0000", 2024-09-17T22:42:04.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8949999999999998e-06 2024-09-17T22:42:04.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241703+0000", 2024-09-17T22:42:04.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3159999999999999e-06 2024-09-17T22:42:04.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281257+0000", 2024-09-17T22:42:04.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039553862000000002 2024-09-17T22:42:04.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281267+0000", 2024-09-17T22:42:04.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.024e-05 2024-09-17T22:42:04.729 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.729 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.729 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.730 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158698 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.730 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:04.730 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429312679000001, 2024-09-17T22:42:04.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039576493999999997, 2024-09-17T22:42:04.730 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.730 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.730 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.730 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.730 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.730 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.730 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158698 2024-09-17T22:42:04.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.730 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:04.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:04.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241707+0000", 2024-09-17T22:42:04.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:42:04.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241708+0000", 2024-09-17T22:42:04.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5000000000000001e-07 2024-09-17T22:42:04.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241711+0000", 2024-09-17T22:42:04.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9670000000000001e-06 2024-09-17T22:42:04.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241713+0000", 2024-09-17T22:42:04.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2689999999999998e-06 2024-09-17T22:42:04.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281278+0000", 2024-09-17T22:42:04.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039565206999999998 2024-09-17T22:42:04.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281283+0000", 2024-09-17T22:42:04.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:42:04.733 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.733 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.733 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.734 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158700 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.734 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:04.734 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429303468000001, 2024-09-17T22:42:04.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039604292999999999, 2024-09-17T22:42:04.734 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.734 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.734 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.734 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.734 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.734 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.734 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158700 2024-09-17T22:42:04.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.735 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:04.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:04.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:04.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:04.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241717+0000", 2024-09-17T22:42:04.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.78e-07 2024-09-17T22:42:04.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241720+0000", 2024-09-17T22:42:04.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8119999999999999e-06 2024-09-17T22:42:04.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241722+0000", 2024-09-17T22:42:04.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-17T22:42:04.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281311+0000", 2024-09-17T22:42:04.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039588894999999999 2024-09-17T22:42:04.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281320+0000", 2024-09-17T22:42:04.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9989999999999997e-06 2024-09-17T22:42:04.737 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.737 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.737 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.738 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158702 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.738 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:04.738 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429292146, 2024-09-17T22:42:04.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039616142, 2024-09-17T22:42:04.738 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.738 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.738 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.738 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.738 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.738 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.738 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158702 2024-09-17T22:42:04.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.739 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:04.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:04.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:04.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4400000000000001e-07 2024-09-17T22:42:04.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241728+0000", 2024-09-17T22:42:04.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0839999999999999e-06 2024-09-17T22:42:04.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241731+0000", 2024-09-17T22:42:04.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0690000000000002e-06 2024-09-17T22:42:04.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241734+0000", 2024-09-17T22:42:04.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4159999999999998e-06 2024-09-17T22:42:04.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281334+0000", 2024-09-17T22:42:04.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039600110000000001 2024-09-17T22:42:04.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281343+0000", 2024-09-17T22:42:04.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2189999999999993e-06 2024-09-17T22:42:04.741 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.742 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.742 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.742 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158704 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.742 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:04.742 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429282282000001, 2024-09-17T22:42:04.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039641485999999997, 2024-09-17T22:42:04.742 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.742 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.742 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.742 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.742 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.742 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.743 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158704 2024-09-17T22:42:04.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.743 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:04.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:04.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:04.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:42:04.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241738+0000", 2024-09-17T22:42:04.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0589999999999999e-06 2024-09-17T22:42:04.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241741+0000", 2024-09-17T22:42:04.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.943e-06 2024-09-17T22:42:04.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241744+0000", 2024-09-17T22:42:04.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4819999999999999e-06 2024-09-17T22:42:04.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281370+0000", 2024-09-17T22:42:04.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039625661 2024-09-17T22:42:04.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281378+0000", 2024-09-17T22:42:04.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1310000000000006e-06 2024-09-17T22:42:04.746 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.746 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.746 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.746 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158705 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.746 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:04.746 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429271732, 2024-09-17T22:42:04.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039652637999999997, 2024-09-17T22:42:04.746 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.746 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.746 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.746 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.747 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.747 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.747 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158705 2024-09-17T22:42:04.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.747 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:04.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:04.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:42:04.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-07 2024-09-17T22:42:04.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:42:04.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0599999999999999e-07 2024-09-17T22:42:04.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241751+0000", 2024-09-17T22:42:04.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.926e-06 2024-09-17T22:42:04.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241754+0000", 2024-09-17T22:42:04.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4360000000000001e-06 2024-09-17T22:42:04.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281392+0000", 2024-09-17T22:42:04.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039638566 2024-09-17T22:42:04.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281400+0000", 2024-09-17T22:42:04.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.571e-06 2024-09-17T22:42:04.750 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.750 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.750 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.750 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158707 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.750 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:04.750 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429250631, 2024-09-17T22:42:04.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039654809999999999, 2024-09-17T22:42:04.750 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.750 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.751 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.751 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.751 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.751 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.751 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158707 2024-09-17T22:42:04.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.751 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:04.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:04.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241769+0000", 2024-09-17T22:42:04.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4299999999999999e-07 2024-09-17T22:42:04.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241770+0000", 2024-09-17T22:42:04.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3769999999999999e-06 2024-09-17T22:42:04.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241773+0000", 2024-09-17T22:42:04.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2109999999999998e-06 2024-09-17T22:42:04.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241776+0000", 2024-09-17T22:42:04.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5069999999999999e-06 2024-09-17T22:42:04.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281415+0000", 2024-09-17T22:42:04.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039639499000000002 2024-09-17T22:42:04.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281423+0000", 2024-09-17T22:42:04.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8730000000000007e-06 2024-09-17T22:42:04.754 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.754 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.754 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.755 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158709 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:04.755 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:04.755 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.429240022, 2024-09-17T22:42:04.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039661937000000001, 2024-09-17T22:42:04.755 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:04.755 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:04.755 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:04.755 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:04.755 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:04.755 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:04.755 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158709 2024-09-17T22:42:04.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.755 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:04.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:04.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:04.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:04.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:04.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:04.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:04.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242843+0000", 2024-09-17T22:42:04.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010634780000000001 2024-09-17T22:42:04.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:04.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242845+0000", 2024-09-17T22:42:04.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7209999999999999e-06 2024-09-17T22:42:04.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:04.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242851+0000", 2024-09-17T22:42:04.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6949999999999997e-06 2024-09-17T22:42:04.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:04.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242855+0000", 2024-09-17T22:42:04.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7960000000000002e-06 2024-09-17T22:42:04.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:04.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281434+0000", 2024-09-17T22:42:04.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038579519 2024-09-17T22:42:04.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:04.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:04.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:04.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281441+0000", 2024-09-17T22:42:04.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7279999999999998e-06 2024-09-17T22:42:04.758 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.759 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.759 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.759 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:04.759 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:04.759 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:04.760 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:04.760 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:04.761 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:04.761 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:04.757+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:04.761 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:42:04.775 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:04.776 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:04.791 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:04.791 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:04.861 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:04.858+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:04.865 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:04.865 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:04.865 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:04.865 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:04.874 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:42:04.961 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:04.959+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:05.009 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:05.009 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:05.019 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:05.019 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:05.019 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:05.019 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:05.019 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:05.025 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:05.025 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:05.028 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:42:05.041 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:05.041 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:05.057 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:05.058 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:05.061 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:05.061 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:05.102 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:05.100+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:05.131 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:05.132 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:05.140 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:42:05.184 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:05.182+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:05.248 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:05.248 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:05.252 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone21 from 0x995c215 (overlap 0x15e7866) to 0x8c3028 2024-09-17T22:42:05.257 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:42:05.298 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:05.296+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:05.364 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-17T22:42:05.364 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:05.362+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:05.403 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:05.403 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:05.414 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:42:05.465 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:05.463+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:05.517 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:05.517 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:05.525 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:42:05.557 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:05.558 INFO:teuthology.orchestra.run.smithi115.stdout:2286 trunc from 0x995c215 to 0xa3315b6 2024-09-17T22:42:05.568 INFO:teuthology.orchestra.run.smithi115.stdout:2287 clone 23 order 24 su 2097152 sc 4 2024-09-17T22:42:05.569 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:05.567+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:05.570 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:42:05.571 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:42:05.571 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:42:05.629 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:05.630 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:05.638 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:42:05.666 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:05.664+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:05.767 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:05.765+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:05.784 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:05.784 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:05.793 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:42:05.847 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:42:05.848 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:42:05.859 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-09-17T22:42:05.164811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":577775913,"num_objects":1136,"num_object_clones":278,"num_object_copies":2272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":292,"num_objects_misplaced":101,"num_objects_unfound":0,"num_objects_dirty":1136,"num_whiteouts":616,"num_read":197053,"num_read_kb":482698,"num_write":6056,"num_write_kb":107591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3161143,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5448,"ondisk_log_size":5448,"up":69,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282000,"kb_used_data":24184,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754692720,"statfs":{"total":773094113280,"available":772805345280,"internally_reserved":0,"allocated":24764416,"data_stored":149382387,"data_compressed":4399987,"data_compressed_allocated":14184448,"data_compressed_original":144234646,"omap_allocated":12750,"internal_metadata":263835186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34004417,"num_objects":51,"num_object_clones":14,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":-32,"num_objects_degraded":13,"num_objects_misplaced":55,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":14,"num_read":2255,"num_read_kb":20505,"num_write":551,"num_write_kb":7011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3161143,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.019149"},"pg_stats":[{"pgid":"3.1f","version":"57'271","reported_seq":6145,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T22:42:01.207875+0000","last_change":"2024-09-17T22:41:21.453392+0000","last_active":"2024-09-17T22:42:01.207875+0000","last_peered":"2024-09-17T22:42:01.207875+0000","last_clean":"2024-09-17T22:42:01.207875+0000","last_became_active":"2024-09-17T22:41:19.443658+0000","last_became_peered":"2024-09-17T22:41:19.443658+0000","last_unstale":"2024-09-17T22:42:01.207875+0000","last_undegraded":"2024-09-17T22:42:01.207875+0000","last_fullsized":"2024-09-17T22:42:01.207875+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:57:37.102656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047335399999999999,"stat_sum":{"num_bytes":19906717,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":13,"num_read":6717,"num_read_kb":12001,"num_write":319,"num_write_kb":2227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"58'100","reported_seq":5548,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T22:42:01.214586+0000","last_change":"2024-09-17T22:41:21.453587+0000","last_active":"2024-09-17T22:42:01.214586+0000","last_peered":"2024-09-17T22:42:01.214586+0000","last_clean":"2024-09-17T22:42:01.214586+0000","last_became_active":"2024-09-17T22:41:19.449357+0000","last_became_peered":"2024-09-17T22:41:19.449357+0000","last_unstale":"2024-09-17T22:42:01.214586+0000","last_undegraded":"2024-09-17T22:42:01.214586+0000","last_fullsized":"2024-09-17T22:42:01.214586+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:51:59.977299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996099999999995,"stat_sum":{"num_bytes":13061601,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":21,"num_read":5423,"num_read_kb":10424,"num_write":83,"num_write_kb":3701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"58'69","reported_seq":111,"reported_epoch":58,"state":"active+clean+remapped","last_fresh":"2024-09-17T22:42:01.246257+0000","last_change":"2024-09-17T22:41:57.183293+0000","last_active":"2024-09-17T22:42:01.246257+0000","last_peered":"2024-09-17T22:42:01.246257+0000","last_clean":"2024-09-17T22:42:01.246257+0000","last_became_active":"2024-09-17T22:41:57.133365+0000","last_became_peered":"2024-09-17T22:41:57.133365+0000","last_unstale":"2024-09-17T22:42:01.246257+0000","last_undegraded":"2024-09-17T22:42:01.246257+0000","last_fullsized":"2024-09-17T22:42:01.246257+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:07:57.790999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000382498,"stat_sum":{"num_bytes":26956791,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":26,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":13,"num_read":13396,"num_read_kb":39847,"num_write":57,"num_write_kb":3113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3161143,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[{"shards":"3,7","objects":26}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"59'104","reported_seq":973,"reported_epoch":59,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:02.278769+0000","last_change":"2024-09-17T22:41:54.113405+0000","last_active":"2024-09-17T22:42:02.278769+0000","last_peered":"2024-09-17T22:42:02.278769+0000","last_clean":"2024-09-17T22:41:52.379845+0000","last_became_active":"2024-09-17T22:41:54.113405+0000","last_became_peered":"2024-09-17T22:41:54.113405+0000","last_unstale":"2024-09-17T22:42:02.278769+0000","last_undegraded":"2024-09-17T22:41:54.112692+0000","last_fullsized":"2024-09-17T22:41:54.112591+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:22:02.098810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001451078,"stat_sum":{"num_bytes":8912322,"num_objects":31,"num_object_clones":11,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":16,"num_read":772,"num_read_kb":2919,"num_write":92,"num_write_kb":4270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":31}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"58'92","reported_seq":7021,"reported_epoch":58,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:01.210480+0000","last_change":"2024-09-17T22:41:54.118027+0000","last_active":"2024-09-17T22:42:01.210480+0000","last_peered":"2024-09-17T22:42:01.210480+0000","last_clean":"2024-09-17T22:41:52.368521+0000","last_became_active":"2024-09-17T22:41:54.118027+0000","last_became_peered":"2024-09-17T22:41:54.118027+0000","last_unstale":"2024-09-17T22:42:01.210480+0000","last_undegraded":"2024-09-17T22:41:54.116518+0000","last_fullsized":"2024-09-17T22:41:54.116411+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:04:34.487427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055813,"stat_sum":{"num_bytes":23483388,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":17,"num_read":6867,"num_read_kb":15974,"num_write":76,"num_write_kb":1546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":33}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"59'142","reported_seq":1608,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T22:42:02.309414+0000","last_change":"2024-09-17T22:41:21.446945+0000","last_active":"2024-09-17T22:42:02.309414+0000","last_peered":"2024-09-17T22:42:02.309414+0000","last_clean":"2024-09-17T22:42:02.309414+0000","last_became_active":"2024-09-17T22:41:19.446541+0000","last_became_peered":"2024-09-17T22:41:19.446541+0000","last_unstale":"2024-09-17T22:42:02.309414+0000","last_undegraded":"2024-09-17T22:42:02.309414+0000","last_fullsized":"2024-09-17T22:42:02.309414+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:10:34.220991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001630631,"stat_sum":{"num_bytes":24575507,"num_objects":42,"num_object_clones":9,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":24,"num_read":1766,"num_read_kb":10220,"num_write":127,"num_write_kb":3196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"57'134","reported_seq":5548,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T22:42:01.307053+0000","last_change":"2024-09-17T22:41:21.448965+0000","last_active":"2024-09-17T22:42:01.307053+0000","last_peered":"2024-09-17T22:42:01.307053+0000","last_clean":"2024-09-17T22:42:01.307053+0000","last_became_active":"2024-09-17T22:41:19.449940+0000","last_became_peered":"2024-09-17T22:41:19.449940+0000","last_unstale":"2024-09-17T22:42:01.307053+0000","last_undegraded":"2024-09-17T22:42:01.307053+0000","last_fullsized":"2024-09-17T22:42:01.307053+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:42:42.724066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011385119999999999,"stat_sum":{"num_bytes":22335159,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":5452,"num_read_kb":18151,"num_write":125,"num_write_kb":7372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"57'100","reported_seq":2015,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T22:42:01.307138+0000","last_change":"2024-09-17T22:41:21.453701+0000","last_active":"2024-09-17T22:42:01.307138+0000","last_peered":"2024-09-17T22:42:01.307138+0000","last_clean":"2024-09-17T22:42:01.307138+0000","last_became_active":"2024-09-17T22:41:19.446287+0000","last_became_peered":"2024-09-17T22:41:19.446287+0000","last_unstale":"2024-09-17T22:42:01.307138+0000","last_undegraded":"2024-09-17T22:42:01.307138+0000","last_fullsized":"2024-09-17T22:42:01.307138+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:24:54.435548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067184299999999998,"stat_sum":{"num_bytes":11067482,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":17,"num_read":2311,"num_read_kb":13174,"num_write":84,"num_write_kb":5106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"57'155","reported_seq":1049,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T22:42:01.207961+0000","last_change":"2024-09-17T22:41:21.453344+0000","last_active":"2024-09-17T22:42:01.207961+0000","last_peered":"2024-09-17T22:42:01.207961+0000","last_clean":"2024-09-17T22:42:01.207961+0000","last_became_active":"2024-09-17T22:41:19.445923+0000","last_became_peered":"2024-09-17T22:41:19.445923+0000","last_unstale":"2024-09-17T22:42:01.207961+0000","last_undegraded":"2024-09-17T22:42:01.207961+0000","last_fullsized":"2024-09-17T22:42:01.207961+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:50:53.526358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043560199999999999,"stat_sum":{"num_bytes":6632089,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":21,"num_read":1164,"num_read_kb":5712,"num_write":186,"num_write_kb":3355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"57'175","reported_seq":21233,"reported_epoch":57,"state":"active+clean+remapped","last_fresh":"2024-09-17T22:42:00.017937+0000","last_change":"2024-09-17T22:41:57.132629+0000","last_active":"2024-09-17T22:42:00.017937+0000","last_peered":"2024-09-17T22:42:00.017937+0000","last_clean":"2024-09-17T22:42:00.017937+0000","last_became_active":"2024-09-17T22:41:57.132018+0000","last_became_peered":"2024-09-17T22:41:57.132018+0000","last_unstale":"2024-09-17T22:42:00.017937+0000","last_undegraded":"2024-09-17T22:42:00.017937+0000","last_fullsized":"2024-09-17T22:42:00.017937+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:25:07.721143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000275164,"stat_sum":{"num_bytes":29001121,"num_objects":46,"num_object_clones":16,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":46,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":24,"num_read":21544,"num_read_kb":58777,"num_write":158,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":46}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"57'132","reported_seq":5676,"reported_epoch":58,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:01.148869+0000","last_change":"2024-09-17T22:41:54.119655+0000","last_active":"2024-09-17T22:42:01.148869+0000","last_peered":"2024-09-17T22:42:01.148869+0000","last_clean":"2024-09-17T22:41:52.245886+0000","last_became_active":"2024-09-17T22:41:54.119655+0000","last_became_peered":"2024-09-17T22:41:54.119655+0000","last_unstale":"2024-09-17T22:42:01.148869+0000","last_undegraded":"2024-09-17T22:41:54.119161+0000","last_fullsized":"2024-09-17T22:41:54.119071+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:53:23.460736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041055799999999999,"stat_sum":{"num_bytes":24482622,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":19,"num_read":5793,"num_read_kb":10454,"num_write":87,"num_write_kb":656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":28}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"57'366","reported_seq":6931,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T22:42:00.153519+0000","last_change":"2024-09-17T22:41:21.446210+0000","last_active":"2024-09-17T22:42:00.153519+0000","last_peered":"2024-09-17T22:42:00.153519+0000","last_clean":"2024-09-17T22:42:00.153519+0000","last_became_active":"2024-09-17T22:41:19.451631+0000","last_became_peered":"2024-09-17T22:41:19.451631+0000","last_unstale":"2024-09-17T22:42:00.153519+0000","last_undegraded":"2024-09-17T22:42:00.153519+0000","last_fullsized":"2024-09-17T22:42:00.153519+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:18.174821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081193699999999999,"stat_sum":{"num_bytes":20226292,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":7921,"num_read_kb":13099,"num_write":556,"num_write_kb":1439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":93,"reported_epoch":57,"state":"active+undersized","last_fresh":"2024-09-17T22:41:59.137467+0000","last_change":"2024-09-17T22:41:54.118987+0000","last_active":"2024-09-17T22:41:59.137467+0000","last_peered":"2024-09-17T22:41:59.137467+0000","last_clean":"2024-09-17T22:41:52.098057+0000","last_became_active":"2024-09-17T22:41:54.118987+0000","last_became_peered":"2024-09-17T22:41:54.118987+0000","last_unstale":"2024-09-17T22:41:59.137467+0000","last_undegraded":"2024-09-17T22:41:59.137467+0000","last_fullsized":"2024-09-17T22:41:54.114557+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.941682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"58'171","reported_seq":1057,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T22:42:01.210494+0000","last_change":"2024-09-17T22:41:21.448698+0000","last_active":"2024-09-17T22:42:01.210494+0000","last_peered":"2024-09-17T22:42:01.210494+0000","last_clean":"2024-09-17T22:42:01.210494+0000","last_became_active":"2024-09-17T22:41:19.448966+0000","last_became_peered":"2024-09-17T22:41:19.448966+0000","last_unstale":"2024-09-17T22:42:01.210494+0000","last_undegraded":"2024-09-17T22:42:01.210494+0000","last_fullsized":"2024-09-17T22:42:01.210494+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:14:17.624035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089570000000000003,"stat_sum":{"num_bytes":7095948,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":19,"num_read":1220,"num_read_kb":3429,"num_write":211,"num_write_kb":978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":95,"reported_epoch":58,"state":"active+undersized","last_fresh":"2024-09-17T22:42:01.151599+0000","last_change":"2024-09-17T22:41:54.118378+0000","last_active":"2024-09-17T22:42:01.151599+0000","last_peered":"2024-09-17T22:42:01.151599+0000","last_clean":"2024-09-17T22:41:52.097696+0000","last_became_active":"2024-09-17T22:41:54.118378+0000","last_became_peered":"2024-09-17T22:41:54.118378+0000","last_unstale":"2024-09-17T22:42:01.151599+0000","last_undegraded":"2024-09-17T22:42:01.151599+0000","last_fullsized":"2024-09-17T22:41:54.116433+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:48:40.138581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"57'181","reported_seq":753,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T22:42:01.151668+0000","last_change":"2024-09-17T22:41:21.448596+0000","last_active":"2024-09-17T22:42:01.151668+0000","last_peered":"2024-09-17T22:42:01.151668+0000","last_clean":"2024-09-17T22:42:01.151668+0000","last_became_active":"2024-09-17T22:41:19.451637+0000","last_became_peered":"2024-09-17T22:41:19.451637+0000","last_unstale":"2024-09-17T22:42:01.151668+0000","last_undegraded":"2024-09-17T22:42:01.151668+0000","last_fullsized":"2024-09-17T22:42:01.151668+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:44:24.123002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085737499999999996,"stat_sum":{"num_bytes":17977041,"num_objects":41,"num_object_clones":13,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":23,"num_read":988,"num_read_kb":2831,"num_write":226,"num_write_kb":1803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":91,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T22:41:59.137849+0000","last_change":"2024-09-17T22:41:11.397753+0000","last_active":"2024-09-17T22:41:59.137849+0000","last_peered":"2024-09-17T22:41:59.137849+0000","last_clean":"2024-09-17T22:41:59.137849+0000","last_became_active":"2024-09-17T22:41:09.387651+0000","last_became_peered":"2024-09-17T22:41:09.387651+0000","last_unstale":"2024-09-17T22:41:59.137849+0000","last_undegraded":"2024-09-17T22:41:59.137849+0000","last_fullsized":"2024-09-17T22:41:59.137849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:14:30.915047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"58'88","reported_seq":540,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T22:42:01.148782+0000","last_change":"2024-09-17T22:41:21.466363+0000","last_active":"2024-09-17T22:42:01.148782+0000","last_peered":"2024-09-17T22:42:01.148782+0000","last_clean":"2024-09-17T22:42:01.148782+0000","last_became_active":"2024-09-17T22:41:19.444893+0000","last_became_peered":"2024-09-17T22:41:19.444893+0000","last_unstale":"2024-09-17T22:42:01.148782+0000","last_undegraded":"2024-09-17T22:42:01.148782+0000","last_fullsized":"2024-09-17T22:42:01.148782+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:22:32.930871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053617200000000002,"stat_sum":{"num_bytes":11927315,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":15,"num_read":383,"num_read_kb":2049,"num_write":84,"num_write_kb":1614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":93,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T22:42:01.208166+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:42:01.208166+0000","last_peered":"2024-09-17T22:42:01.208166+0000","last_clean":"2024-09-17T22:42:01.208166+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:42:01.208166+0000","last_undegraded":"2024-09-17T22:42:01.208166+0000","last_fullsized":"2024-09-17T22:42:01.208166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"57'182","reported_seq":14831,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T22:41:59.900948+0000","last_change":"2024-09-17T22:41:21.516349+0000","last_active":"2024-09-17T22:41:59.900948+0000","last_peered":"2024-09-17T22:41:59.900948+0000","last_clean":"2024-09-17T22:41:59.900948+0000","last_became_active":"2024-09-17T22:41:19.451723+0000","last_became_peered":"2024-09-17T22:41:19.451723+0000","last_unstale":"2024-09-17T22:41:59.900948+0000","last_undegraded":"2024-09-17T22:41:59.900948+0000","last_fullsized":"2024-09-17T22:41:59.900948+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:41:31.105146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040033199999999999,"stat_sum":{"num_bytes":37794060,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":15078,"num_read_kb":27556,"num_write":243,"num_write_kb":3253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"57'248","reported_seq":9705,"reported_epoch":57,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:00.195669+0000","last_change":"2024-09-17T22:41:54.119266+0000","last_active":"2024-09-17T22:42:00.195669+0000","last_peered":"2024-09-17T22:42:00.195669+0000","last_clean":"2024-09-17T22:41:52.262287+0000","last_became_active":"2024-09-17T22:41:54.119266+0000","last_became_peered":"2024-09-17T22:41:54.119266+0000","last_unstale":"2024-09-17T22:42:00.195669+0000","last_undegraded":"2024-09-17T22:41:54.116326+0000","last_fullsized":"2024-09-17T22:41:54.116242+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:31:50.721246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063898899999999996,"stat_sum":{"num_bytes":29165323,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":31,"num_read":11478,"num_read_kb":22102,"num_write":209,"num_write_kb":1785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":52}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":93,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T22:41:59.137064+0000","last_change":"2024-09-17T22:41:11.397488+0000","last_active":"2024-09-17T22:41:59.137064+0000","last_peered":"2024-09-17T22:41:59.137064+0000","last_clean":"2024-09-17T22:41:59.137064+0000","last_became_active":"2024-09-17T22:41:09.651390+0000","last_became_peered":"2024-09-17T22:41:09.651390+0000","last_unstale":"2024-09-17T22:41:59.137064+0000","last_undegraded":"2024-09-17T22:41:59.137064+0000","last_fullsized":"2024-09-17T22:41:59.137064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:00.598921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"58'161","reported_seq":3109,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T22:42:01.222643+0000","last_change":"2024-09-17T22:41:21.453941+0000","last_active":"2024-09-17T22:42:01.222643+0000","last_peered":"2024-09-17T22:42:01.222643+0000","last_clean":"2024-09-17T22:42:01.222643+0000","last_became_active":"2024-09-17T22:41:19.444314+0000","last_became_peered":"2024-09-17T22:41:19.444314+0000","last_unstale":"2024-09-17T22:42:01.222643+0000","last_undegraded":"2024-09-17T22:42:01.222643+0000","last_fullsized":"2024-09-17T22:42:01.222643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:43:49.966830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085427099999999998,"stat_sum":{"num_bytes":29345411,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":18,"num_read":3395,"num_read_kb":14193,"num_write":163,"num_write_kb":5736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":93,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T22:42:01.184054+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:42:01.184054+0000","last_peered":"2024-09-17T22:42:01.184054+0000","last_clean":"2024-09-17T22:42:01.184054+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:42:01.184054+0000","last_undegraded":"2024-09-17T22:42:01.184054+0000","last_fullsized":"2024-09-17T22:42:01.184054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"58'128","reported_seq":6085,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T22:42:01.221597+0000","last_change":"2024-09-17T22:41:21.466094+0000","last_active":"2024-09-17T22:42:01.221597+0000","last_peered":"2024-09-17T22:42:01.221597+0000","last_clean":"2024-09-17T22:42:01.221597+0000","last_became_active":"2024-09-17T22:41:19.449791+0000","last_became_peered":"2024-09-17T22:41:19.449791+0000","last_unstale":"2024-09-17T22:42:01.221597+0000","last_undegraded":"2024-09-17T22:42:01.221597+0000","last_fullsized":"2024-09-17T22:42:01.221597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:24:08.486750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042775300000000001,"stat_sum":{"num_bytes":11841751,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":17,"num_read":6118,"num_read_kb":4425,"num_write":152,"num_write_kb":1431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":91,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T22:41:59.136926+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:41:59.136926+0000","last_peered":"2024-09-17T22:41:59.136926+0000","last_clean":"2024-09-17T22:41:59.136926+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:41:59.136926+0000","last_undegraded":"2024-09-17T22:41:59.136926+0000","last_fullsized":"2024-09-17T22:41:59.136926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"58'305","reported_seq":6341,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T22:42:01.252087+0000","last_change":"2024-09-17T22:41:21.448409+0000","last_active":"2024-09-17T22:42:01.252087+0000","last_peered":"2024-09-17T22:42:01.252087+0000","last_clean":"2024-09-17T22:42:01.252087+0000","last_became_active":"2024-09-17T22:41:19.449279+0000","last_became_peered":"2024-09-17T22:41:19.449279+0000","last_unstale":"2024-09-17T22:42:01.252087+0000","last_undegraded":"2024-09-17T22:42:01.252087+0000","last_fullsized":"2024-09-17T22:42:01.252087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:41:32.488975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075217299999999997,"stat_sum":{"num_bytes":17745080,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":23,"num_read":7277,"num_read_kb":13077,"num_write":390,"num_write_kb":1314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T22:41:59.137209+0000","last_change":"2024-09-17T22:41:11.398108+0000","last_active":"2024-09-17T22:41:59.137209+0000","last_peered":"2024-09-17T22:41:59.137209+0000","last_clean":"2024-09-17T22:41:59.137209+0000","last_became_active":"2024-09-17T22:41:09.388184+0000","last_became_peered":"2024-09-17T22:41:09.388184+0000","last_unstale":"2024-09-17T22:41:59.137209+0000","last_undegraded":"2024-09-17T22:41:59.137209+0000","last_fullsized":"2024-09-17T22:41:59.137209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:43.506329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":246,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T22:42:02.161763+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:42:02.161763+0000","last_peered":"2024-09-17T22:42:02.161763+0000","last_clean":"2024-09-17T22:42:02.161763+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:42:02.161763+0000","last_undegraded":"2024-09-17T22:42:02.161763+0000","last_fullsized":"2024-09-17T22:42:02.161763+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"58'112","reported_seq":6800,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T22:42:01.240119+0000","last_change":"2024-09-17T22:41:21.448156+0000","last_active":"2024-09-17T22:42:01.240119+0000","last_peered":"2024-09-17T22:42:01.240119+0000","last_clean":"2024-09-17T22:42:01.240119+0000","last_became_active":"2024-09-17T22:41:19.448956+0000","last_became_peered":"2024-09-17T22:41:19.448956+0000","last_unstale":"2024-09-17T22:42:01.240119+0000","last_undegraded":"2024-09-17T22:42:01.240119+0000","last_fullsized":"2024-09-17T22:42:01.240119+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:39:23.496296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055851200000000005,"stat_sum":{"num_bytes":18366896,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":21,"num_read":6725,"num_read_kb":18611,"num_write":108,"num_write_kb":3315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"57'410","reported_seq":13439,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T22:42:00.157611+0000","last_change":"2024-09-17T22:41:21.445421+0000","last_active":"2024-09-17T22:42:00.157611+0000","last_peered":"2024-09-17T22:42:00.157611+0000","last_clean":"2024-09-17T22:42:00.157611+0000","last_became_active":"2024-09-17T22:41:19.449003+0000","last_became_peered":"2024-09-17T22:41:19.449003+0000","last_unstale":"2024-09-17T22:42:00.157611+0000","last_undegraded":"2024-09-17T22:42:00.157611+0000","last_fullsized":"2024-09-17T22:42:00.157611+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:39:22.722645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051465400000000002,"stat_sum":{"num_bytes":12373298,"num_objects":38,"num_object_clones":11,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":18,"num_read":14601,"num_read_kb":34935,"num_write":641,"num_write_kb":4725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"57'223","reported_seq":6759,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T22:41:59.693991+0000","last_change":"2024-09-17T22:41:21.445728+0000","last_active":"2024-09-17T22:41:59.693991+0000","last_peered":"2024-09-17T22:41:59.693991+0000","last_clean":"2024-09-17T22:41:59.693991+0000","last_became_active":"2024-09-17T22:41:19.448011+0000","last_became_peered":"2024-09-17T22:41:19.448011+0000","last_unstale":"2024-09-17T22:41:59.693991+0000","last_undegraded":"2024-09-17T22:41:59.693991+0000","last_fullsized":"2024-09-17T22:41:59.693991+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:19:54.292866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068494200000000002,"stat_sum":{"num_bytes":25883250,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":21,"num_read":6916,"num_read_kb":21702,"num_write":259,"num_write_kb":5317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"57'81","reported_seq":688,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-17T22:42:01.307311+0000","last_change":"2024-09-17T22:41:21.462738+0000","last_active":"2024-09-17T22:42:01.307311+0000","last_peered":"2024-09-17T22:42:01.307311+0000","last_clean":"2024-09-17T22:42:01.307311+0000","last_became_active":"2024-09-17T22:41:19.447215+0000","last_became_peered":"2024-09-17T22:41:19.447215+0000","last_unstale":"2024-09-17T22:42:01.307311+0000","last_undegraded":"2024-09-17T22:42:01.307311+0000","last_fullsized":"2024-09-17T22:42:01.307311+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:34:54.046588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042703000000000001,"stat_sum":{"num_bytes":8724364,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":18,"num_read":548,"num_read_kb":6313,"num_write":73,"num_write_kb":2356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"58'143","reported_seq":20670,"reported_epoch":58,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:01.210497+0000","last_change":"2024-09-17T22:41:54.119555+0000","last_active":"2024-09-17T22:42:01.210497+0000","last_peered":"2024-09-17T22:42:01.210497+0000","last_clean":"2024-09-17T22:41:52.249181+0000","last_became_active":"2024-09-17T22:41:54.119555+0000","last_became_peered":"2024-09-17T22:41:54.119555+0000","last_unstale":"2024-09-17T22:42:01.210497+0000","last_undegraded":"2024-09-17T22:41:54.118897+0000","last_fullsized":"2024-09-17T22:41:54.118785+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:05:50.162585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000395712,"stat_sum":{"num_bytes":59002842,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":19,"num_read":21028,"num_read_kb":56208,"num_write":127,"num_write_kb":10729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":36}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"57'94","reported_seq":926,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T22:42:00.235551+0000","last_change":"2024-09-17T22:41:21.446562+0000","last_active":"2024-09-17T22:42:00.235551+0000","last_peered":"2024-09-17T22:42:00.235551+0000","last_clean":"2024-09-17T22:42:00.235551+0000","last_became_active":"2024-09-17T22:41:19.447631+0000","last_became_peered":"2024-09-17T22:41:19.447631+0000","last_unstale":"2024-09-17T22:42:00.235551+0000","last_undegraded":"2024-09-17T22:42:00.235551+0000","last_fullsized":"2024-09-17T22:42:00.235551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:28:25.428592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000402657,"stat_sum":{"num_bytes":2214106,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":17,"num_read":771,"num_read_kb":1272,"num_write":80,"num_write_kb":1095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"57'135","reported_seq":7667,"reported_epoch":57,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:41:59.142025+0000","last_change":"2024-09-17T22:41:54.119016+0000","last_active":"2024-09-17T22:41:59.142025+0000","last_peered":"2024-09-17T22:41:59.142025+0000","last_clean":"2024-09-17T22:41:52.398679+0000","last_became_active":"2024-09-17T22:41:54.119016+0000","last_became_peered":"2024-09-17T22:41:54.119016+0000","last_unstale":"2024-09-17T22:41:59.142025+0000","last_undegraded":"2024-09-17T22:41:54.116346+0000","last_fullsized":"2024-09-17T22:41:54.116297+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:11:47.839260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038538199999999998,"stat_sum":{"num_bytes":18326562,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":23,"num_read":8054,"num_read_kb":18322,"num_write":99,"num_write_kb":7501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":38}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"58'114","reported_seq":1010,"reported_epoch":59,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:02.161944+0000","last_change":"2024-09-17T22:41:54.113116+0000","last_active":"2024-09-17T22:42:02.161944+0000","last_peered":"2024-09-17T22:42:02.161944+0000","last_clean":"2024-09-17T22:41:52.346319+0000","last_became_active":"2024-09-17T22:41:54.113116+0000","last_became_peered":"2024-09-17T22:41:54.113116+0000","last_unstale":"2024-09-17T22:42:02.161944+0000","last_undegraded":"2024-09-17T22:41:54.112692+0000","last_fullsized":"2024-09-17T22:41:54.112614+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:13.462880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000602379,"stat_sum":{"num_bytes":3642507,"num_objects":41,"num_object_clones":9,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":24,"num_read":856,"num_read_kb":2844,"num_write":103,"num_write_kb":1713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":41}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"58'191","reported_seq":1328,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-17T22:42:02.161849+0000","last_change":"2024-09-17T22:41:21.446576+0000","last_active":"2024-09-17T22:42:02.161849+0000","last_peered":"2024-09-17T22:42:02.161849+0000","last_clean":"2024-09-17T22:42:02.161849+0000","last_became_active":"2024-09-17T22:41:19.447419+0000","last_became_peered":"2024-09-17T22:41:19.447419+0000","last_unstale":"2024-09-17T22:42:02.161849+0000","last_undegraded":"2024-09-17T22:42:02.161849+0000","last_fullsized":"2024-09-17T22:42:02.161849+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:08:17.653266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001865071,"stat_sum":{"num_bytes":8104380,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":18,"num_read":2505,"num_read_kb":6201,"num_write":170,"num_write_kb":2507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"55'123","reported_seq":1213,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-17T22:41:59.137014+0000","last_change":"2024-09-17T22:41:21.445226+0000","last_active":"2024-09-17T22:41:59.137014+0000","last_peered":"2024-09-17T22:41:59.137014+0000","last_clean":"2024-09-17T22:41:59.137014+0000","last_became_active":"2024-09-17T22:41:19.454151+0000","last_became_peered":"2024-09-17T22:41:19.454151+0000","last_unstale":"2024-09-17T22:41:59.137014+0000","last_undegraded":"2024-09-17T22:41:59.137014+0000","last_fullsized":"2024-09-17T22:41:59.137014+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:31:02.481532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050545600000000005,"stat_sum":{"num_bytes":9251081,"num_objects":41,"num_object_clones":14,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":25,"num_read":1165,"num_read_kb":1629,"num_write":120,"num_write_kb":1029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"55'140","reported_seq":5906,"reported_epoch":58,"state":"active+clean+remapped","last_fresh":"2024-09-17T22:42:01.148821+0000","last_change":"2024-09-17T22:41:57.132756+0000","last_active":"2024-09-17T22:42:01.148821+0000","last_peered":"2024-09-17T22:42:01.148821+0000","last_clean":"2024-09-17T22:42:01.148821+0000","last_became_active":"2024-09-17T22:41:57.131666+0000","last_became_peered":"2024-09-17T22:41:57.131666+0000","last_unstale":"2024-09-17T22:42:01.148821+0000","last_undegraded":"2024-09-17T22:42:01.148821+0000","last_fullsized":"2024-09-17T22:42:01.148821+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:53:07.851585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048487299999999998,"stat_sum":{"num_bytes":6690180,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":29,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":16,"num_read":6455,"num_read_kb":8919,"num_write":124,"num_write_kb":769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6,3],"avail_no_missing":["6","3"],"object_location_counts":[{"shards":"3,6","objects":29}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"57'258","reported_seq":1733,"reported_epoch":57,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:41:59.929715+0000","last_change":"2024-09-17T22:41:54.118972+0000","last_active":"2024-09-17T22:41:59.929715+0000","last_peered":"2024-09-17T22:41:59.929715+0000","last_clean":"2024-09-17T22:41:52.267782+0000","last_became_active":"2024-09-17T22:41:54.118972+0000","last_became_peered":"2024-09-17T22:41:54.118972+0000","last_unstale":"2024-09-17T22:41:59.929715+0000","last_undegraded":"2024-09-17T22:41:54.114427+0000","last_fullsized":"2024-09-17T22:41:54.114366+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:49:17.058493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062146200000000003,"stat_sum":{"num_bytes":11073040,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":21,"num_read":2250,"num_read_kb":5228,"num_write":363,"num_write_kb":1659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":33}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":577185526,"num_objects":1132,"num_object_clones":278,"num_object_copies":2264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":292,"num_objects_misplaced":101,"num_objects_unfound":0,"num_objects_dirty":1132,"num_whiteouts":616,"num_read":196937,"num_read_kb":482598,"num_write":5895,"num_write_kb":105173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3161143,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19574784,"data_stored":146223833,"data_compressed":4421774,"data_compressed_allocated":14168064,"data_compressed_original":143370390,"omap_allocated":0,"internal_metadata":0},"log_size":5328,"ondisk_log_size":5328,"up":53,"acting":56,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":14,"acting":14,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33960,"kb_used_data":2776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337880,"statfs":{"total":96636764160,"available":96601989120,"internally_reserved":0,"allocated":2842624,"data_stored":17255544,"data_compressed":459250,"data_compressed_allocated":1564672,"data_compressed_original":16639785,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542156,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34156,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337684,"statfs":{"total":96636764160,"available":96601788416,"internally_reserved":0,"allocated":3043328,"data_stored":23926169,"data_compressed":437861,"data_compressed_allocated":1896448,"data_compressed_original":23315877,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34316,"kb_used_data":3128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337524,"statfs":{"total":96636764160,"available":96601624576,"internally_reserved":0,"allocated":3203072,"data_stored":22482729,"data_compressed":565957,"data_compressed_allocated":2035712,"data_compressed_original":21932842,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33776,"kb_used_data":2584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338064,"statfs":{"total":96636764160,"available":96602177536,"internally_reserved":0,"allocated":2646016,"data_stored":14432377,"data_compressed":456987,"data_compressed_allocated":1433600,"data_compressed_original":13850284,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34608,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337232,"statfs":{"total":96636764160,"available":96601325568,"internally_reserved":0,"allocated":3502080,"data_stored":22922070,"data_compressed":673757,"data_compressed_allocated":2179072,"data_compressed_original":22290460,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542158,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33316,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338524,"statfs":{"total":96636764160,"available":96602648576,"internally_reserved":0,"allocated":2183168,"data_stored":8240515,"data_compressed":298288,"data_compressed_allocated":909312,"data_compressed_original":7638881,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39132,"kb_used_data":3780,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332708,"statfs":{"total":96636764160,"available":96596692992,"internally_reserved":0,"allocated":3870720,"data_stored":21063482,"data_compressed":844946,"data_compressed_allocated":2260992,"data_compressed_original":20286069,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38736,"kb_used_data":3392,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333104,"statfs":{"total":96636764160,"available":96597098496,"internally_reserved":0,"allocated":3473408,"data_stored":19059501,"data_compressed":662941,"data_compressed_allocated":1904640,"data_compressed_original":18280448,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2514944,"data_stored":23924031,"data_compressed":438892,"data_compressed_allocated":1921024,"data_compressed_original":23549349,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2121728,"data_stored":16369699,"data_compressed":454272,"data_compressed_allocated":1527808,"data_compressed_original":16066345,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2842624,"data_stored":18797833,"data_compressed":678297,"data_compressed_allocated":1921024,"data_compressed_original":18305024,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3235840,"data_stored":20801784,"data_compressed":860302,"data_compressed_allocated":2277376,"data_compressed_original":20310645,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1531904,"data_stored":7951302,"data_compressed":298288,"data_compressed_allocated":909312,"data_compressed_original":7638881,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2789376,"data_stored":22042495,"data_compressed":668779,"data_compressed_allocated":2142208,"data_compressed_original":21717020,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1986560,"data_stored":14143151,"data_compressed":456987,"data_compressed_allocated":1433600,"data_compressed_original":13850284,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2551808,"data_stored":22193538,"data_compressed":565957,"data_compressed_allocated":2035712,"data_compressed_original":21932842,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:42:05.900 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:05.898+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:05.937 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:05.937 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:05.945 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:42:05.969 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:05.967+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:06.049 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:06.049 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:06.058 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:42:06.102 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:06.100+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:06.160 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:06.161 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:06.169 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:42:06.170 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:06.168+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:06.185 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:42:06.272 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:06.269+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:06.278 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone22 from 0xa3315b6 (overlap 0x995c215) to 0x0 2024-09-17T22:42:06.341 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:06.341 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:06.355 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:42:06.396 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:06.393+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:06.473 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:06.471+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:06.519 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:06.520 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:06.529 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:42:06.550 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-17T22:42:06.567 INFO:teuthology.orchestra.run.smithi115.stdout:2288 writesame 0x5ac184c thru 0x5ace508 data_size 0x1(0xccbd bytes) 2024-09-17T22:42:06.574 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:06.572+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:06.598 INFO:teuthology.orchestra.run.smithi115.stdout:2289 writesame 0x6f07967 thru 0x6f150fb data_size 0x1(0xd795 bytes) 2024-09-17T22:42:06.637 INFO:teuthology.orchestra.run.smithi115.stdout:2290 trunc from 0xa3315b6 to 0x36c8a65 2024-09-17T22:42:06.675 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:06.675 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:06.675 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:06.673+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:06.688 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:42:06.776 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:06.774+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:06.794 INFO:teuthology.orchestra.run.smithi115.stdout:2294 write 0x9fc7615 thru 0x9fd17f0 (0xa1dc bytes) 2024-09-17T22:42:06.797 INFO:teuthology.orchestra.run.smithi115.stdout:2295 punch from 0x8ac5506 to 0x8acf7c6, (0xa2c0 bytes) 2024-09-17T22:42:06.797 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:06.797 INFO:teuthology.orchestra.run.smithi115.stdout:2299 read 0x64f60e3 thru 0x65029dc (0xc8fa bytes) 2024-09-17T22:42:06.798 INFO:teuthology.orchestra.run.smithi115.stdout:2300 punch from 0x5d8993a to 0x5d95969, (0xc02f bytes) 2024-09-17T22:42:06.807 INFO:teuthology.orchestra.run.smithi115.stdout:2301 trunc from 0x9fd17f1 to 0x1784914 2024-09-17T22:42:06.823 INFO:teuthology.orchestra.run.smithi115.stdout:2302 trunc from 0x1784914 to 0x3756ee 2024-09-17T22:42:06.846 INFO:teuthology.orchestra.run.smithi115.stdout:2304 writesame 0x530d119 thru 0x5310d13 data_size 0x25(0x3bfb bytes) 2024-09-17T22:42:06.846 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:06.846 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:06.847 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:06.847 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:06.852 INFO:teuthology.orchestra.run.smithi115.stdout:2308 write 0x65b6705 thru 0x65ba156 (0x3a52 bytes) 2024-09-17T22:42:06.856 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:42:06.869 INFO:teuthology.orchestra.run.smithi115.stdout:2309 write 0xd62259b thru 0xd62db2a (0xb590 bytes) 2024-09-17T22:42:06.880 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:06.875+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:06.881 INFO:teuthology.orchestra.run.smithi115.stdout:2311 trunc from 0xd62db2b to 0x90487f5 2024-09-17T22:42:06.898 INFO:teuthology.orchestra.run.smithi115.stdout:2312 trunc from 0x90487f5 to 0x327b64 2024-09-17T22:42:06.909 INFO:teuthology.orchestra.run.smithi115.stdout:2313 read 0x232d6f thru 0x238cad (0x5f3f bytes) 2024-09-17T22:42:06.925 INFO:teuthology.orchestra.run.smithi115.stdout:2314 trunc from 0x327b64 to 0xc10065f 2024-09-17T22:42:06.931 INFO:teuthology.orchestra.run.smithi115.stdout:2317 write 0x33e379c thru 0x33e37dc (0x41 bytes) 2024-09-17T22:42:06.932 INFO:teuthology.orchestra.run.smithi115.stdout:2318 write 0xa42ff54 thru 0xa4363cc (0x6479 bytes) 2024-09-17T22:42:06.935 INFO:teuthology.orchestra.run.smithi115.stdout:2319 punch from 0xc0c306e to 0xc0ce8ba, (0xb84c bytes) 2024-09-17T22:42:06.941 INFO:teuthology.orchestra.run.smithi115.stdout:2321 write 0xd0b349f thru 0xd0c0504 (0xd066 bytes) 2024-09-17T22:42:06.944 INFO:teuthology.orchestra.run.smithi115.stdout:2322 read 0x275120 thru 0x27e942 (0x9823 bytes) 2024-09-17T22:42:06.945 INFO:teuthology.orchestra.run.smithi115.stdout:2324 write 0x933bd1d thru 0x9349df4 (0xe0d8 bytes) 2024-09-17T22:42:06.948 INFO:teuthology.orchestra.run.smithi115.stdout:2325 write 0x3be4776 thru 0x3bf049a (0xbd25 bytes) 2024-09-17T22:42:06.950 INFO:teuthology.orchestra.run.smithi115.stdout:2326 write 0xc4320ee thru 0xc435281 (0x3194 bytes) 2024-09-17T22:42:06.953 INFO:teuthology.orchestra.run.smithi115.stdout:2327 read 0xb8db69b thru 0xb8e0881 (0x51e7 bytes) 2024-09-17T22:42:06.953 INFO:teuthology.orchestra.run.smithi115.stdout:2329 punch from 0xbf8b28c to 0xbf92355, (0x70c9 bytes) 2024-09-17T22:42:06.953 INFO:teuthology.orchestra.run.smithi115.stdout:2330 write 0x7f3fd11 thru 0x7f46f34 (0x7224 bytes) 2024-09-17T22:42:06.956 INFO:teuthology.orchestra.run.smithi115.stdout:2331 read 0x867e95f thru 0x868249a (0x3b3c bytes) 2024-09-17T22:42:06.956 INFO:teuthology.orchestra.run.smithi115.stdout:2332 read 0x8a4e6ae thru 0x8a5c05c (0xd9af bytes) 2024-09-17T22:42:06.957 INFO:teuthology.orchestra.run.smithi115.stdout:2333 trunc from 0xd0c0505 to 0xc20fc72 2024-09-17T22:42:06.966 INFO:teuthology.orchestra.run.smithi115.stdout:2334 write 0x273c9c9 thru 0x2741713 (0x4d4b bytes) 2024-09-17T22:42:06.969 INFO:teuthology.orchestra.run.smithi115.stdout:2336 write 0x536b73c thru 0x537714d (0xba12 bytes) 2024-09-17T22:42:06.973 INFO:teuthology.orchestra.run.smithi115.stdout:2337 write 0x9088a2a thru 0x908d0ca (0x46a1 bytes) 2024-09-17T22:42:06.976 INFO:teuthology.orchestra.run.smithi115.stdout:2338 write 0x9bc542b thru 0x9bccf03 (0x7ad9 bytes) 2024-09-17T22:42:06.978 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:06.975+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:06.980 INFO:teuthology.orchestra.run.smithi115.stdout:2341 trunc from 0xc20fc72 to 0x9aae388 2024-09-17T22:42:06.980 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:06.981 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:06.990 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:42:06.992 INFO:teuthology.orchestra.run.smithi115.stdout:2343 write 0x2d44aa0 thru 0x2d470e0 (0x2641 bytes) 2024-09-17T22:42:06.994 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:06.995 INFO:teuthology.orchestra.run.smithi115.stdout:2348 writesame 0x61646c2 thru 0x61726fa data_size 0x1(0xe039 bytes) 2024-09-17T22:42:07.025 INFO:teuthology.orchestra.run.smithi115.stdout:2349 writesame 0x2caa65 thru 0x2d0cd3 data_size 0x1(0x626f bytes) 2024-09-17T22:42:07.037 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.042 INFO:teuthology.orchestra.run.smithi115.stdout:2352 write 0xa754b64 thru 0xa75888b (0x3d28 bytes) 2024-09-17T22:42:07.046 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.046 INFO:teuthology.orchestra.run.smithi115.stdout:2355 write 0x948d708 thru 0x9491ed4 (0x47cd bytes) 2024-09-17T22:42:07.050 INFO:teuthology.orchestra.run.smithi115.stdout:2356 trunc from 0xa75888c to 0xbd00d44 2024-09-17T22:42:07.054 INFO:teuthology.orchestra.run.smithi115.stdout:2357 writesame 0x883d21c thru 0x8849036 data_size 0x1(0xbe1b bytes) 2024-09-17T22:42:07.062 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:42:07.077 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.077 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.077 INFO:teuthology.orchestra.run.smithi115.stdout:2361 writesame 0x1ac415c thru 0x1acdc82 data_size 0x1(0x9b27 bytes) 2024-09-17T22:42:07.078 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:07.076+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:07.095 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.095 INFO:teuthology.orchestra.run.smithi115.stdout:2363 punch from 0x2b84055 to 0x2b8ae0e, (0x6db9 bytes) 2024-09-17T22:42:07.095 INFO:teuthology.orchestra.run.smithi115.stdout:2364 write 0x7f67edc thru 0x7f758f1 (0xda16 bytes) 2024-09-17T22:42:07.097 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.097 INFO:teuthology.orchestra.run.smithi115.stdout:2367 write 0x2e30090 thru 0x2e32b4a (0x2abb bytes) 2024-09-17T22:42:07.100 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:07.101 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:07.101 INFO:teuthology.orchestra.run.smithi115.stdout:2368 trunc from 0xbd00d44 to 0xdb0940f 2024-09-17T22:42:07.106 INFO:teuthology.orchestra.run.smithi115.stdout:2369 punch from 0x1e48e59 to 0x1e4bbb4, (0x2d5b bytes) 2024-09-17T22:42:07.106 INFO:teuthology.orchestra.run.smithi115.stdout:2370 punch from 0x8c9ec22 to 0x8caa37d, (0xb75b bytes) 2024-09-17T22:42:07.106 INFO:teuthology.orchestra.run.smithi115.stdout:2372 write 0x8bf1344 thru 0x8bf3324 (0x1fe1 bytes) 2024-09-17T22:42:07.107 INFO:teuthology.orchestra.run.smithi115.stdout:2373 read 0x7696878 thru 0x76a39e6 (0xd16f bytes) 2024-09-17T22:42:07.107 INFO:teuthology.orchestra.run.smithi115.stdout:2374 write 0x794fd4e thru 0x795081a (0xacd bytes) 2024-09-17T22:42:07.109 INFO:teuthology.orchestra.run.smithi115.stdout:2376 write 0x16703f5 thru 0x1679344 (0x8f50 bytes) 2024-09-17T22:42:07.111 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:42:07.111 INFO:teuthology.orchestra.run.smithi115.stdout:2377 write 0xc0c089c thru 0xc0c18fd (0x1062 bytes) 2024-09-17T22:42:07.113 INFO:teuthology.orchestra.run.smithi115.stdout:2378 write 0x14c2668 thru 0x14cf6c5 (0xd05e bytes) 2024-09-17T22:42:07.115 INFO:teuthology.orchestra.run.smithi115.stdout:2379 write 0x6144036 thru 0x614ce4f (0x8e1a bytes) 2024-09-17T22:42:07.116 INFO:teuthology.orchestra.run.smithi115.stdout:2380 read 0x611d4d4 thru 0x6128408 (0xaf35 bytes) 2024-09-17T22:42:07.116 INFO:teuthology.orchestra.run.smithi115.stdout:2385 write 0xb60b1 thru 0xbb82b (0x577b bytes) 2024-09-17T22:42:07.118 INFO:teuthology.orchestra.run.smithi115.stdout:2387 punch from 0xf864d0 to 0xf8f383, (0x8eb3 bytes) 2024-09-17T22:42:07.118 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.118 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.118 INFO:teuthology.orchestra.run.smithi115.stdout:2392 read 0x8f51a95 thru 0x8f5c48a (0xa9f6 bytes) 2024-09-17T22:42:07.119 INFO:teuthology.orchestra.run.smithi115.stdout:2393 write 0x9f511e9 thru 0x9f5d9a6 (0xc7be bytes) 2024-09-17T22:42:07.120 INFO:teuthology.orchestra.run.smithi115.stdout:2397 writesame 0x5cee6c5 thru 0x5cf3cb1 data_size 0x1(0x55ed bytes) 2024-09-17T22:42:07.133 INFO:teuthology.orchestra.run.smithi115.stdout:2398 trunc from 0xdb0940f to 0xc005534 2024-09-17T22:42:07.140 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.153 INFO:teuthology.orchestra.run.smithi115.stdout:2400 trunc from 0xc005534 to 0xbe4002 2024-09-17T22:42:07.165 INFO:teuthology.orchestra.run.smithi115.stdout:2401 read 0x197d30 thru 0x198d77 (0x1048 bytes) 2024-09-17T22:42:07.165 INFO:teuthology.orchestra.run.smithi115.stdout:2403 read 0x5fd67c thru 0x60306a (0x59ef bytes) 2024-09-17T22:42:07.165 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.166 INFO:teuthology.orchestra.run.smithi115.stdout:2409 read 0xb4c3e2 thru 0xb5151f (0x513e bytes) 2024-09-17T22:42:07.166 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.177 INFO:teuthology.orchestra.run.smithi115.stdout:2412 trunc from 0xbe4002 to 0x997aaed 2024-09-17T22:42:07.179 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:07.177+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:07.183 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is set 2024-09-17T22:42:07.184 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:07.185 INFO:teuthology.orchestra.run.smithi115.stdout:2413 read 0x7c395fe thru 0x7c3fd65 (0x6768 bytes) 2024-09-17T22:42:07.185 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.185 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.185 INFO:teuthology.orchestra.run.smithi115.stdout:2418 punch from 0x5467bea to 0x546fba6, (0x7fbc bytes) 2024-09-17T22:42:07.190 INFO:teuthology.orchestra.run.smithi115.stdout:2419 write 0x9bb0e07 thru 0x9bbd85d (0xca57 bytes) 2024-09-17T22:42:07.205 INFO:teuthology.orchestra.run.smithi115.stdout:2420 write 0xe4526e5 thru 0xe4558ed (0x3209 bytes) 2024-09-17T22:42:07.207 INFO:teuthology.orchestra.run.smithi115.stdout:2421 read 0x7ce32a0 thru 0x7ce4b3c (0x189d bytes) 2024-09-17T22:42:07.208 INFO:teuthology.orchestra.run.smithi115.stdout:2422 read 0xab66f51 thru 0xab69947 (0x29f7 bytes) 2024-09-17T22:42:07.209 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:07.209 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:07.212 INFO:teuthology.orchestra.run.smithi115.stdout:2423 trunc from 0xe4558ee to 0xa30774a 2024-09-17T22:42:07.227 INFO:teuthology.orchestra.run.smithi115.stdout:2424 write 0x130b1c5 thru 0x131b150 (0xff8c bytes) 2024-09-17T22:42:07.227 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:07.227 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:07.229 INFO:teuthology.orchestra.run.smithi115.stdout:2426 writesame 0x928dd74 thru 0x929dc02 data_size 0x1(0xfe8f bytes) 2024-09-17T22:42:07.233 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:07.233 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:07.242 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:42:07.244 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:07.244 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:07.258 INFO:teuthology.orchestra.run.smithi115.stdout:2427 punch from 0x5ba4ef0 to 0x5bb1a83, (0xcb93 bytes) 2024-09-17T22:42:07.262 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:07.262 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:07.267 INFO:teuthology.orchestra.run.smithi115.stdout:2428 write 0xd936ef2 thru 0xd9388ee (0x19fd bytes) 2024-09-17T22:42:07.270 INFO:teuthology.orchestra.run.smithi115.stdout:2429 read 0xc1b585b thru 0xc1c2db4 (0xd55a bytes) 2024-09-17T22:42:07.270 INFO:teuthology.orchestra.run.smithi115.stdout:2431 read 0x2160efa thru 0x216e881 (0xd988 bytes) 2024-09-17T22:42:07.280 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:07.280 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:07.288 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:07.277+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:07.289 INFO:teuthology.orchestra.run.smithi115.stdout:2432 writesame 0x60968f0 thru 0x609caf2 data_size 0x1(0x6203 bytes) 2024-09-17T22:42:07.297 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:07.297 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:07.303 INFO:teuthology.orchestra.run.smithi115.stdout:2433 punch from 0x38f49c7 to 0x38f8acd, (0x4106 bytes) 2024-09-17T22:42:07.303 INFO:teuthology.orchestra.run.smithi115.stdout:2434 write 0x52c94e4 thru 0x52d3ae7 (0xa604 bytes) 2024-09-17T22:42:07.306 INFO:teuthology.orchestra.run.smithi115.stdout:2435 writesame 0x2c283e5 thru 0x2c2c2ef data_size 0x1(0x3f0b bytes) 2024-09-17T22:42:07.311 INFO:teuthology.orchestra.run.smithi115.stdout:2436 write 0x35c219b thru 0x35cb185 (0x8feb bytes) 2024-09-17T22:42:07.313 INFO:teuthology.orchestra.run.smithi115.stdout:2437 punch from 0x4057d8c to 0x405db9d, (0x5e11 bytes) 2024-09-17T22:42:07.313 INFO:teuthology.orchestra.run.smithi115.stdout:2438 writesame 0x5b7f4db thru 0x5b8853e data_size 0x1(0x9064 bytes) 2024-09-17T22:42:07.314 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:07.314 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:07.318 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:07.318 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:07.342 INFO:teuthology.orchestra.run.smithi115.stdout:2441 trunc from 0xd9388ef to 0x3883cb5 2024-09-17T22:42:07.353 INFO:teuthology.orchestra.run.smithi115.stdout:2442 read 0x2b96228 thru 0x2ba29ee (0xc7c7 bytes) 2024-09-17T22:42:07.353 INFO:teuthology.orchestra.run.smithi115.stdout:2443 punch from 0xb6525d to 0xb67263, (0x2006 bytes) 2024-09-17T22:42:07.353 INFO:teuthology.orchestra.run.smithi115.stdout:2445 read 0x34626a6 thru 0x3463d64 (0x16bf bytes) 2024-09-17T22:42:07.354 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:07.354 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:07.354 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:07.354 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:07.355 INFO:teuthology.orchestra.run.smithi115.stdout:2446 trunc from 0x3883cb5 to 0x2779aff 2024-09-17T22:42:07.363 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:42:07.379 INFO:teuthology.orchestra.run.smithi115.stdout:2447 write 0xcacbf03 thru 0xcadb573 (0xf671 bytes) 2024-09-17T22:42:07.381 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:07.379+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:07.382 INFO:teuthology.orchestra.run.smithi115.stdout:2449 write 0xd4a8d1 thru 0xd58493 (0xdbc3 bytes) 2024-09-17T22:42:07.384 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.388 INFO:teuthology.orchestra.run.smithi115.stdout:2453 trunc from 0xcadb574 to 0x8ddadb4 2024-09-17T22:42:07.398 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.398 INFO:teuthology.orchestra.run.smithi115.stdout:2456 writesame 0x254850b thru 0x254c41c data_size 0x36(0x3f12 bytes) 2024-09-17T22:42:07.400 INFO:teuthology.orchestra.run.smithi115.stdout:2457 read 0x251203 thru 0x257913 (0x6711 bytes) 2024-09-17T22:42:07.400 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.400 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.401 INFO:teuthology.orchestra.run.smithi115.stdout:2460 write 0x583670c thru 0x583c808 (0x60fd bytes) 2024-09-17T22:42:07.403 INFO:teuthology.orchestra.run.smithi115.stdout:2463 writesame 0x8cadee3 thru 0x8cb0ea8 data_size 0x5(0x2fc6 bytes) 2024-09-17T22:42:07.407 INFO:teuthology.orchestra.run.smithi115.stdout:2465 punch from 0x1d3113e to 0x1d3a166, (0x9028 bytes) 2024-09-17T22:42:07.407 INFO:teuthology.orchestra.run.smithi115.stdout:2467 write 0x68283a2 thru 0x6830484 (0x80e3 bytes) 2024-09-17T22:42:07.408 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.408 INFO:teuthology.orchestra.run.smithi115.stdout:2469 read 0x833650d thru 0x83428e2 (0xc3d6 bytes) 2024-09-17T22:42:07.409 INFO:teuthology.orchestra.run.smithi115.stdout:2470 write 0x411589c thru 0x4125699 (0xfdfe bytes) 2024-09-17T22:42:07.410 INFO:teuthology.orchestra.run.smithi115.stdout:2471 read 0x3b56130 thru 0x3b5cd5f (0x6c30 bytes) 2024-09-17T22:42:07.410 INFO:teuthology.orchestra.run.smithi115.stdout:2472 punch from 0x12281d6 to 0x1235fcd, (0xddf7 bytes) 2024-09-17T22:42:07.410 INFO:teuthology.orchestra.run.smithi115.stdout:2473 punch from 0x67d592f to 0x67d7fed, (0x26be bytes) 2024-09-17T22:42:07.419 INFO:teuthology.orchestra.run.smithi115.stdout:2475 trunc from 0x8ddadb4 to 0x11b4bb1 2024-09-17T22:42:07.432 INFO:teuthology.orchestra.run.smithi115.stdout:2476 read 0x8ee0b3 thru 0x8eef6c (0xeba bytes) 2024-09-17T22:42:07.432 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.451 INFO:teuthology.orchestra.run.smithi115.stdout:2480 writesame 0xbe2fe6b thru 0xbe32660 data_size 0x21(0x27f6 bytes) 2024-09-17T22:42:07.451 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.451 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.451 INFO:teuthology.orchestra.run.smithi115.stdout:2484 write 0x9673e17 thru 0x96832f9 (0xf4e3 bytes) 2024-09-17T22:42:07.455 INFO:teuthology.orchestra.run.smithi115.stdout:2486 punch from 0x14b0ee6 to 0x14b40e8, (0x3202 bytes) 2024-09-17T22:42:07.455 INFO:teuthology.orchestra.run.smithi115.stdout:2487 trunc from 0xbe32661 to 0xc2e5abf 2024-09-17T22:42:07.463 INFO:teuthology.orchestra.run.smithi115.stdout:2490 flatten 2024-09-17T22:42:07.469 INFO:teuthology.orchestra.run.smithi115.stdout:2492 read 0x2162d02 thru 0x216982e (0x6b2d bytes) 2024-09-17T22:42:07.474 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:07.474 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:07.474 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:07.474 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:07.474 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:07.478 INFO:teuthology.orchestra.run.smithi115.stdout:2493 write 0xd4f6c43 thru 0xd4fd79e (0x6b5c bytes) 2024-09-17T22:42:07.481 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.482 INFO:teuthology.orchestra.run.smithi115.stdout:2495 writesame 0xc311749 thru 0xc31bb09 data_size 0x25(0xa3c1 bytes) 2024-09-17T22:42:07.482 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:07.479+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:07.483 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:42:07.485 INFO:teuthology.orchestra.run.smithi115.stdout:2498 read 0x9555b09 thru 0x95627dc (0xccd4 bytes) 2024-09-17T22:42:07.485 INFO:teuthology.orchestra.run.smithi115.stdout:2499 write 0x56fca66 thru 0x57082d8 (0xb873 bytes) 2024-09-17T22:42:07.494 INFO:teuthology.orchestra.run.smithi115.stdout:2500 trunc from 0xd4fd79f to 0x7dc5e85 2024-09-17T22:42:07.505 INFO:teuthology.orchestra.run.smithi115.stdout:2502 writesame 0x2febdf thru 0x305dd0 data_size 0x5(0x71f2 bytes) 2024-09-17T22:42:07.508 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.509 INFO:teuthology.orchestra.run.smithi115.stdout:2506 writesame 0x6bf9a1b thru 0x6c05e1f data_size 0x2b(0xc405 bytes) 2024-09-17T22:42:07.512 INFO:teuthology.orchestra.run.smithi115.stdout:2507 writesame 0x57b6da3 thru 0x57c67f3 data_size 0x1(0xfa51 bytes) 2024-09-17T22:42:07.550 INFO:teuthology.orchestra.run.smithi115.stdout:2508 trunc from 0x7dc5e85 to 0x1b6b6fe 2024-09-17T22:42:07.570 INFO:teuthology.orchestra.run.smithi115.stdout:2509 trunc from 0x1b6b6fe to 0x8394b37 2024-09-17T22:42:07.576 INFO:teuthology.orchestra.run.smithi115.stdout:2510 read 0x7a732af thru 0x7a746da (0x142c bytes) 2024-09-17T22:42:07.576 INFO:teuthology.orchestra.run.smithi115.stdout:2512 write 0x61a9ab7 thru 0x61ae3ff (0x4949 bytes) 2024-09-17T22:42:07.579 INFO:teuthology.orchestra.run.smithi115.stdout:2514 read 0x21a449b thru 0x21a4fde (0xb44 bytes) 2024-09-17T22:42:07.579 INFO:teuthology.orchestra.run.smithi115.stdout:2515 writesame 0xec830d thru 0xec9e96 data_size 0x2f(0x1b8a bytes) 2024-09-17T22:42:07.580 INFO:teuthology.orchestra.run.smithi115.stdout:2516 read 0x27bf8e2 thru 0x27cabf4 (0xb313 bytes) 2024-09-17T22:42:07.581 INFO:teuthology.orchestra.run.smithi115.stdout:2517 punch from 0x5cdc4fb to 0x5cdc529, (0x2e bytes) 2024-09-17T22:42:07.583 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:07.580+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:07.588 INFO:teuthology.orchestra.run.smithi115.stdout:2518 writesame 0x980fc34 thru 0x98151db data_size 0x4(0x55a8 bytes) 2024-09-17T22:42:07.588 INFO:teuthology.orchestra.run.smithi115.stdout:2519 writesame 0x72c6028 thru 0x72c6125 data_size 0x1(0xfe bytes) 2024-09-17T22:42:07.591 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.591 INFO:teuthology.orchestra.run.smithi115.stdout:2522 read 0x62b3f6b thru 0x62c0af6 (0xcb8c bytes) 2024-09-17T22:42:07.591 INFO:teuthology.orchestra.run.smithi115.stdout:2525 write 0x56d5d5c thru 0x56e38ee (0xdb93 bytes) 2024-09-17T22:42:07.594 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:07.594 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:07.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:07.594 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:07.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.595 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158673 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.595 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:07.595 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349717734999999, 2024-09-17T22:42:07.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039400614, 2024-09-17T22:42:07.595 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.595 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.595 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.595 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.595 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.595 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.595 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158673 2024-09-17T22:42:07.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.595 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:07.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:07.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:07.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.04e-07 2024-09-17T22:42:07.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241563+0000", 2024-09-17T22:42:07.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5799999999999998e-07 2024-09-17T22:42:07.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241566+0000", 2024-09-17T22:42:07.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8279999999999999e-06 2024-09-17T22:42:07.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241568+0000", 2024-09-17T22:42:07.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2709999999999999e-06 2024-09-17T22:42:07.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280954+0000", 2024-09-17T22:42:07.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039386051999999998 2024-09-17T22:42:07.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280962+0000", 2024-09-17T22:42:07.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3010000000000007e-06 2024-09-17T22:42:07.602 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.602 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.602 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.602 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158675 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.602 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:07.602 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349708444000001, 2024-09-17T22:42:07.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039408033000000002, 2024-09-17T22:42:07.602 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.602 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.602 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.603 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.603 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.603 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.603 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158675 2024-09-17T22:42:07.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.603 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:07.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:07.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:07.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.36e-07 2024-09-17T22:42:07.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241572+0000", 2024-09-17T22:42:07.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6599999999999994e-07 2024-09-17T22:42:07.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241575+0000", 2024-09-17T22:42:07.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8389999999999998e-06 2024-09-17T22:42:07.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241577+0000", 2024-09-17T22:42:07.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4119999999999999e-06 2024-09-17T22:42:07.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280975+0000", 2024-09-17T22:42:07.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039397205999999997 2024-09-17T22:42:07.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280979+0000", 2024-09-17T22:42:07.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.374e-06 2024-09-17T22:42:07.606 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.606 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.606 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.606 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158677 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.606 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:07.607 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349686000999998, 2024-09-17T22:42:07.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039409307999999997, 2024-09-17T22:42:07.607 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.607 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.607 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.607 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.607 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.607 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.607 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158677 2024-09-17T22:42:07.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.607 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:07.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:07.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241594+0000", 2024-09-17T22:42:07.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:42:07.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241595+0000", 2024-09-17T22:42:07.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2729999999999999e-06 2024-09-17T22:42:07.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241602+0000", 2024-09-17T22:42:07.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:42:07.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241606+0000", 2024-09-17T22:42:07.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3019999999999999e-06 2024-09-17T22:42:07.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280994+0000", 2024-09-17T22:42:07.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039388484000000001 2024-09-17T22:42:07.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281003+0000", 2024-09-17T22:42:07.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5459999999999992e-06 2024-09-17T22:42:07.610 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.610 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.610 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.610 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158679 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.611 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:07.611 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349670323000002, 2024-09-17T22:42:07.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039413403999999999, 2024-09-17T22:42:07.611 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.611 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.611 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.611 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.611 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.611 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.611 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158679 2024-09-17T22:42:07.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.611 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:07.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:07.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.612 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.612 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:07.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4999999999999999e-07 2024-09-17T22:42:07.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241613+0000", 2024-09-17T22:42:07.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.207e-06 2024-09-17T22:42:07.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241616+0000", 2024-09-17T22:42:07.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.275e-06 2024-09-17T22:42:07.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241618+0000", 2024-09-17T22:42:07.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:42:07.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281017+0000", 2024-09-17T22:42:07.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039398385000000001 2024-09-17T22:42:07.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281022+0000", 2024-09-17T22:42:07.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.896e-06 2024-09-17T22:42:07.614 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.614 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.614 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.615 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158681 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.615 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:07.615 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349658144999999, 2024-09-17T22:42:07.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039425292000000001, 2024-09-17T22:42:07.615 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.615 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.615 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.615 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.615 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.615 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.615 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158681 2024-09-17T22:42:07.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.615 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:07.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:07.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:07.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:42:07.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241622+0000", 2024-09-17T22:42:07.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:42:07.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241625+0000", 2024-09-17T22:42:07.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9629999999999998e-06 2024-09-17T22:42:07.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241628+0000", 2024-09-17T22:42:07.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4339999999999999e-06 2024-09-17T22:42:07.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281039+0000", 2024-09-17T22:42:07.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039410672000000001 2024-09-17T22:42:07.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281047+0000", 2024-09-17T22:42:07.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9969999999999998e-06 2024-09-17T22:42:07.618 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.618 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.618 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.619 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158682 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.619 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:07.619 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349648860999999, 2024-09-17T22:42:07.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039436455000000002, 2024-09-17T22:42:07.619 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.619 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.619 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.619 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158682 2024-09-17T22:42:07.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.619 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:42:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241632+0000", 2024-09-17T22:42:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:42:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241635+0000", 2024-09-17T22:42:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8940000000000002e-06 2024-09-17T22:42:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241637+0000", 2024-09-17T22:42:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3659999999999999e-06 2024-09-17T22:42:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281061+0000", 2024-09-17T22:42:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039423664999999997 2024-09-17T22:42:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281067+0000", 2024-09-17T22:42:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.285e-06 2024-09-17T22:42:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.622 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158684 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349639515, 2024-09-17T22:42:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462980000000002, 2024-09-17T22:42:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158684 2024-09-17T22:42:07.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241641+0000", 2024-09-17T22:42:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:42:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241644+0000", 2024-09-17T22:42:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8870000000000001e-06 2024-09-17T22:42:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241646+0000", 2024-09-17T22:42:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3530000000000002e-06 2024-09-17T22:42:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281093+0000", 2024-09-17T22:42:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039446947000000003 2024-09-17T22:42:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281103+0000", 2024-09-17T22:42:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6299999999999993e-06 2024-09-17T22:42:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.626 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158686 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349630345000001, 2024-09-17T22:42:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039474214000000001, 2024-09-17T22:42:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.627 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158686 2024-09-17T22:42:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:42:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241650+0000", 2024-09-17T22:42:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2999999999999999e-07 2024-09-17T22:42:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241653+0000", 2024-09-17T22:42:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9689999999999999e-06 2024-09-17T22:42:07.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241655+0000", 2024-09-17T22:42:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3460000000000001e-06 2024-09-17T22:42:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281118+0000", 2024-09-17T22:42:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462876000000001 2024-09-17T22:42:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281123+0000", 2024-09-17T22:42:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8840000000000002e-06 2024-09-17T22:42:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158688 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349621114000001, 2024-09-17T22:42:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039490458999999999, 2024-09-17T22:42:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.631 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158688 2024-09-17T22:42:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:42:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241659+0000", 2024-09-17T22:42:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3699999999999999e-07 2024-09-17T22:42:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241662+0000", 2024-09-17T22:42:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8499999999999998e-06 2024-09-17T22:42:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241665+0000", 2024-09-17T22:42:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.356e-06 2024-09-17T22:42:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281141+0000", 2024-09-17T22:42:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039476031000000002 2024-09-17T22:42:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281149+0000", 2024-09-17T22:42:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0709999999999995e-06 2024-09-17T22:42:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158690 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349612055000001, 2024-09-17T22:42:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039502378999999997, 2024-09-17T22:42:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.635 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158690 2024-09-17T22:42:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241668+0000", 2024-09-17T22:42:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241669+0000", 2024-09-17T22:42:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1009999999999999e-06 2024-09-17T22:42:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241672+0000", 2024-09-17T22:42:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9059999999999998e-06 2024-09-17T22:42:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241674+0000", 2024-09-17T22:42:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4439999999999998e-06 2024-09-17T22:42:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281162+0000", 2024-09-17T22:42:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039488189 2024-09-17T22:42:07.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281170+0000", 2024-09-17T22:42:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5310000000000004e-06 2024-09-17T22:42:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158692 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349602621999999, 2024-09-17T22:42:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039531064999999997, 2024-09-17T22:42:07.639 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158692 2024-09-17T22:42:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241678+0000", 2024-09-17T22:42:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0800000000000003e-07 2024-09-17T22:42:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241681+0000", 2024-09-17T22:42:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8729999999999999e-06 2024-09-17T22:42:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241683+0000", 2024-09-17T22:42:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.447e-06 2024-09-17T22:42:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281199+0000", 2024-09-17T22:42:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039515388999999998 2024-09-17T22:42:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281208+0000", 2024-09-17T22:42:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2399999999999996e-06 2024-09-17T22:42:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158694 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.643 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349593451, 2024-09-17T22:42:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039543986000000003, 2024-09-17T22:42:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158694 2024-09-17T22:42:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:42:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241687+0000", 2024-09-17T22:42:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:42:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241691+0000", 2024-09-17T22:42:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.811e-06 2024-09-17T22:42:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241693+0000", 2024-09-17T22:42:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.385e-06 2024-09-17T22:42:07.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281223+0000", 2024-09-17T22:42:07.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039529332 2024-09-17T22:42:07.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281230+0000", 2024-09-17T22:42:07.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2139999999999999e-06 2024-09-17T22:42:07.647 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.647 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.647 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.647 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158696 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349583063000001, 2024-09-17T22:42:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039570458000000003, 2024-09-17T22:42:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158696 2024-09-17T22:42:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:42:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:42:07.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:42:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.33e-07 2024-09-17T22:42:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241700+0000", 2024-09-17T22:42:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8949999999999998e-06 2024-09-17T22:42:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241703+0000", 2024-09-17T22:42:07.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3159999999999999e-06 2024-09-17T22:42:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281257+0000", 2024-09-17T22:42:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039553862000000002 2024-09-17T22:42:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281267+0000", 2024-09-17T22:42:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.024e-05 2024-09-17T22:42:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158698 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349572985000002, 2024-09-17T22:42:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039576493999999997, 2024-09-17T22:42:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158698 2024-09-17T22:42:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.652 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241707+0000", 2024-09-17T22:42:07.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:42:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241708+0000", 2024-09-17T22:42:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5000000000000001e-07 2024-09-17T22:42:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241711+0000", 2024-09-17T22:42:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9670000000000001e-06 2024-09-17T22:42:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241713+0000", 2024-09-17T22:42:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2689999999999998e-06 2024-09-17T22:42:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281278+0000", 2024-09-17T22:42:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039565206999999998 2024-09-17T22:42:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281283+0000", 2024-09-17T22:42:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:42:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.655 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158700 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349563774, 2024-09-17T22:42:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039604292999999999, 2024-09-17T22:42:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.656 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158700 2024-09-17T22:42:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241717+0000", 2024-09-17T22:42:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.78e-07 2024-09-17T22:42:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241720+0000", 2024-09-17T22:42:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8119999999999999e-06 2024-09-17T22:42:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241722+0000", 2024-09-17T22:42:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-17T22:42:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281311+0000", 2024-09-17T22:42:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039588894999999999 2024-09-17T22:42:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281320+0000", 2024-09-17T22:42:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9989999999999997e-06 2024-09-17T22:42:07.659 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158702 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349552452000001, 2024-09-17T22:42:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039616142, 2024-09-17T22:42:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.660 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158702 2024-09-17T22:42:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4400000000000001e-07 2024-09-17T22:42:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241728+0000", 2024-09-17T22:42:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0839999999999999e-06 2024-09-17T22:42:07.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241731+0000", 2024-09-17T22:42:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0690000000000002e-06 2024-09-17T22:42:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241734+0000", 2024-09-17T22:42:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4159999999999998e-06 2024-09-17T22:42:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281334+0000", 2024-09-17T22:42:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039600110000000001 2024-09-17T22:42:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281343+0000", 2024-09-17T22:42:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2189999999999993e-06 2024-09-17T22:42:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158704 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.664 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349542587999998, 2024-09-17T22:42:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039641485999999997, 2024-09-17T22:42:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158704 2024-09-17T22:42:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:42:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241738+0000", 2024-09-17T22:42:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0589999999999999e-06 2024-09-17T22:42:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241741+0000", 2024-09-17T22:42:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.943e-06 2024-09-17T22:42:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241744+0000", 2024-09-17T22:42:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4819999999999999e-06 2024-09-17T22:42:07.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281370+0000", 2024-09-17T22:42:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039625661 2024-09-17T22:42:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281378+0000", 2024-09-17T22:42:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1310000000000006e-06 2024-09-17T22:42:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158705 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349532038, 2024-09-17T22:42:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039652637999999997, 2024-09-17T22:42:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158705 2024-09-17T22:42:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:42:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-07 2024-09-17T22:42:07.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:42:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0599999999999999e-07 2024-09-17T22:42:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241751+0000", 2024-09-17T22:42:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.926e-06 2024-09-17T22:42:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241754+0000", 2024-09-17T22:42:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4360000000000001e-06 2024-09-17T22:42:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281392+0000", 2024-09-17T22:42:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039638566 2024-09-17T22:42:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281400+0000", 2024-09-17T22:42:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.571e-06 2024-09-17T22:42:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.672 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158707 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349510937000002, 2024-09-17T22:42:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039654809999999999, 2024-09-17T22:42:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.673 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158707 2024-09-17T22:42:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241769+0000", 2024-09-17T22:42:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4299999999999999e-07 2024-09-17T22:42:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241770+0000", 2024-09-17T22:42:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3769999999999999e-06 2024-09-17T22:42:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241773+0000", 2024-09-17T22:42:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2109999999999998e-06 2024-09-17T22:42:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241776+0000", 2024-09-17T22:42:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5069999999999999e-06 2024-09-17T22:42:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281415+0000", 2024-09-17T22:42:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039639499000000002 2024-09-17T22:42:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281423+0000", 2024-09-17T22:42:07.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8730000000000007e-06 2024-09-17T22:42:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158709 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.349500328000001, 2024-09-17T22:42:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039661937000000001, 2024-09-17T22:42:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:07.677 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158709 2024-09-17T22:42:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242843+0000", 2024-09-17T22:42:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010634780000000001 2024-09-17T22:42:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242845+0000", 2024-09-17T22:42:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7209999999999999e-06 2024-09-17T22:42:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242851+0000", 2024-09-17T22:42:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6949999999999997e-06 2024-09-17T22:42:07.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242855+0000", 2024-09-17T22:42:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7960000000000002e-06 2024-09-17T22:42:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281434+0000", 2024-09-17T22:42:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038579519 2024-09-17T22:42:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:07.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281441+0000", 2024-09-17T22:42:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7279999999999998e-06 2024-09-17T22:42:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:07.681 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:07.681 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:07.683 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:42:07.684 INFO:teuthology.orchestra.run.smithi115.stdout:2527 punch from 0x29d64e4 to 0x29d9c2c, (0x3748 bytes) 2024-09-17T22:42:07.684 INFO:teuthology.orchestra.run.smithi115.stdout:2528 punch from 0x735e9f0 to 0x736d39b, (0xe9ab bytes) 2024-09-17T22:42:07.684 INFO:teuthology.orchestra.run.smithi115.stdout:2529 write 0x57bd3a0 thru 0x57cc025 (0xec86 bytes) 2024-09-17T22:42:07.684 INFO:teuthology.orchestra.run.smithi115.stdout:2530 write 0xaed5180 thru 0xaede6f8 (0x9579 bytes) 2024-09-17T22:42:07.685 INFO:teuthology.orchestra.run.smithi115.stdout:2531 read 0x44d39cc thru 0x44de7b2 (0xade7 bytes) 2024-09-17T22:42:07.685 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.685 INFO:teuthology.orchestra.run.smithi115.stdout:2533 read 0x83334b8 thru 0x8337b42 (0x468b bytes) 2024-09-17T22:42:07.685 INFO:teuthology.orchestra.run.smithi115.stdout:2534 read 0xac38318 thru 0xac46a4e (0xe737 bytes) 2024-09-17T22:42:07.685 INFO:teuthology.orchestra.run.smithi115.stdout:2537 write 0xce815e4 thru 0xce84b0b (0x3528 bytes) 2024-09-17T22:42:07.685 INFO:teuthology.orchestra.run.smithi115.stdout:2538 read 0x62acaf0 thru 0x62b6482 (0x9993 bytes) 2024-09-17T22:42:07.685 INFO:teuthology.orchestra.run.smithi115.stdout:2539 read 0x452c7fd thru 0x453c581 (0xfd85 bytes) 2024-09-17T22:42:07.685 INFO:teuthology.orchestra.run.smithi115.stdout:2540 read 0x25d61ee thru 0x25d8a99 (0x28ac bytes) 2024-09-17T22:42:07.685 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.685 INFO:teuthology.orchestra.run.smithi115.stdout:2542 write 0xa2fab13 thru 0xa301525 (0x6a13 bytes) 2024-09-17T22:42:07.685 INFO:teuthology.orchestra.run.smithi115.stdout:2543 trunc from 0xce84b0c to 0x3cd9be 2024-09-17T22:42:07.685 INFO:teuthology.orchestra.run.smithi115.stdout:2545 read 0x24689f thru 0x2519bb (0xb11d bytes) 2024-09-17T22:42:07.685 INFO:teuthology.orchestra.run.smithi115.stdout:2548 punch from 0x39f7cb to 0x3acfdf, (0xd814 bytes) 2024-09-17T22:42:07.685 INFO:teuthology.orchestra.run.smithi115.stdout:2549 read 0x2dcacf thru 0x2ec12e (0xf660 bytes) 2024-09-17T22:42:07.685 INFO:teuthology.orchestra.run.smithi115.stdout:2550 writesame 0x1725156 thru 0x1733b13 data_size 0x3(0xe9be bytes) 2024-09-17T22:42:07.686 INFO:teuthology.orchestra.run.smithi115.stdout:2551 writesame 0x7f03689 thru 0x7f12667 data_size 0x3(0xefdf bytes) 2024-09-17T22:42:07.687 INFO:teuthology.orchestra.run.smithi115.stdout:2553 write 0xcfd7cd3 thru 0xcfe239c (0xa6ca bytes) 2024-09-17T22:42:07.689 INFO:teuthology.orchestra.run.smithi115.stdout:2554 punch from 0x1b2ee5a to 0x1b36a5a, (0x7c00 bytes) 2024-09-17T22:42:07.695 INFO:teuthology.orchestra.run.smithi115.stdout:2556 write 0xd45fb64 thru 0xd4689b1 (0x8e4e bytes) 2024-09-17T22:42:07.713 INFO:teuthology.orchestra.run.smithi115.stdout:2557 trunc from 0xd4689b2 to 0x8e24d5 2024-09-17T22:42:07.723 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:07.721+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:07.727 INFO:teuthology.orchestra.run.smithi115.stdout:2558 punch from 0x6d644c to 0x6dd9e3, (0x7597 bytes) 2024-09-17T22:42:07.736 INFO:teuthology.orchestra.run.smithi115.stdout:2559 writesame 0x3b84afa thru 0x3b8966b data_size 0x25(0x4b72 bytes) 2024-09-17T22:42:07.752 INFO:teuthology.orchestra.run.smithi115.stdout:2560 write 0xb371011 thru 0xb3732b7 (0x22a7 bytes) 2024-09-17T22:42:07.756 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.761 INFO:teuthology.orchestra.run.smithi115.stdout:2565 write 0xb8f949f thru 0xb90699a (0xd4fc bytes) 2024-09-17T22:42:07.764 INFO:teuthology.orchestra.run.smithi115.stdout:2566 read 0x2f5bed8 thru 0x2f62fd3 (0x70fc bytes) 2024-09-17T22:42:07.765 INFO:teuthology.orchestra.run.smithi115.stdout:2568 read 0x62a7b40 thru 0x62acbb6 (0x5077 bytes) 2024-09-17T22:42:07.773 INFO:teuthology.orchestra.run.smithi115.stdout:2569 trunc from 0xb90699b to 0x3e6d870 2024-09-17T22:42:07.784 INFO:teuthology.orchestra.run.smithi115.stdout:2570 read 0x3b700d5 thru 0x3b79704 (0x9630 bytes) 2024-09-17T22:42:07.784 INFO:teuthology.orchestra.run.smithi115.stdout:2571 punch from 0x3836b6c to 0x3841619, (0xaaad bytes) 2024-09-17T22:42:07.787 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:07.785+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:07.799 INFO:teuthology.orchestra.run.smithi115.stdout:2572 writesame 0xb5e2732 thru 0xb5ed08f data_size 0x85(0xa95e bytes) 2024-09-17T22:42:07.799 INFO:teuthology.orchestra.run.smithi115.stdout:2574 read 0x265f86a thru 0x2669f49 (0xa6e0 bytes) 2024-09-17T22:42:07.799 INFO:teuthology.orchestra.run.smithi115.stdout:2575 write 0x444c82a thru 0x444e355 (0x1b2c bytes) 2024-09-17T22:42:07.802 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:07.802 INFO:teuthology.orchestra.run.smithi115.stdout:2577 read 0x72d4249 thru 0x72d815c (0x3f14 bytes) 2024-09-17T22:42:07.802 INFO:teuthology.orchestra.run.smithi115.stdout:2578 writesame 0x4d91892 thru 0x4da0fb5 data_size 0x2(0xf724 bytes) 2024-09-17T22:42:07.818 INFO:teuthology.orchestra.run.smithi115.stdout:2579 writesame 0x72b5752 thru 0x72c4092 data_size 0x1(0xe941 bytes) 2024-09-17T22:42:07.832 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:07.833 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:07.833 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:07.833 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:07.842 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:42:07.847 INFO:teuthology.orchestra.run.smithi115.stdout:2580 trunc from 0xb5ed090 to 0x496873a 2024-09-17T22:42:07.854 INFO:teuthology.orchestra.run.smithi115.stdout:2583 read 0x10f3f57 thru 0x10fef2e (0xafd8 bytes) 2024-09-17T22:42:07.857 INFO:teuthology.orchestra.run.smithi115.stdout:2585 trunc from 0x496873a to 0x1cadf3d 2024-09-17T22:42:07.869 INFO:teuthology.orchestra.run.smithi115.stdout:2586 read 0x14efeea thru 0x14f18e6 (0x19fd bytes) 2024-09-17T22:42:07.881 INFO:teuthology.orchestra.run.smithi115.stdout:2587 trunc from 0x1cadf3d to 0xbdee610 2024-09-17T22:42:07.886 INFO:teuthology.orchestra.run.smithi115.stdout:2588 write 0x5c3c76f thru 0x5c43246 (0x6ad8 bytes) 2024-09-17T22:42:07.889 INFO:teuthology.orchestra.run.smithi115.stdout:2589 punch from 0x16d2817 to 0x16d53c0, (0x2ba9 bytes) 2024-09-17T22:42:07.896 INFO:teuthology.orchestra.run.smithi115.stdout:2590 writesame 0xc05d672 thru 0xc06ce11 data_size 0x38(0xf7a0 bytes) 2024-09-17T22:42:07.896 INFO:teuthology.orchestra.run.smithi115.stdout:2591 punch from 0xa053431 to 0xa05fc9d, (0xc86c bytes) 2024-09-17T22:42:07.896 INFO:teuthology.orchestra.run.smithi115.stdout:2592 read 0x3cc6023 thru 0x3cd2ddb (0xcdb9 bytes) 2024-09-17T22:42:07.896 INFO:teuthology.orchestra.run.smithi115.stdout:2593 write 0x5b0bb2e thru 0x5b0feb0 (0x4383 bytes) 2024-09-17T22:42:07.899 INFO:teuthology.orchestra.run.smithi115.stdout:2594 clone 24 order 20 su 8192 sc 11 2024-09-17T22:42:07.925 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:07.923+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:07.986 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:07.986 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:07.986 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:07.986 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:07.987 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:07.989 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:07.986+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:07.995 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:42:08.089 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:08.087+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:08.141 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:08.141 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:08.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:08.141 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:08.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.141 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157229 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4073472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.141 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:08.141 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.991318751000001, 2024-09-17T22:42:08.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041957959000000003, 2024-09-17T22:42:08.141 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.141 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.142 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.142 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.142 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.142 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.142 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157229 2024-09-17T22:42:08.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.142 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:08.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:08.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146207+0000", 2024-09-17T22:42:08.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0299999999999999e-07 2024-09-17T22:42:08.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146215+0000", 2024-09-17T22:42:08.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3059999999999998e-06 2024-09-17T22:42:08.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146222+0000", 2024-09-17T22:42:08.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1180000000000003e-06 2024-09-17T22:42:08.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146227+0000", 2024-09-17T22:42:08.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1749999999999998e-06 2024-09-17T22:42:08.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.187985+0000", 2024-09-17T22:42:08.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041758391999999998 2024-09-17T22:42:08.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188065+0000", 2024-09-17T22:42:08.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0290000000000005e-05 2024-09-17T22:42:08.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188164+0000", 2024-09-17T22:42:08.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9074999999999996e-05 2024-09-17T22:42:08.145 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.146 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.146 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.146 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157231 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4079616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.146 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:08.146 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.991181127000001, 2024-09-17T22:42:08.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042034581000000001, 2024-09-17T22:42:08.146 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.146 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.146 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.146 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.146 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.147 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.147 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157231 2024-09-17T22:42:08.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.147 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:08.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:08.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146345+0000", 2024-09-17T22:42:08.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:42:08.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146347+0000", 2024-09-17T22:42:08.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.357e-06 2024-09-17T22:42:08.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146352+0000", 2024-09-17T22:42:08.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T22:42:08.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146361+0000", 2024-09-17T22:42:08.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0219999999999993e-06 2024-09-17T22:42:08.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188199+0000", 2024-09-17T22:42:08.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041838130000000001 2024-09-17T22:42:08.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188285+0000", 2024-09-17T22:42:08.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5160000000000005e-05 2024-09-17T22:42:08.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188379+0000", 2024-09-17T22:42:08.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3949000000000005e-05 2024-09-17T22:42:08.150 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.150 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.151 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.151 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157233 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4085760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.151 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:08.151 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.990964978000001, 2024-09-17T22:42:08.151 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042008960999999997, 2024-09-17T22:42:08.151 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.151 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.151 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.151 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.151 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.151 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.152 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157233 2024-09-17T22:42:08.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.152 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:08.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:08.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146561+0000", 2024-09-17T22:42:08.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6300000000000005e-07 2024-09-17T22:42:08.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146564+0000", 2024-09-17T22:42:08.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-17T22:42:08.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.153 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146575+0000", 2024-09-17T22:42:08.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0891e-05 2024-09-17T22:42:08.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146579+0000", 2024-09-17T22:42:08.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.284e-06 2024-09-17T22:42:08.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188407+0000", 2024-09-17T22:42:08.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041828390999999999 2024-09-17T22:42:08.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188476+0000", 2024-09-17T22:42:08.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9129999999999997e-05 2024-09-17T22:42:08.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188569+0000", 2024-09-17T22:42:08.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2669000000000003e-05 2024-09-17T22:42:08.155 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.155 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.155 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.155 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157235 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4091904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.155 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:08.156 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.990770981000001, 2024-09-17T22:42:08.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042047517, 2024-09-17T22:42:08.156 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.156 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.156 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.156 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.156 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.156 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.156 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157235 2024-09-17T22:42:08.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.156 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:08.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:08.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146755+0000", 2024-09-17T22:42:08.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0900000000000001e-07 2024-09-17T22:42:08.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146758+0000", 2024-09-17T22:42:08.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.399e-06 2024-09-17T22:42:08.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146768+0000", 2024-09-17T22:42:08.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4730000000000006e-06 2024-09-17T22:42:08.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146774+0000", 2024-09-17T22:42:08.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6939999999999998e-06 2024-09-17T22:42:08.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188616+0000", 2024-09-17T22:42:08.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041841481999999999 2024-09-17T22:42:08.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188700+0000", 2024-09-17T22:42:08.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4227999999999994e-05 2024-09-17T22:42:08.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188802+0000", 2024-09-17T22:42:08.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101532 2024-09-17T22:42:08.160 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.160 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.160 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.160 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157237 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4098048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.160 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:08.160 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.990740710000001, 2024-09-17T22:42:08.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165452999999999, 2024-09-17T22:42:08.160 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.160 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.160 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.161 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.161 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.161 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.161 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157237 2024-09-17T22:42:08.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.161 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:08.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:08.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146785+0000", 2024-09-17T22:42:08.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6800000000000001e-07 2024-09-17T22:42:08.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146787+0000", 2024-09-17T22:42:08.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.046e-06 2024-09-17T22:42:08.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146794+0000", 2024-09-17T22:42:08.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9249999999999998e-06 2024-09-17T22:42:08.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146799+0000", 2024-09-17T22:42:08.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9289999999999997e-06 2024-09-17T22:42:08.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188844+0000", 2024-09-17T22:42:08.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042045386999999997 2024-09-17T22:42:08.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188888+0000", 2024-09-17T22:42:08.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3760000000000001e-05 2024-09-17T22:42:08.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188950+0000", 2024-09-17T22:42:08.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1938000000000002e-05 2024-09-17T22:42:08.165 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.165 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.165 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.166 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157239 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4104192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.166 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:08.166 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.990712324, 2024-09-17T22:42:08.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042347193999999998, 2024-09-17T22:42:08.166 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.166 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.166 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.166 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.166 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.166 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.166 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157239 2024-09-17T22:42:08.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.166 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:08.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:08.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:08.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:42:08.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146816+0000", 2024-09-17T22:42:08.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4700000000000001e-06 2024-09-17T22:42:08.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146822+0000", 2024-09-17T22:42:08.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.579e-06 2024-09-17T22:42:08.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146826+0000", 2024-09-17T22:42:08.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3780000000000001e-06 2024-09-17T22:42:08.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188991+0000", 2024-09-17T22:42:08.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165553000000001 2024-09-17T22:42:08.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189068+0000", 2024-09-17T22:42:08.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6548e-05 2024-09-17T22:42:08.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189160+0000", 2024-09-17T22:42:08.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2163000000000005e-05 2024-09-17T22:42:08.170 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.170 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.170 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.170 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157242 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4110336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.170 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:08.170 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.990674538, 2024-09-17T22:42:08.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042499783999999999, 2024-09-17T22:42:08.171 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.171 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.171 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.171 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.171 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.171 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.171 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157242 2024-09-17T22:42:08.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.171 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:08.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:08.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:08.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:42:08.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146852+0000", 2024-09-17T22:42:08.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.37e-06 2024-09-17T22:42:08.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146856+0000", 2024-09-17T22:42:08.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1690000000000002e-06 2024-09-17T22:42:08.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146863+0000", 2024-09-17T22:42:08.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9920000000000001e-06 2024-09-17T22:42:08.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189193+0000", 2024-09-17T22:42:08.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042329355999999999 2024-09-17T22:42:08.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189246+0000", 2024-09-17T22:42:08.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3643000000000001e-05 2024-09-17T22:42:08.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189350+0000", 2024-09-17T22:42:08.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010394 2024-09-17T22:42:08.175 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.175 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.175 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.176 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157244 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4116480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.176 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:08.176 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.990655326999999, 2024-09-17T22:42:08.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042675684999999998, 2024-09-17T22:42:08.176 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.176 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.176 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.176 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.176 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.176 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.176 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157244 2024-09-17T22:42:08.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.177 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:08.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:08.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:08.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1e-07 2024-09-17T22:42:08.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146871+0000", 2024-09-17T22:42:08.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.189e-06 2024-09-17T22:42:08.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146877+0000", 2024-09-17T22:42:08.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8100000000000003e-06 2024-09-17T22:42:08.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146880+0000", 2024-09-17T22:42:08.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.83e-06 2024-09-17T22:42:08.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189388+0000", 2024-09-17T22:42:08.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042507637000000001 2024-09-17T22:42:08.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189443+0000", 2024-09-17T22:42:08.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5742999999999998e-05 2024-09-17T22:42:08.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189545+0000", 2024-09-17T22:42:08.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102166 2024-09-17T22:42:08.180 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.180 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.180 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.180 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157246 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4122624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.180 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:08.180 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.990626275, 2024-09-17T22:42:08.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042847083000000001, 2024-09-17T22:42:08.181 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.181 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.181 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.181 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.181 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.181 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.181 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157246 2024-09-17T22:42:08.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.181 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:08.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:08.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:08.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0100000000000001e-07 2024-09-17T22:42:08.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146901+0000", 2024-09-17T22:42:08.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.531e-06 2024-09-17T22:42:08.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146905+0000", 2024-09-17T22:42:08.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0759999999999996e-06 2024-09-17T22:42:08.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146908+0000", 2024-09-17T22:42:08.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.083e-06 2024-09-17T22:42:08.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189585+0000", 2024-09-17T22:42:08.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042677009000000002 2024-09-17T22:42:08.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189656+0000", 2024-09-17T22:42:08.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0712999999999999e-05 2024-09-17T22:42:08.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189746+0000", 2024-09-17T22:42:08.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0370000000000004e-05 2024-09-17T22:42:08.184 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.185 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.185 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.185 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157247 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4128768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.185 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:08.185 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.990554439, 2024-09-17T22:42:08.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043016916000000002, 2024-09-17T22:42:08.185 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.185 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.185 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.186 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.186 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.186 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.186 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157247 2024-09-17T22:42:08.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.186 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:08.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:08.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.187 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147071+0000", 2024-09-17T22:42:08.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100769 2024-09-17T22:42:08.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.187 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147081+0000", 2024-09-17T22:42:08.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9229999999999997e-06 2024-09-17T22:42:08.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147090+0000", 2024-09-17T22:42:08.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.935e-06 2024-09-17T22:42:08.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147097+0000", 2024-09-17T22:42:08.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T22:42:08.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189787+0000", 2024-09-17T22:42:08.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042690525 2024-09-17T22:42:08.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189884+0000", 2024-09-17T22:42:08.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6858e-05 2024-09-17T22:42:08.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189988+0000", 2024-09-17T22:42:08.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103336 2024-09-17T22:42:08.190 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.190 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.190 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.190 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157250 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4134912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.190 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:08.190 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.990410978, 2024-09-17T22:42:08.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043099446, 2024-09-17T22:42:08.191 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.191 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.191 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.191 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.191 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.191 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.191 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157250 2024-09-17T22:42:08.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.191 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:08.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:08.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147235+0000", 2024-09-17T22:42:08.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120975 2024-09-17T22:42:08.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147238+0000", 2024-09-17T22:42:08.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5289999999999998e-06 2024-09-17T22:42:08.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147244+0000", 2024-09-17T22:42:08.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6050000000000003e-06 2024-09-17T22:42:08.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147249+0000", 2024-09-17T22:42:08.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4939999999999997e-06 2024-09-17T22:42:08.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190020+0000", 2024-09-17T22:42:08.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042771300999999998 2024-09-17T22:42:08.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190102+0000", 2024-09-17T22:42:08.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1600000000000005e-05 2024-09-17T22:42:08.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190214+0000", 2024-09-17T22:42:08.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011194200000000001 2024-09-17T22:42:08.195 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.195 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.195 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.195 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157252 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4141056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.195 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:08.196 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.990268309000001, 2024-09-17T22:42:08.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043165508999999998, 2024-09-17T22:42:08.196 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.196 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.196 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.196 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.196 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.196 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.196 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157252 2024-09-17T22:42:08.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.196 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:08.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:08.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147402+0000", 2024-09-17T22:42:08.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014475600000000001 2024-09-17T22:42:08.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147411+0000", 2024-09-17T22:42:08.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4350000000000003e-06 2024-09-17T22:42:08.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147420+0000", 2024-09-17T22:42:08.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4790000000000002e-06 2024-09-17T22:42:08.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147427+0000", 2024-09-17T22:42:08.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5400000000000001e-06 2024-09-17T22:42:08.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190254+0000", 2024-09-17T22:42:08.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042826731 2024-09-17T22:42:08.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190326+0000", 2024-09-17T22:42:08.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1798999999999995e-05 2024-09-17T22:42:08.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.200 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190422+0000", 2024-09-17T22:42:08.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6768999999999995e-05 2024-09-17T22:42:08.200 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.200 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.200 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.200 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157253 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4147200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.200 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:08.200 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.990081943, 2024-09-17T22:42:08.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043157752000000001, 2024-09-17T22:42:08.200 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.200 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.201 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.201 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.201 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.201 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.201 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157253 2024-09-17T22:42:08.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.201 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.201 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:08.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:08.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147590+0000", 2024-09-17T22:42:08.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014709 2024-09-17T22:42:08.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147593+0000", 2024-09-17T22:42:08.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5840000000000002e-06 2024-09-17T22:42:08.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147598+0000", 2024-09-17T22:42:08.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2469999999999999e-06 2024-09-17T22:42:08.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147602+0000", 2024-09-17T22:42:08.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0049999999999998e-06 2024-09-17T22:42:08.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190475+0000", 2024-09-17T22:42:08.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872767999999999 2024-09-17T22:42:08.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190528+0000", 2024-09-17T22:42:08.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2889999999999997e-05 2024-09-17T22:42:08.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.205 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.205 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190601+0000", 2024-09-17T22:42:08.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3168000000000001e-05 2024-09-17T22:42:08.205 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.205 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.205 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.206 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157255 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4153344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.206 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:08.206 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.989915962000001, 2024-09-17T22:42:08.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043132251000000003, 2024-09-17T22:42:08.206 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.207 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.207 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.207 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.207 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.207 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.207 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157255 2024-09-17T22:42:08.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.207 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:08.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:08.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147820+0000", 2024-09-17T22:42:08.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021123600000000001 2024-09-17T22:42:08.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147824+0000", 2024-09-17T22:42:08.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6780000000000002e-06 2024-09-17T22:42:08.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147833+0000", 2024-09-17T22:42:08.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4250000000000004e-06 2024-09-17T22:42:08.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147842+0000", 2024-09-17T22:42:08.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0679999999999997e-06 2024-09-17T22:42:08.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190632+0000", 2024-09-17T22:42:08.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042790863999999998 2024-09-17T22:42:08.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190674+0000", 2024-09-17T22:42:08.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1804000000000002e-05 2024-09-17T22:42:08.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190741+0000", 2024-09-17T22:42:08.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7175999999999995e-05 2024-09-17T22:42:08.211 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.211 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.211 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.211 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157257 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4159488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.212 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:08.212 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.989675471999998, 2024-09-17T22:42:08.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.04307035, 2024-09-17T22:42:08.212 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.212 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.212 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.212 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.212 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.212 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.212 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157257 2024-09-17T22:42:08.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.212 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:08.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:08.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148844+0000", 2024-09-17T22:42:08.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00099455599999999996 2024-09-17T22:42:08.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148848+0000", 2024-09-17T22:42:08.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0180000000000003e-06 2024-09-17T22:42:08.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148858+0000", 2024-09-17T22:42:08.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.825e-06 2024-09-17T22:42:08.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148865+0000", 2024-09-17T22:42:08.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8290000000000002e-06 2024-09-17T22:42:08.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190789+0000", 2024-09-17T22:42:08.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041924630999999997 2024-09-17T22:42:08.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190860+0000", 2024-09-17T22:42:08.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0557999999999998e-05 2024-09-17T22:42:08.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190920+0000", 2024-09-17T22:42:08.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9932999999999997e-05 2024-09-17T22:42:08.216 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.216 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.216 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.216 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157259 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4165632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.216 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:08.216 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.988649453000001, 2024-09-17T22:42:08.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042261411999999998, 2024-09-17T22:42:08.216 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.217 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.217 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.217 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.217 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.217 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.217 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157259 2024-09-17T22:42:08.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.217 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:08.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:08.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:08.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:42:08.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148878+0000", 2024-09-17T22:42:08.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:42:08.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148885+0000", 2024-09-17T22:42:08.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7290000000000003e-06 2024-09-17T22:42:08.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148893+0000", 2024-09-17T22:42:08.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8110000000000003e-06 2024-09-17T22:42:08.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190967+0000", 2024-09-17T22:42:08.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042073948999999999 2024-09-17T22:42:08.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191069+0000", 2024-09-17T22:42:08.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102075 2024-09-17T22:42:08.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191137+0000", 2024-09-17T22:42:08.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8281000000000006e-05 2024-09-17T22:42:08.220 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.220 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.221 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.221 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157261 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4171776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.221 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:08.221 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.988599846, 2024-09-17T22:42:08.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042437255, 2024-09-17T22:42:08.221 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.221 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.221 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.221 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.221 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.221 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.222 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157261 2024-09-17T22:42:08.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.222 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:08.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:08.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148926+0000", 2024-09-17T22:42:08.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T22:42:08.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148928+0000", 2024-09-17T22:42:08.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4329999999999998e-06 2024-09-17T22:42:08.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148936+0000", 2024-09-17T22:42:08.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6969999999999993e-06 2024-09-17T22:42:08.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148942+0000", 2024-09-17T22:42:08.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9479999999999996e-06 2024-09-17T22:42:08.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191185+0000", 2024-09-17T22:42:08.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042243147000000002 2024-09-17T22:42:08.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191266+0000", 2024-09-17T22:42:08.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1186000000000005e-05 2024-09-17T22:42:08.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191362+0000", 2024-09-17T22:42:08.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6384000000000004e-05 2024-09-17T22:42:08.225 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.225 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.225 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.226 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157263 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4177920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.226 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:08.226 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.988575446999999, 2024-09-17T22:42:08.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042636395000000001, 2024-09-17T22:42:08.226 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.226 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.226 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.226 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.226 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.226 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.226 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157263 2024-09-17T22:42:08.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.226 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:08.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:08.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:08.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.37e-07 2024-09-17T22:42:08.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148952+0000", 2024-09-17T22:42:08.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.88e-06 2024-09-17T22:42:08.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148959+0000", 2024-09-17T22:42:08.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5649999999999998e-06 2024-09-17T22:42:08.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148963+0000", 2024-09-17T22:42:08.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8570000000000003e-06 2024-09-17T22:42:08.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191408+0000", 2024-09-17T22:42:08.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042444707999999998 2024-09-17T22:42:08.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191489+0000", 2024-09-17T22:42:08.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0612999999999995e-05 2024-09-17T22:42:08.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191586+0000", 2024-09-17T22:42:08.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7235e-05 2024-09-17T22:42:08.233 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.233 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.233 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.234 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157265 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4184064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.234 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:08.234 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.988553897999999, 2024-09-17T22:42:08.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042791830000000003, 2024-09-17T22:42:08.235 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.235 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.235 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.235 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.235 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.235 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.236 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157265 2024-09-17T22:42:08.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.236 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:08.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:08.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148972+0000", 2024-09-17T22:42:08.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:42:08.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148973+0000", 2024-09-17T22:42:08.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.894e-06 2024-09-17T22:42:08.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148985+0000", 2024-09-17T22:42:08.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1995e-05 2024-09-17T22:42:08.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148990+0000", 2024-09-17T22:42:08.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8269999999999997e-06 2024-09-17T22:42:08.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191630+0000", 2024-09-17T22:42:08.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042639639 2024-09-17T22:42:08.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191686+0000", 2024-09-17T22:42:08.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6066000000000002e-05 2024-09-17T22:42:08.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191763+0000", 2024-09-17T22:42:08.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7035999999999999e-05 2024-09-17T22:42:08.239 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.239 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.240 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.240 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157267 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4190208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:08.240 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:08.240 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.988528084999999, 2024-09-17T22:42:08.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043028813999999999, 2024-09-17T22:42:08.240 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.240 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.240 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.240 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157267 2024-09-17T22:42:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:42:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148999+0000", 2024-09-17T22:42:08.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.432e-06 2024-09-17T22:42:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149002+0000", 2024-09-17T22:42:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7850000000000002e-06 2024-09-17T22:42:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149005+0000", 2024-09-17T22:42:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:42:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191878+0000", 2024-09-17T22:42:08.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872312000000003 2024-09-17T22:42:08.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191961+0000", 2024-09-17T22:42:08.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3418000000000007e-05 2024-09-17T22:42:08.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.192026+0000", 2024-09-17T22:42:08.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4689000000000004e-05 2024-09-17T22:42:08.244 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.244 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.244 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.245 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.245 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.245 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:08.246 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:42:08.247 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:08.243+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:08.347 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:08.344+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:08.352 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:08.353 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:08.353 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:08.353 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:08.363 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:42:08.368 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-17T22:42:08.370 INFO:teuthology.orchestra.run.smithi115.stdout:2599 writesame 0x6f31ebb thru 0x6f37ff9 data_size 0xd(0x613f bytes) 2024-09-17T22:42:08.391 INFO:teuthology.orchestra.run.smithi115.stdout:2601 read 0xf0a8bc thru 0xf18e3c (0xe581 bytes) 2024-09-17T22:42:08.391 INFO:teuthology.orchestra.run.smithi115.stdout:2603 read 0x6bcf115 thru 0x6bdb0ac (0xbf98 bytes) 2024-09-17T22:42:08.392 INFO:teuthology.orchestra.run.smithi115.stdout:2605 read 0x9c2f8c4 thru 0x9c31c58 (0x2395 bytes) 2024-09-17T22:42:08.392 INFO:teuthology.orchestra.run.smithi115.stdout:2606 writesame 0x3eba1e1 thru 0x3ec1bbb data_size 0x11(0x79db bytes) 2024-09-17T22:42:08.417 INFO:teuthology.orchestra.run.smithi115.stdout:2607 trunc from 0xc06ce12 to 0xdb7b825 2024-09-17T22:42:08.419 INFO:teuthology.orchestra.run.smithi115.stdout:2608 read 0x912184b thru 0x9124275 (0x2a2b bytes) 2024-09-17T22:42:08.419 INFO:teuthology.orchestra.run.smithi115.stdout:2609 write 0x67a956c thru 0x67b6bfe (0xd693 bytes) 2024-09-17T22:42:08.447 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:08.445+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:08.504 INFO:teuthology.orchestra.run.smithi115.stdout:2610 trunc from 0xdb7b825 to 0x5cf4a09 2024-09-17T22:42:08.528 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:08.528 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:08.528 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:08.528 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:08.528 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:08.538 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:42:08.579 INFO:teuthology.orchestra.run.smithi115.stdout:2611 read 0x1aab588 thru 0x1ab7044 (0xbabd bytes) 2024-09-17T22:42:08.582 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:08.580+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:08.595 INFO:teuthology.orchestra.run.smithi115.stdout:2613 write 0xd50026e thru 0xd503689 (0x341c bytes) 2024-09-17T22:42:08.602 INFO:teuthology.orchestra.run.smithi115.stdout:2617 write 0xa2df5c5 thru 0xa2e5235 (0x5c71 bytes) 2024-09-17T22:42:08.612 INFO:teuthology.orchestra.run.smithi115.stdout:2619 read 0x7f66d4d thru 0x7f73db1 (0xd065 bytes) 2024-09-17T22:42:08.613 INFO:teuthology.orchestra.run.smithi115.stdout:2621 write 0x5b286db thru 0x5b312a2 (0x8bc8 bytes) 2024-09-17T22:42:08.614 INFO:teuthology.orchestra.run.smithi115.stdout:2623 writesame 0xb69df06 thru 0xb6a0bfd data_size 0x4(0x2cf8 bytes) 2024-09-17T22:42:08.620 INFO:teuthology.orchestra.run.smithi115.stdout:2624 punch from 0x8be3b73 to 0x8bea764, (0x6bf1 bytes) 2024-09-17T22:42:08.626 INFO:teuthology.orchestra.run.smithi115.stdout:2625 writesame 0xe941f9b thru 0xe944ba7 data_size 0x15(0x2c0d bytes) 2024-09-17T22:42:08.626 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:08.626 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:08.626 INFO:teuthology.orchestra.run.smithi115.stdout:2628 write 0x37d0500 thru 0x37dca16 (0xc517 bytes) 2024-09-17T22:42:08.649 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:08.647+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:08.650 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:08.655 INFO:teuthology.orchestra.run.smithi115.stdout:2630 trunc from 0xe944ba8 to 0xb06acff 2024-09-17T22:42:08.656 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38083 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.763873+0000", 2024-09-17T22:42:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.889126388999998, 2024-09-17T22:42:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023344747999999999, 2024-09-17T22:42:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38083 2024-09-17T22:42:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763873+0000", 2024-09-17T22:42:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763873+0000", 2024-09-17T22:42:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763874+0000", 2024-09-17T22:42:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0899999999999997e-07 2024-09-17T22:42:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763875+0000", 2024-09-17T22:42:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8530000000000001e-06 2024-09-17T22:42:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763881+0000", 2024-09-17T22:42:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4879999999999998e-06 2024-09-17T22:42:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763886+0000", 2024-09-17T22:42:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.587e-06 2024-09-17T22:42:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787057+0000", 2024-09-17T22:42:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023170611000000001 2024-09-17T22:42:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787127+0000", 2024-09-17T22:42:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0259000000000006e-05 2024-09-17T22:42:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787218+0000", 2024-09-17T22:42:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0641000000000002e-05 2024-09-17T22:42:08.661 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.661 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.661 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.661 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38105 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 159744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:08.661 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.763938+0000", 2024-09-17T22:42:08.661 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.889061581999997, 2024-09-17T22:42:08.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023471315999999999, 2024-09-17T22:42:08.661 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.661 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.661 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.662 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.662 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.662 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38105 2024-09-17T22:42:08.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.662 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763938+0000", 2024-09-17T22:42:08.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763938+0000", 2024-09-17T22:42:08.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763939+0000", 2024-09-17T22:42:08.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1200000000000003e-07 2024-09-17T22:42:08.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763941+0000", 2024-09-17T22:42:08.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9489999999999999e-06 2024-09-17T22:42:08.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763946+0000", 2024-09-17T22:42:08.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9139999999999996e-06 2024-09-17T22:42:08.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763951+0000", 2024-09-17T22:42:08.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6589999999999998e-06 2024-09-17T22:42:08.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787249+0000", 2024-09-17T22:42:08.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023297989000000002 2024-09-17T22:42:08.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787319+0000", 2024-09-17T22:42:08.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9836e-05 2024-09-17T22:42:08.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.665 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787409+0000", 2024-09-17T22:42:08.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0457000000000005e-05 2024-09-17T22:42:08.665 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.665 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.666 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.666 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38107 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:08.666 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:42:08.666 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.889042793000002, 2024-09-17T22:42:08.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023669550000000001, 2024-09-17T22:42:08.666 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.666 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.667 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.667 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.667 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.667 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.667 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38107 2024-09-17T22:42:08.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.667 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:42:08.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:42:08.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763957+0000", 2024-09-17T22:42:08.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:42:08.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763959+0000", 2024-09-17T22:42:08.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.849e-06 2024-09-17T22:42:08.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763965+0000", 2024-09-17T22:42:08.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6110000000000001e-06 2024-09-17T22:42:08.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.763969+0000", 2024-09-17T22:42:08.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5179999999999998e-06 2024-09-17T22:42:08.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787451+0000", 2024-09-17T22:42:08.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023482281000000001 2024-09-17T22:42:08.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787516+0000", 2024-09-17T22:42:08.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4930000000000003e-05 2024-09-17T22:42:08.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787626+0000", 2024-09-17T22:42:08.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011000700000000001 2024-09-17T22:42:08.670 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.670 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.670 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.671 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38141 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 292864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:08.671 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764408+0000", 2024-09-17T22:42:08.671 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.888591560999998, 2024-09-17T22:42:08.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023422268, 2024-09-17T22:42:08.671 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.671 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.671 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.671 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.671 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.671 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.671 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38141 2024-09-17T22:42:08.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.672 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764408+0000", 2024-09-17T22:42:08.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764408+0000", 2024-09-17T22:42:08.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764409+0000", 2024-09-17T22:42:08.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5400000000000001e-07 2024-09-17T22:42:08.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764412+0000", 2024-09-17T22:42:08.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4470000000000002e-06 2024-09-17T22:42:08.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764421+0000", 2024-09-17T22:42:08.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7299999999999994e-06 2024-09-17T22:42:08.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764427+0000", 2024-09-17T22:42:08.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.392e-06 2024-09-17T22:42:08.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.674 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787669+0000", 2024-09-17T22:42:08.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023241385 2024-09-17T22:42:08.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.674 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787743+0000", 2024-09-17T22:42:08.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4260999999999999e-05 2024-09-17T22:42:08.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787830+0000", 2024-09-17T22:42:08.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7398999999999994e-05 2024-09-17T22:42:08.675 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.675 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.675 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.675 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38143 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 348160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:08.676 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:42:08.676 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.888561633000002, 2024-09-17T22:42:08.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023565691, 2024-09-17T22:42:08.676 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.676 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.676 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.676 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.676 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.676 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.676 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38143 2024-09-17T22:42:08.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.676 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:42:08.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:42:08.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764438+0000", 2024-09-17T22:42:08.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6199999999999998e-07 2024-09-17T22:42:08.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764441+0000", 2024-09-17T22:42:08.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1179999999999999e-06 2024-09-17T22:42:08.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764447+0000", 2024-09-17T22:42:08.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1940000000000003e-06 2024-09-17T22:42:08.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764452+0000", 2024-09-17T22:42:08.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9150000000000004e-06 2024-09-17T22:42:08.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787881+0000", 2024-09-17T22:42:08.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023429141000000001 2024-09-17T22:42:08.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.787947+0000", 2024-09-17T22:42:08.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5902999999999996e-05 2024-09-17T22:42:08.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788004+0000", 2024-09-17T22:42:08.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6957999999999999e-05 2024-09-17T22:42:08.680 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.680 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.680 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.680 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38150 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:08.680 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:42:08.680 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.888541631000002, 2024-09-17T22:42:08.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023741399, 2024-09-17T22:42:08.680 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.681 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.681 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.681 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.681 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.681 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.681 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38150 2024-09-17T22:42:08.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.681 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:42:08.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:42:08.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764458+0000", 2024-09-17T22:42:08.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8299999999999998e-07 2024-09-17T22:42:08.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764460+0000", 2024-09-17T22:42:08.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.855e-06 2024-09-17T22:42:08.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764466+0000", 2024-09-17T22:42:08.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8200000000000002e-06 2024-09-17T22:42:08.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764471+0000", 2024-09-17T22:42:08.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8319999999999996e-06 2024-09-17T22:42:08.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.683 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788032+0000", 2024-09-17T22:42:08.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023561480999999999 2024-09-17T22:42:08.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788106+0000", 2024-09-17T22:42:08.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3406999999999997e-05 2024-09-17T22:42:08.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788199+0000", 2024-09-17T22:42:08.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3621000000000004e-05 2024-09-17T22:42:08.684 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.684 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.684 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.685 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38152 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 83968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:08.685 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:42:08.685 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.888522913999999, 2024-09-17T22:42:08.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023859726000000001, 2024-09-17T22:42:08.685 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.685 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.685 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.685 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.685 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.685 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.685 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38152 2024-09-17T22:42:08.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.686 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:42:08.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:42:08.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764477+0000", 2024-09-17T22:42:08.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3000000000000001e-07 2024-09-17T22:42:08.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764479+0000", 2024-09-17T22:42:08.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8920000000000001e-06 2024-09-17T22:42:08.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764485+0000", 2024-09-17T22:42:08.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9920000000000004e-06 2024-09-17T22:42:08.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764490+0000", 2024-09-17T22:42:08.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6890000000000003e-06 2024-09-17T22:42:08.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788232+0000", 2024-09-17T22:42:08.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023742452000000001 2024-09-17T22:42:08.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788276+0000", 2024-09-17T22:42:08.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3689000000000001e-05 2024-09-17T22:42:08.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788336+0000", 2024-09-17T22:42:08.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0581999999999997e-05 2024-09-17T22:42:08.689 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.689 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.689 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.689 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38184 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 161792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:08.689 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764589+0000", 2024-09-17T22:42:08.689 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.888410520999997, 2024-09-17T22:42:08.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023956801, 2024-09-17T22:42:08.690 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.690 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.690 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.690 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.690 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38184 2024-09-17T22:42:08.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.690 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764589+0000", 2024-09-17T22:42:08.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764589+0000", 2024-09-17T22:42:08.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764590+0000", 2024-09-17T22:42:08.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7800000000000001e-07 2024-09-17T22:42:08.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764592+0000", 2024-09-17T22:42:08.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6929999999999999e-06 2024-09-17T22:42:08.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764600+0000", 2024-09-17T22:42:08.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1140000000000006e-06 2024-09-17T22:42:08.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764606+0000", 2024-09-17T22:42:08.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6200000000000004e-06 2024-09-17T22:42:08.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788374+0000", 2024-09-17T22:42:08.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023768083999999998 2024-09-17T22:42:08.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788447+0000", 2024-09-17T22:42:08.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2922000000000004e-05 2024-09-17T22:42:08.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788546+0000", 2024-09-17T22:42:08.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8789999999999994e-05 2024-09-17T22:42:08.694 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.694 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.694 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.694 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38186 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:08.694 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:42:08.694 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.888385571000001, 2024-09-17T22:42:08.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024156486000000001, 2024-09-17T22:42:08.694 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.694 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.694 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.695 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.695 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.695 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38186 2024-09-17T22:42:08.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.695 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:42:08.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:42:08.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764614+0000", 2024-09-17T22:42:08.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:42:08.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764621+0000", 2024-09-17T22:42:08.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6409999999999996e-06 2024-09-17T22:42:08.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764628+0000", 2024-09-17T22:42:08.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0360000000000001e-06 2024-09-17T22:42:08.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764633+0000", 2024-09-17T22:42:08.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0300000000000001e-06 2024-09-17T22:42:08.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788594+0000", 2024-09-17T22:42:08.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023960921999999999 2024-09-17T22:42:08.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788676+0000", 2024-09-17T22:42:08.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2123e-05 2024-09-17T22:42:08.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788771+0000", 2024-09-17T22:42:08.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4284999999999996e-05 2024-09-17T22:42:08.698 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.698 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.698 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.698 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38218 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 350208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:08.699 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764706+0000", 2024-09-17T22:42:08.699 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.888293318000002, 2024-09-17T22:42:08.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024324732000000002, 2024-09-17T22:42:08.699 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.699 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.699 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.699 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.699 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38218 2024-09-17T22:42:08.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.699 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764706+0000", 2024-09-17T22:42:08.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764706+0000", 2024-09-17T22:42:08.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764707+0000", 2024-09-17T22:42:08.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7199999999999999e-07 2024-09-17T22:42:08.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764710+0000", 2024-09-17T22:42:08.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6819999999999999e-06 2024-09-17T22:42:08.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764716+0000", 2024-09-17T22:42:08.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7900000000000002e-06 2024-09-17T22:42:08.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764722+0000", 2024-09-17T22:42:08.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2560000000000002e-06 2024-09-17T22:42:08.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788867+0000", 2024-09-17T22:42:08.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024144982999999998 2024-09-17T22:42:08.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.788941+0000", 2024-09-17T22:42:08.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4529000000000004e-05 2024-09-17T22:42:08.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789031+0000", 2024-09-17T22:42:08.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9920000000000006e-05 2024-09-17T22:42:08.703 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.703 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.703 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.703 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38222 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 30720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:08.703 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.764765+0000", 2024-09-17T22:42:08.703 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.888234464, 2024-09-17T22:42:08.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024443910999999999, 2024-09-17T22:42:08.703 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.704 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.704 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.704 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.704 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.704 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.704 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38222 2024-09-17T22:42:08.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.704 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764765+0000", 2024-09-17T22:42:08.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764765+0000", 2024-09-17T22:42:08.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764766+0000", 2024-09-17T22:42:08.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8999999999999996e-07 2024-09-17T22:42:08.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764769+0000", 2024-09-17T22:42:08.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7659999999999999e-06 2024-09-17T22:42:08.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764776+0000", 2024-09-17T22:42:08.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4229999999999999e-06 2024-09-17T22:42:08.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.764783+0000", 2024-09-17T22:42:08.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9600000000000003e-06 2024-09-17T22:42:08.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789060+0000", 2024-09-17T22:42:08.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024277343 2024-09-17T22:42:08.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789129+0000", 2024-09-17T22:42:08.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8367999999999993e-05 2024-09-17T22:42:08.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.789209+0000", 2024-09-17T22:42:08.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0461000000000001e-05 2024-09-17T22:42:08.707 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.708 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.708 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.708 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38671 3.12 3:4a5b92ac:::rbd_data.115fe0c30ef6.0000000000000011:5 [read 243712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:08.708 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778865+0000", 2024-09-17T22:42:08.708 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.874134404000003, 2024-09-17T22:42:08.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023227351, 2024-09-17T22:42:08.708 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.708 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.708 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.709 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38671 2024-09-17T22:42:08.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.709 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778865+0000", 2024-09-17T22:42:08.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778865+0000", 2024-09-17T22:42:08.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778866+0000", 2024-09-17T22:42:08.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0900000000000002e-07 2024-09-17T22:42:08.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778868+0000", 2024-09-17T22:42:08.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4339999999999999e-06 2024-09-17T22:42:08.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778875+0000", 2024-09-17T22:42:08.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4559999999999997e-06 2024-09-17T22:42:08.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778880+0000", 2024-09-17T22:42:08.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2689999999999999e-06 2024-09-17T22:42:08.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.801978+0000", 2024-09-17T22:42:08.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023098225999999999 2024-09-17T22:42:08.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802035+0000", 2024-09-17T22:42:08.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7123999999999997e-05 2024-09-17T22:42:08.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802093+0000", 2024-09-17T22:42:08.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7432999999999997e-05 2024-09-17T22:42:08.712 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.712 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.712 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.712 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38673 3.12 3:4a5b92ac:::rbd_data.115fe0c30ef6.0000000000000011:5 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:08.713 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778886+0000", 2024-09-17T22:42:08.713 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.874113256000001, 2024-09-17T22:42:08.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023389259999999999, 2024-09-17T22:42:08.713 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.713 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.713 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.713 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.713 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.713 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.713 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38673 2024-09-17T22:42:08.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.713 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778886+0000", 2024-09-17T22:42:08.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778886+0000", 2024-09-17T22:42:08.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778887+0000", 2024-09-17T22:42:08.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:42:08.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778889+0000", 2024-09-17T22:42:08.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9379999999999999e-06 2024-09-17T22:42:08.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778895+0000", 2024-09-17T22:42:08.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0360000000000003e-06 2024-09-17T22:42:08.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778900+0000", 2024-09-17T22:42:08.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1220000000000003e-06 2024-09-17T22:42:08.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802126+0000", 2024-09-17T22:42:08.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023226607 2024-09-17T22:42:08.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802208+0000", 2024-09-17T22:42:08.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1157999999999998e-05 2024-09-17T22:42:08.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802276+0000", 2024-09-17T22:42:08.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8122999999999998e-05 2024-09-17T22:42:08.717 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.717 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.717 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.717 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38675 3.12 3:4a5b92ac:::rbd_data.115fe0c30ef6.0000000000000011:5 [read 258048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:08.717 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:42:08.717 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.874078050999998, 2024-09-17T22:42:08.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023484221999999999, 2024-09-17T22:42:08.718 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.718 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.718 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.718 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.718 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.718 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.718 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38675 2024-09-17T22:42:08.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.718 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:42:08.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:42:08.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778922+0000", 2024-09-17T22:42:08.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9999999999999997e-07 2024-09-17T22:42:08.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778924+0000", 2024-09-17T22:42:08.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3039999999999999e-06 2024-09-17T22:42:08.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778931+0000", 2024-09-17T22:42:08.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9519999999999997e-06 2024-09-17T22:42:08.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778937+0000", 2024-09-17T22:42:08.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3639999999999998e-06 2024-09-17T22:42:08.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802300+0000", 2024-09-17T22:42:08.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023363544 2024-09-17T22:42:08.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802347+0000", 2024-09-17T22:42:08.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6897999999999997e-05 2024-09-17T22:42:08.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802406+0000", 2024-09-17T22:42:08.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8560000000000002e-05 2024-09-17T22:42:08.722 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.722 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.722 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.722 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38679 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:08.722 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778945+0000", 2024-09-17T22:42:08.722 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.874054434999998, 2024-09-17T22:42:08.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023578015000000001, 2024-09-17T22:42:08.722 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.722 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.722 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.722 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.722 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.723 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.723 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38679 2024-09-17T22:42:08.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.723 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778945+0000", 2024-09-17T22:42:08.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778945+0000", 2024-09-17T22:42:08.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778946+0000", 2024-09-17T22:42:08.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.44e-07 2024-09-17T22:42:08.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778948+0000", 2024-09-17T22:42:08.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2680000000000001e-06 2024-09-17T22:42:08.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778955+0000", 2024-09-17T22:42:08.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.528e-06 2024-09-17T22:42:08.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778960+0000", 2024-09-17T22:42:08.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1919999999999996e-06 2024-09-17T22:42:08.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802429+0000", 2024-09-17T22:42:08.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023469561999999999 2024-09-17T22:42:08.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802470+0000", 2024-09-17T22:42:08.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1068000000000001e-05 2024-09-17T22:42:08.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802523+0000", 2024-09-17T22:42:08.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2852999999999997e-05 2024-09-17T22:42:08.726 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.726 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.726 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.726 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38680 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:08.727 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.778966+0000", 2024-09-17T22:42:08.727 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.874033476999998, 2024-09-17T22:42:08.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023670668999999998, 2024-09-17T22:42:08.727 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.727 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.727 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.727 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.727 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.727 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.727 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38680 2024-09-17T22:42:08.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.727 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778966+0000", 2024-09-17T22:42:08.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778966+0000", 2024-09-17T22:42:08.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778967+0000", 2024-09-17T22:42:08.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.89e-07 2024-09-17T22:42:08.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778969+0000", 2024-09-17T22:42:08.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0219999999999999e-06 2024-09-17T22:42:08.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778975+0000", 2024-09-17T22:42:08.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3779999999999998e-06 2024-09-17T22:42:08.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.778980+0000", 2024-09-17T22:42:08.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0699999999999997e-06 2024-09-17T22:42:08.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802545+0000", 2024-09-17T22:42:08.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023565358000000002 2024-09-17T22:42:08.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802585+0000", 2024-09-17T22:42:08.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0136999999999999e-05 2024-09-17T22:42:08.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802637+0000", 2024-09-17T22:42:08.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1314999999999999e-05 2024-09-17T22:42:08.731 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.731 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.731 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.731 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38697 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 675840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:08.731 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.779074+0000", 2024-09-17T22:42:08.731 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.873925223999997, 2024-09-17T22:42:08.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023676402999999999, 2024-09-17T22:42:08.731 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.731 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.732 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.732 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.732 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.732 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.732 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38697 2024-09-17T22:42:08.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.732 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779074+0000", 2024-09-17T22:42:08.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779074+0000", 2024-09-17T22:42:08.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779075+0000", 2024-09-17T22:42:08.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8599999999999998e-07 2024-09-17T22:42:08.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779077+0000", 2024-09-17T22:42:08.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8810000000000001e-06 2024-09-17T22:42:08.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779084+0000", 2024-09-17T22:42:08.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.691e-06 2024-09-17T22:42:08.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779088+0000", 2024-09-17T22:42:08.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6970000000000001e-06 2024-09-17T22:42:08.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802659+0000", 2024-09-17T22:42:08.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023571214 2024-09-17T22:42:08.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802699+0000", 2024-09-17T22:42:08.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9762000000000003e-05 2024-09-17T22:42:08.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802751+0000", 2024-09-17T22:42:08.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1572e-05 2024-09-17T22:42:08.735 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.735 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.735 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.736 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38698 3.12 3:4e6a2123:::rbd_data.115f1ca54530.0000000000000001:4 [read 749568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:08.736 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.779094+0000", 2024-09-17T22:42:08.736 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.873905286999999, 2024-09-17T22:42:08.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023778845999999999, 2024-09-17T22:42:08.736 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.736 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.736 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:08.736 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.736 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.736 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.736 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38698 2024-09-17T22:42:08.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.737 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779094+0000", 2024-09-17T22:42:08.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779094+0000", 2024-09-17T22:42:08.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779095+0000", 2024-09-17T22:42:08.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8799999999999998e-07 2024-09-17T22:42:08.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779097+0000", 2024-09-17T22:42:08.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8530000000000001e-06 2024-09-17T22:42:08.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779103+0000", 2024-09-17T22:42:08.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0410000000000003e-06 2024-09-17T22:42:08.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.779107+0000", 2024-09-17T22:42:08.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5580000000000002e-06 2024-09-17T22:42:08.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802774+0000", 2024-09-17T22:42:08.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023666348 2024-09-17T22:42:08.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802814+0000", 2024-09-17T22:42:08.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0173000000000003e-05 2024-09-17T22:42:08.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.802873+0000", 2024-09-17T22:42:08.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9484999999999997e-05 2024-09-17T22:42:08.740 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.740 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.740 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.740 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:38843 3.12 3.c4845672 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:08.740 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:42:08.740 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.87135601, 2024-09-17T22:42:08.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023236265999999998, 2024-09-17T22:42:08.740 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.741 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.741 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:08.741 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.741 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.741 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.741 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 38843 2024-09-17T22:42:08.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.741 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:42:08.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:42:08.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781644+0000", 2024-09-17T22:42:08.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:42:08.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781646+0000", 2024-09-17T22:42:08.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9290000000000001e-06 2024-09-17T22:42:08.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781652+0000", 2024-09-17T22:42:08.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7389999999999996e-06 2024-09-17T22:42:08.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.781656+0000", 2024-09-17T22:42:08.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3830000000000002e-06 2024-09-17T22:42:08.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804871+0000", 2024-09-17T22:42:08.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023215257999999999 2024-09-17T22:42:08.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.804880+0000", 2024-09-17T22:42:08.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5769999999999994e-06 2024-09-17T22:42:08.744 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.744 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.744 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.745 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:180371 3.12 3:4ad9ab9d:::rbd_data.115f1f6bca8.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-17T22:42:08.745 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:03.397646+0000", 2024-09-17T22:42:08.745 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 5.2553533459999997, 2024-09-17T22:42:08.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025101674000000001, 2024-09-17T22:42:08.745 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:08.745 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:08.745 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:08.745 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:08.745 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:08.745 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:08.745 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 180371 2024-09-17T22:42:08.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.745 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:08.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:08.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:03.397646+0000", 2024-09-17T22:42:08.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:08.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:03.397646+0000", 2024-09-17T22:42:08.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:08.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:08.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:03.397647+0000", 2024-09-17T22:42:08.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4499999999999996e-07 2024-09-17T22:42:08.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:08.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:03.399208+0000", 2024-09-17T22:42:08.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.001560636 2024-09-17T22:42:08.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:08.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:03.399220+0000", 2024-09-17T22:42:08.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1779e-05 2024-09-17T22:42:08.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:08.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:03.399225+0000", 2024-09-17T22:42:08.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3369999999999999e-06 2024-09-17T22:42:08.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:08.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:03.399277+0000", 2024-09-17T22:42:08.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2154000000000002e-05 2024-09-17T22:42:08.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:08.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:03.399365+0000", 2024-09-17T22:42:08.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8467999999999994e-05 2024-09-17T22:42:08.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-17T22:42:08.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:03.408225+0000", 2024-09-17T22:42:08.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.008859291 2024-09-17T22:42:08.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:42:08.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:03.422601+0000", 2024-09-17T22:42:08.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.014375814000000001 2024-09-17T22:42:08.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:08.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:03.422642+0000", 2024-09-17T22:42:08.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1700999999999999e-05 2024-09-17T22:42:08.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:08.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:03.422686+0000", 2024-09-17T22:42:08.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3476000000000002e-05 2024-09-17T22:42:08.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:08.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:08.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:08.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:03.422748+0000", 2024-09-17T22:42:08.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2173000000000003e-05 2024-09-17T22:42:08.750 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.750 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.750 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.750 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:08.750 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:08.751 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:08.753 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:42:08.754 INFO:teuthology.orchestra.run.smithi115.stdout:2632 writesame 0xb9c3a01 thru 0xb9c51b1 data_size 0x1(0x17b1 bytes) 2024-09-17T22:42:08.754 INFO:teuthology.orchestra.run.smithi115.stdout:2633 writesame 0xb397dc8 thru 0xb39d00c data_size 0x1(0x5245 bytes) 2024-09-17T22:42:08.754 INFO:teuthology.orchestra.run.smithi115.stdout:2636 writesame 0x82bf1f6 thru 0x82c5883 data_size 0x1(0x668e bytes) 2024-09-17T22:42:08.754 INFO:teuthology.orchestra.run.smithi115.stdout:2637 write 0x4c3e28e thru 0x4c3e2dc (0x4f bytes) 2024-09-17T22:42:08.754 INFO:teuthology.orchestra.run.smithi115.stdout:2638 punch from 0x5eb5d9 to 0x5ed6a2, (0x20c9 bytes) 2024-09-17T22:42:08.754 INFO:teuthology.orchestra.run.smithi115.stdout:2639 write 0xa0f156f thru 0xa0fe6b4 (0xd146 bytes) 2024-09-17T22:42:08.754 INFO:teuthology.orchestra.run.smithi115.stdout:2641 punch from 0xaa8e0b6 to 0xaa96bfa, (0x8b44 bytes) 2024-09-17T22:42:08.754 INFO:teuthology.orchestra.run.smithi115.stdout:2642 punch from 0x2835b92 to 0x283be86, (0x62f4 bytes) 2024-09-17T22:42:08.754 INFO:teuthology.orchestra.run.smithi115.stdout:2643 read 0xa07f6fb thru 0xa0851af (0x5ab5 bytes) 2024-09-17T22:42:08.754 INFO:teuthology.orchestra.run.smithi115.stdout:2644 writesame 0x4ea5545 thru 0x4eae6bd data_size 0x1(0x9179 bytes) 2024-09-17T22:42:08.754 INFO:teuthology.orchestra.run.smithi115.stdout:2645 write 0x6f8d327 thru 0x6f8ed34 (0x1a0e bytes) 2024-09-17T22:42:08.755 INFO:teuthology.orchestra.run.smithi115.stdout:2646 write 0x95e2e9a thru 0x95e399c (0xb03 bytes) 2024-09-17T22:42:08.755 INFO:teuthology.orchestra.run.smithi115.stdout:2647 trunc from 0xb9c51b2 to 0xb1949f0 2024-09-17T22:42:08.755 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:08.749+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:08.755 INFO:teuthology.orchestra.run.smithi115.stdout:2648 read 0x6ca5410 thru 0x6ca5deb (0x9dc bytes) 2024-09-17T22:42:08.755 INFO:teuthology.orchestra.run.smithi115.stdout:2649 write 0xaff5467 thru 0xb0053f0 (0xff8a bytes) 2024-09-17T22:42:08.772 INFO:teuthology.orchestra.run.smithi115.stdout:2650 write 0xed5ae12 thru 0xed619a0 (0x6b8f bytes) 2024-09-17T22:42:08.777 INFO:teuthology.orchestra.run.smithi115.stdout:2651 write 0xa6afb66 thru 0xa6bc9a7 (0xce42 bytes) 2024-09-17T22:42:08.782 INFO:teuthology.orchestra.run.smithi115.stdout:2652 punch from 0x4e3601a to 0x4e3ab2b, (0x4b11 bytes) 2024-09-17T22:42:08.782 INFO:teuthology.orchestra.run.smithi115.stdout:2653 read 0xad10feb thru 0xad19a5c (0x8a72 bytes) 2024-09-17T22:42:08.783 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:08.783 INFO:teuthology.orchestra.run.smithi115.stdout:2658 read 0x51bf561 thru 0x51ce6cd (0xf16d bytes) 2024-09-17T22:42:08.783 INFO:teuthology.orchestra.run.smithi115.stdout:2659 read 0x315a4c7 thru 0x316a4b9 (0xfff3 bytes) 2024-09-17T22:42:08.784 INFO:teuthology.orchestra.run.smithi115.stdout:2660 write 0xd48a4c5 thru 0xd497d3d (0xd879 bytes) 2024-09-17T22:42:08.797 INFO:teuthology.orchestra.run.smithi115.stdout:2661 read 0xcc77282 thru 0xcc818c2 (0xa641 bytes) 2024-09-17T22:42:08.797 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:08.797 INFO:teuthology.orchestra.run.smithi115.stdout:2665 read 0xbedc36c thru 0xbee245e (0x60f3 bytes) 2024-09-17T22:42:08.797 INFO:teuthology.orchestra.run.smithi115.stdout:2667 punch from 0xaf743a4 to 0xaf78c97, (0x48f3 bytes) 2024-09-17T22:42:08.797 INFO:teuthology.orchestra.run.smithi115.stdout:2668 punch from 0x84dcce0 to 0x84ea37a, (0xd69a bytes) 2024-09-17T22:42:08.798 INFO:teuthology.orchestra.run.smithi115.stdout:2669 read 0x83988dc thru 0x83a35b3 (0xacd8 bytes) 2024-09-17T22:42:08.798 INFO:teuthology.orchestra.run.smithi115.stdout:2670 write 0x79e697a thru 0x79ecb4a (0x61d1 bytes) 2024-09-17T22:42:08.800 INFO:teuthology.orchestra.run.smithi115.stdout:2671 write 0x811f72f thru 0x812f55d (0xfe2f bytes) 2024-09-17T22:42:08.813 INFO:teuthology.orchestra.run.smithi115.stdout:2672 read 0x5f342a8 thru 0x5f3a17f (0x5ed8 bytes) 2024-09-17T22:42:08.814 INFO:teuthology.orchestra.run.smithi115.stdout:2674 write 0xe024bd1 thru 0xe02d4ee (0x891e bytes) 2024-09-17T22:42:08.825 INFO:teuthology.orchestra.run.smithi115.stdout:2675 read 0x1100a22 thru 0x110db2a (0xd109 bytes) 2024-09-17T22:42:08.825 INFO:teuthology.orchestra.run.smithi115.stdout:2676 write 0xafe507 thru 0xb0561c (0x7116 bytes) 2024-09-17T22:42:08.853 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:08.850+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:08.860 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:08.860 INFO:teuthology.orchestra.run.smithi115.stdout:2681 read 0xa18de2a thru 0xa19a634 (0xc80b bytes) 2024-09-17T22:42:08.861 INFO:teuthology.orchestra.run.smithi115.stdout:2682 punch from 0x49ff4ac to 0x4a02b4c, (0x36a0 bytes) 2024-09-17T22:42:08.861 INFO:teuthology.orchestra.run.smithi115.stdout:2683 write 0xe60b0cb thru 0xe6118a3 (0x67d9 bytes) 2024-09-17T22:42:08.866 INFO:teuthology.orchestra.run.smithi115.stdout:2684 read 0xacccce3 thru 0xacd994b (0xcc69 bytes) 2024-09-17T22:42:08.866 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:08.867 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:08.867 INFO:teuthology.orchestra.run.smithi115.stdout:2687 write 0xc85bdab thru 0xc868adf (0xcd35 bytes) 2024-09-17T22:42:08.876 INFO:teuthology.orchestra.run.smithi115.stdout:2688 read 0xa54b20e thru 0xa54f579 (0x436c bytes) 2024-09-17T22:42:08.877 INFO:teuthology.orchestra.run.smithi115.stdout:2689 write 0xa82f368 thru 0xa839d5b (0xa9f4 bytes) 2024-09-17T22:42:08.881 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:08.881 INFO:teuthology.orchestra.run.smithi115.stdout:2692 punch from 0xdb42b23 to 0xdb52583, (0xfa60 bytes) 2024-09-17T22:42:08.881 INFO:teuthology.orchestra.run.smithi115.stdout:2695 clone 25 order 20 su 65536 sc 5 2024-09-17T22:42:08.910 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:08.910 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:08.910 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:08.910 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:08.919 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:42:08.954 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:08.952+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:09.055 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:09.052+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:09.062 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:09.062 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:09.062 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:09.062 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:09.062 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:09.071 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:42:09.156 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:09.154+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:09.175 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:09.175 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:09.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:09.175 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:09.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.176 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36437 3.b 3:d64eaf12:::rbd_data.115f86d8437d.000000000000001c:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:09.176 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:42:09.176 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.456959283000003, 2024-09-17T22:42:09.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068296390000000002, 2024-09-17T22:42:09.176 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.176 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.176 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:09.176 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:09.177 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:09.177 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:09.177 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36437 2024-09-17T22:42:09.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.177 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:42:09.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:42:09.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715035+0000", 2024-09-17T22:42:09.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.288e-06 2024-09-17T22:42:09.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715041+0000", 2024-09-17T22:42:09.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.643e-06 2024-09-17T22:42:09.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715051+0000", 2024-09-17T22:42:09.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0597e-05 2024-09-17T22:42:09.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715056+0000", 2024-09-17T22:42:09.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1379999999999998e-06 2024-09-17T22:42:09.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715101+0000", 2024-09-17T22:42:09.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4433e-05 2024-09-17T22:42:09.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715187+0000", 2024-09-17T22:42:09.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6148000000000003e-05 2024-09-17T22:42:09.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:42:09.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.716264+0000", 2024-09-17T22:42:09.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010770179999999999 2024-09-17T22:42:09.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:09.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721735+0000", 2024-09-17T22:42:09.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0054709019999999997 2024-09-17T22:42:09.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:42:09.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721754+0000", 2024-09-17T22:42:09.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9133000000000001e-05 2024-09-17T22:42:09.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:09.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721807+0000", 2024-09-17T22:42:09.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2831999999999998e-05 2024-09-17T22:42:09.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721863+0000", 2024-09-17T22:42:09.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6507e-05 2024-09-17T22:42:09.182 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.182 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.182 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.183 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36744 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:09.183 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:42:09.183 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.436955140999999, 2024-09-17T22:42:09.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063082559999999999, 2024-09-17T22:42:09.183 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.183 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.183 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:09.183 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:09.183 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:09.183 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:09.183 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36744 2024-09-17T22:42:09.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.184 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:42:09.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:42:09.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:42:09.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:42:09.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735040+0000", 2024-09-17T22:42:09.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9630000000000001e-06 2024-09-17T22:42:09.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735047+0000", 2024-09-17T22:42:09.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3439999999999997e-06 2024-09-17T22:42:09.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735052+0000", 2024-09-17T22:42:09.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9250000000000003e-06 2024-09-17T22:42:09.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741147+0000", 2024-09-17T22:42:09.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0060954210000000002 2024-09-17T22:42:09.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.187 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741243+0000", 2024-09-17T22:42:09.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5674999999999995e-05 2024-09-17T22:42:09.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.187 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741346+0000", 2024-09-17T22:42:09.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103522 2024-09-17T22:42:09.187 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.187 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.187 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.188 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36745 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:09.188 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:42:09.188 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.436935157000001, 2024-09-17T22:42:09.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065186740000000003, 2024-09-17T22:42:09.188 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.188 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.188 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:09.188 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:09.188 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:09.188 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:09.188 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36745 2024-09-17T22:42:09.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.189 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:42:09.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:42:09.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:42:09.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2500000000000001e-07 2024-09-17T22:42:09.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735061+0000", 2024-09-17T22:42:09.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.824e-06 2024-09-17T22:42:09.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735068+0000", 2024-09-17T22:42:09.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8580000000000002e-06 2024-09-17T22:42:09.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735074+0000", 2024-09-17T22:42:09.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5829999999999997e-06 2024-09-17T22:42:09.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741379+0000", 2024-09-17T22:42:09.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063051909999999999 2024-09-17T22:42:09.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.191 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.191 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741472+0000", 2024-09-17T22:42:09.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3238000000000004e-05 2024-09-17T22:42:09.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741577+0000", 2024-09-17T22:42:09.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000104555 2024-09-17T22:42:09.192 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.192 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.192 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.192 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36746 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:09.192 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:42:09.192 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.436912190999998, 2024-09-17T22:42:09.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067275240000000004, 2024-09-17T22:42:09.193 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.193 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.193 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:09.193 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:09.193 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:09.193 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:09.193 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36746 2024-09-17T22:42:09.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.193 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:42:09.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:42:09.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:42:09.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.03e-07 2024-09-17T22:42:09.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735083+0000", 2024-09-17T22:42:09.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T22:42:09.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735090+0000", 2024-09-17T22:42:09.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7800000000000003e-06 2024-09-17T22:42:09.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735095+0000", 2024-09-17T22:42:09.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8500000000000002e-06 2024-09-17T22:42:09.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741608+0000", 2024-09-17T22:42:09.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065131809999999998 2024-09-17T22:42:09.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741703+0000", 2024-09-17T22:42:09.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4914999999999995e-05 2024-09-17T22:42:09.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741808+0000", 2024-09-17T22:42:09.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000105448 2024-09-17T22:42:09.196 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.197 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.197 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.197 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36747 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:09.197 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:42:09.197 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.436891637999999, 2024-09-17T22:42:09.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069468790000000004, 2024-09-17T22:42:09.197 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.197 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.197 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:09.197 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:09.197 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:09.197 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:09.198 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36747 2024-09-17T22:42:09.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.198 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:42:09.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:42:09.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735102+0000", 2024-09-17T22:42:09.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:42:09.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735104+0000", 2024-09-17T22:42:09.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7400000000000001e-06 2024-09-17T22:42:09.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735111+0000", 2024-09-17T22:42:09.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4189999999999996e-06 2024-09-17T22:42:09.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.200 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.200 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735116+0000", 2024-09-17T22:42:09.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0860000000000001e-06 2024-09-17T22:42:09.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.200 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.200 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741847+0000", 2024-09-17T22:42:09.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067306859999999996 2024-09-17T22:42:09.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.200 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.200 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741942+0000", 2024-09-17T22:42:09.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5180999999999996e-05 2024-09-17T22:42:09.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.201 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742048+0000", 2024-09-17T22:42:09.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106387 2024-09-17T22:42:09.201 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.201 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.201 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.201 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36748 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:09.201 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:42:09.201 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.436870317, 2024-09-17T22:42:09.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071864119999999997, 2024-09-17T22:42:09.202 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.202 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.202 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:09.202 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:09.202 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:09.202 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:09.202 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36748 2024-09-17T22:42:09.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.202 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:42:09.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:42:09.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:42:09.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.34e-07 2024-09-17T22:42:09.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735125+0000", 2024-09-17T22:42:09.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0020000000000001e-06 2024-09-17T22:42:09.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735132+0000", 2024-09-17T22:42:09.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3489999999999997e-06 2024-09-17T22:42:09.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735138+0000", 2024-09-17T22:42:09.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9709999999999998e-06 2024-09-17T22:42:09.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.205 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742087+0000", 2024-09-17T22:42:09.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069480669999999996 2024-09-17T22:42:09.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.205 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.205 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742188+0000", 2024-09-17T22:42:09.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101279 2024-09-17T22:42:09.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.205 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.205 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742309+0000", 2024-09-17T22:42:09.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012131 2024-09-17T22:42:09.206 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.206 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.206 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.206 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36749 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:09.206 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:42:09.206 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.436847630000003, 2024-09-17T22:42:09.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0074112570000000001, 2024-09-17T22:42:09.206 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.206 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.206 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:09.206 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:09.206 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:09.207 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:09.207 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36749 2024-09-17T22:42:09.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.207 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:42:09.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:42:09.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735146+0000", 2024-09-17T22:42:09.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9200000000000002e-07 2024-09-17T22:42:09.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735158+0000", 2024-09-17T22:42:09.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2508e-05 2024-09-17T22:42:09.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735166+0000", 2024-09-17T22:42:09.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6290000000000001e-06 2024-09-17T22:42:09.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735174+0000", 2024-09-17T22:42:09.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1000000000000004e-06 2024-09-17T22:42:09.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742354+0000", 2024-09-17T22:42:09.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071795660000000001 2024-09-17T22:42:09.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742454+0000", 2024-09-17T22:42:09.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100408 2024-09-17T22:42:09.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742557+0000", 2024-09-17T22:42:09.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102654 2024-09-17T22:42:09.210 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.210 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.210 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.211 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36750 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:09.211 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:42:09.211 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.436810504, 2024-09-17T22:42:09.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0077413890000000004, 2024-09-17T22:42:09.211 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.211 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.211 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:09.211 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:09.211 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:09.211 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:09.211 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36750 2024-09-17T22:42:09.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.212 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:42:09.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:42:09.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:42:09.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2399999999999998e-07 2024-09-17T22:42:09.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735185+0000", 2024-09-17T22:42:09.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1710000000000001e-06 2024-09-17T22:42:09.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735192+0000", 2024-09-17T22:42:09.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.81e-06 2024-09-17T22:42:09.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735197+0000", 2024-09-17T22:42:09.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9080000000000003e-06 2024-09-17T22:42:09.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742587+0000", 2024-09-17T22:42:09.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.007390359 2024-09-17T22:42:09.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.214 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742694+0000", 2024-09-17T22:42:09.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106901 2024-09-17T22:42:09.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742924+0000", 2024-09-17T22:42:09.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022961600000000001 2024-09-17T22:42:09.215 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.215 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.215 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.215 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36751 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:09.215 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:42:09.216 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.436788530000001, 2024-09-17T22:42:09.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079273859999999998, 2024-09-17T22:42:09.216 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.216 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.216 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:09.216 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:09.216 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:09.216 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:09.216 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36751 2024-09-17T22:42:09.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.216 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:42:09.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:42:09.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:42:09.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1600000000000001e-07 2024-09-17T22:42:09.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735207+0000", 2024-09-17T22:42:09.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1909999999999999e-06 2024-09-17T22:42:09.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735214+0000", 2024-09-17T22:42:09.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9269999999999999e-06 2024-09-17T22:42:09.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735220+0000", 2024-09-17T22:42:09.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3739999999999997e-06 2024-09-17T22:42:09.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.218 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742957+0000", 2024-09-17T22:42:09.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0077371699999999998 2024-09-17T22:42:09.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.743037+0000", 2024-09-17T22:42:09.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0370000000000005e-05 2024-09-17T22:42:09.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.743132+0000", 2024-09-17T22:42:09.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4838000000000002e-05 2024-09-17T22:42:09.219 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.220 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.220 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.220 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37659 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:09.220 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:42:09.220 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.422220887000002, 2024-09-17T22:42:09.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063144200000000003, 2024-09-17T22:42:09.220 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.220 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.220 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:09.220 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:09.220 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:09.220 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:09.221 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37659 2024-09-17T22:42:09.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.221 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:42:09.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:42:09.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749773+0000", 2024-09-17T22:42:09.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:42:09.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749774+0000", 2024-09-17T22:42:09.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:42:09.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749781+0000", 2024-09-17T22:42:09.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5740000000000001e-06 2024-09-17T22:42:09.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749786+0000", 2024-09-17T22:42:09.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9099999999999996e-06 2024-09-17T22:42:09.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.223 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.223 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755889+0000", 2024-09-17T22:42:09.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061034260000000003 2024-09-17T22:42:09.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755972+0000", 2024-09-17T22:42:09.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2795999999999998e-05 2024-09-17T22:42:09.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756087+0000", 2024-09-17T22:42:09.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000114418 2024-09-17T22:42:09.224 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.224 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.224 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.225 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37660 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:09.225 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:42:09.225 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.422191048999998, 2024-09-17T22:42:09.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065587800000000002, 2024-09-17T22:42:09.225 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.225 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.225 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:09.225 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:09.225 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:09.225 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:09.225 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37660 2024-09-17T22:42:09.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.225 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:42:09.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:42:09.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:42:09.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:42:09.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749805+0000", 2024-09-17T22:42:09.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-17T22:42:09.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749812+0000", 2024-09-17T22:42:09.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270000000000002e-06 2024-09-17T22:42:09.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749819+0000", 2024-09-17T22:42:09.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.759e-06 2024-09-17T22:42:09.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756153+0000", 2024-09-17T22:42:09.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063345809999999997 2024-09-17T22:42:09.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.228 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756257+0000", 2024-09-17T22:42:09.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103583 2024-09-17T22:42:09.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.228 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756361+0000", 2024-09-17T22:42:09.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103853 2024-09-17T22:42:09.229 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.229 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.229 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.229 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37661 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:09.229 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:42:09.229 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.422162454999999, 2024-09-17T22:42:09.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067442300000000004, 2024-09-17T22:42:09.229 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.229 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.229 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:09.230 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:09.230 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:09.230 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:09.230 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37661 2024-09-17T22:42:09.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.230 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:42:09.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:42:09.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:42:09.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9399999999999995e-07 2024-09-17T22:42:09.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749835+0000", 2024-09-17T22:42:09.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1049999999999997e-06 2024-09-17T22:42:09.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749855+0000", 2024-09-17T22:42:09.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9343000000000001e-05 2024-09-17T22:42:09.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749860+0000", 2024-09-17T22:42:09.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:42:09.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756395+0000", 2024-09-17T22:42:09.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065349900000000001 2024-09-17T22:42:09.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756479+0000", 2024-09-17T22:42:09.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4030999999999996e-05 2024-09-17T22:42:09.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756575+0000", 2024-09-17T22:42:09.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5835999999999995e-05 2024-09-17T22:42:09.233 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.233 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.234 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.234 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37662 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:09.234 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:42:09.234 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.422108649000002, 2024-09-17T22:42:09.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006927295, 2024-09-17T22:42:09.234 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.234 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.234 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:09.234 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:09.234 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:09.234 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:09.235 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37662 2024-09-17T22:42:09.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.235 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:42:09.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:42:09.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749885+0000", 2024-09-17T22:42:09.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2699999999999999e-07 2024-09-17T22:42:09.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749888+0000", 2024-09-17T22:42:09.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6599999999999999e-06 2024-09-17T22:42:09.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749895+0000", 2024-09-17T22:42:09.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4200000000000001e-06 2024-09-17T22:42:09.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749901+0000", 2024-09-17T22:42:09.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5090000000000001e-06 2024-09-17T22:42:09.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756605+0000", 2024-09-17T22:42:09.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067040629999999997 2024-09-17T22:42:09.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756691+0000", 2024-09-17T22:42:09.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5846000000000004e-05 2024-09-17T22:42:09.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756812+0000", 2024-09-17T22:42:09.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012116999999999999 2024-09-17T22:42:09.238 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.238 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.238 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.239 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37663 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:09.239 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:42:09.239 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.422084439000002, 2024-09-17T22:42:09.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071811440000000004, 2024-09-17T22:42:09.239 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.239 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.239 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:09.239 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:09.239 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:09.239 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:09.239 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37663 2024-09-17T22:42:09.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.239 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:42:09.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:42:09.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.240 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.240 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:42:09.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:42:09.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749911+0000", 2024-09-17T22:42:09.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.035e-06 2024-09-17T22:42:09.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749917+0000", 2024-09-17T22:42:09.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4370000000000004e-06 2024-09-17T22:42:09.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749922+0000", 2024-09-17T22:42:09.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6249999999999998e-06 2024-09-17T22:42:09.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756845+0000", 2024-09-17T22:42:09.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069227280000000004 2024-09-17T22:42:09.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756994+0000", 2024-09-17T22:42:09.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000149068 2024-09-17T22:42:09.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757090+0000", 2024-09-17T22:42:09.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5851000000000001e-05 2024-09-17T22:42:09.243 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.243 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.243 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.243 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37664 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:09.243 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:42:09.243 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.422063594000001, 2024-09-17T22:42:09.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.007384743, 2024-09-17T22:42:09.243 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.244 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.244 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:09.244 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:09.244 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:09.244 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:09.244 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37664 2024-09-17T22:42:09.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.244 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:42:09.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:42:09.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749930+0000", 2024-09-17T22:42:09.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:42:09.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.245 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.245 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.245 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749933+0000", 2024-09-17T22:42:09.245 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8430000000000001e-06 2024-09-17T22:42:09.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749939+0000", 2024-09-17T22:42:09.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:42:09.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749944+0000", 2024-09-17T22:42:09.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9230000000000001e-06 2024-09-17T22:42:09.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.246 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.246 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.246 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757124+0000", 2024-09-17T22:42:09.246 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071800020000000004 2024-09-17T22:42:09.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757212+0000", 2024-09-17T22:42:09.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7383e-05 2024-09-17T22:42:09.247 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.247 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.247 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.247 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757314+0000", 2024-09-17T22:42:09.247 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010247399999999999 2024-09-17T22:42:09.247 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.247 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.247 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.248 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.248 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:39363 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:09.248 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:42:09.248 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 39.380075255000001, 2024-09-17T22:42:09.248 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006226245, 2024-09-17T22:42:09.248 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.248 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.248 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:09.248 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:09.248 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:09.248 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:09.248 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 39363 2024-09-17T22:42:09.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.249 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:42:09.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.249 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.249 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:42:09.249 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:42:09.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3500000000000002e-07 2024-09-17T22:42:09.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791920+0000", 2024-09-17T22:42:09.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.869e-06 2024-09-17T22:42:09.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.250 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.250 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.250 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791926+0000", 2024-09-17T22:42:09.250 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8490000000000002e-06 2024-09-17T22:42:09.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791931+0000", 2024-09-17T22:42:09.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4269999999999999e-06 2024-09-17T22:42:09.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.797912+0000", 2024-09-17T22:42:09.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059808819999999999 2024-09-17T22:42:09.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.251 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.251 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.251 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.798020+0000", 2024-09-17T22:42:09.251 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010783800000000001 2024-09-17T22:42:09.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.252 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.252 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.798144+0000", 2024-09-17T22:42:09.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012404500000000001 2024-09-17T22:42:09.252 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.252 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.252 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.252 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.252 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4447.0:51869 3.6 e35/23 3:6037e35f:::rbd_object_map.115fe9ac8535:head v 35'71, mlcod=35'66)", 2024-09-17T22:42:09.252 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:42:09.252 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 35.394752955000001, 2024-09-17T22:42:09.252 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0066208669999999999, 2024-09-17T22:42:09.252 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.253 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.253 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:09.253 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:42:09.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.253 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.253 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:42:09.253 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.253 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777241+0000", 2024-09-17T22:42:09.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4000000000000001e-07 2024-09-17T22:42:09.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777246+0000", 2024-09-17T22:42:09.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2789999999999998e-06 2024-09-17T22:42:09.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.254 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.254 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.254 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777255+0000", 2024-09-17T22:42:09.254 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6209999999999993e-06 2024-09-17T22:42:09.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777259+0000", 2024-09-17T22:42:09.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7650000000000001e-06 2024-09-17T22:42:09.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777298+0000", 2024-09-17T22:42:09.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8797999999999997e-05 2024-09-17T22:42:09.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.255 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.255 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.255 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777311+0000", 2024-09-17T22:42:09.255 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2364999999999999e-05 2024-09-17T22:42:09.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:09.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.783814+0000", 2024-09-17T22:42:09.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065039169999999997 2024-09-17T22:42:09.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.256 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.256 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.256 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.783861+0000", 2024-09-17T22:42:09.256 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6482e-05 2024-09-17T22:42:09.256 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.256 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.256 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.257 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.257 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:105336 3.b 3:d3549b8d:::rbd_data.115ff9e8d63c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:42:09.257 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:43.787529+0000", 2024-09-17T22:42:09.257 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 25.384463650000001, 2024-09-17T22:42:09.257 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0099864259999999996, 2024-09-17T22:42:09.257 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.257 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.257 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:09.257 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:09.257 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:09.257 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:09.257 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 105336 2024-09-17T22:42:09.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.258 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.787529+0000", 2024-09-17T22:42:09.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.258 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.258 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.787529+0000", 2024-09-17T22:42:09.258 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.258 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.787531+0000", 2024-09-17T22:42:09.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9759999999999998e-06 2024-09-17T22:42:09.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788120+0000", 2024-09-17T22:42:09.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00058899799999999997 2024-09-17T22:42:09.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.259 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.259 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788138+0000", 2024-09-17T22:42:09.259 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7393000000000001e-05 2024-09-17T22:42:09.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.259 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788146+0000", 2024-09-17T22:42:09.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8220000000000007e-06 2024-09-17T22:42:09.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788199+0000", 2024-09-17T22:42:09.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3718000000000003e-05 2024-09-17T22:42:09.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.260 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.260 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.260 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788316+0000", 2024-09-17T22:42:09.260 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000116999 2024-09-17T22:42:09.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:42:09.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.791608+0000", 2024-09-17T22:42:09.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003291188 2024-09-17T22:42:09.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:09.261 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797379+0000", 2024-09-17T22:42:09.261 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057716829999999997 2024-09-17T22:42:09.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.261 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.261 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:42:09.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797401+0000", 2024-09-17T22:42:09.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1729000000000002e-05 2024-09-17T22:42:09.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:09.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797459+0000", 2024-09-17T22:42:09.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8115000000000002e-05 2024-09-17T22:42:09.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.262 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.262 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.262 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797516+0000", 2024-09-17T22:42:09.262 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6805000000000002e-05 2024-09-17T22:42:09.262 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.262 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.263 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.263 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4447.0:177615 3.0 e57/23 3:036f4184:::rbd_data.115f31bbb4b1.000000000000001c:head v 57'151, mlcod=55'148)", 2024-09-17T22:42:09.263 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:59.581018+0000", 2024-09-17T22:42:09.263 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.5909752469999994, 2024-09-17T22:42:09.263 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067345199999999999, 2024-09-17T22:42:09.263 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.263 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.263 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:09.263 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.263 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.263 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581018+0000", 2024-09-17T22:42:09.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581018+0000", 2024-09-17T22:42:09.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.264 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.264 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.264 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581018+0000", 2024-09-17T22:42:09.264 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4499999999999997e-07 2024-09-17T22:42:09.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581024+0000", 2024-09-17T22:42:09.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-17T22:42:09.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581036+0000", 2024-09-17T22:42:09.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1287999999999999e-05 2024-09-17T22:42:09.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.265 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.265 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.265 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581044+0000", 2024-09-17T22:42:09.265 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8220000000000007e-06 2024-09-17T22:42:09.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581094+0000", 2024-09-17T22:42:09.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0475e-05 2024-09-17T22:42:09.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.266 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.266 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581112+0000", 2024-09-17T22:42:09.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7932e-05 2024-09-17T22:42:09.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.266 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:09.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.587716+0000", 2024-09-17T22:42:09.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0066042879999999998 2024-09-17T22:42:09.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.267 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.267 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.587752+0000", 2024-09-17T22:42:09.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6106999999999998e-05 2024-09-17T22:42:09.267 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.267 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.267 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.268 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:178091 3.b 3:d1597d64:::rbd_data.115f31bbb4b1.000000000000006b:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T22:42:09.268 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:00.078810+0000", 2024-09-17T22:42:09.268 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.0931831410000008, 2024-09-17T22:42:09.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017828758, 2024-09-17T22:42:09.268 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:09.268 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:09.268 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:09.268 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:09.268 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:09.269 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:09.269 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 178091 2024-09-17T22:42:09.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.269 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:09.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:09.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078810+0000", 2024-09-17T22:42:09.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:09.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078810+0000", 2024-09-17T22:42:09.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:09.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:09.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078811+0000", 2024-09-17T22:42:09.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.176e-06 2024-09-17T22:42:09.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:09.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078817+0000", 2024-09-17T22:42:09.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6540000000000004e-06 2024-09-17T22:42:09.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:09.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078828+0000", 2024-09-17T22:42:09.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0875e-05 2024-09-17T22:42:09.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:09.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078833+0000", 2024-09-17T22:42:09.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6099999999999997e-06 2024-09-17T22:42:09.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:09.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078884+0000", 2024-09-17T22:42:09.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1013000000000001e-05 2024-09-17T22:42:09.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:09.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078933+0000", 2024-09-17T22:42:09.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8380000000000001e-05 2024-09-17T22:42:09.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:42:09.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.081828+0000", 2024-09-17T22:42:09.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0028952219999999998 2024-09-17T22:42:09.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:09.272 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.096563+0000", 2024-09-17T22:42:09.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.014734860000000001 2024-09-17T22:42:09.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:42:09.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.096575+0000", 2024-09-17T22:42:09.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2126e-05 2024-09-17T22:42:09.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:09.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.096608+0000", 2024-09-17T22:42:09.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2836000000000002e-05 2024-09-17T22:42:09.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:09.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:09.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:09.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.096639+0000", 2024-09-17T22:42:09.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1006000000000001e-05 2024-09-17T22:42:09.274 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.274 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.274 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.274 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:09.274 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:09.274 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:09.274 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:42:09.276 INFO:teuthology.orchestra.run.smithi115.stdout:leaving image image_client.0-clone24 intact 2024-09-17T22:42:09.277 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:09.273+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:09.277 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:42:09.319 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:42:09.376 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:09.374+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:09.394 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:09.394 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:09.406 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:42:09.416 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-17T22:42:09.440 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:09.473 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:09.474 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:09.477 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:09.474+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:09.506 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:09.507 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:09.539 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:09.540 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:09.573 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:09.573 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:09.573 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:09.573 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:09.578 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:09.575+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:09.585 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:42:09.595 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:09.595 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:09.612 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:09.612 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:09.629 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:09.629 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:09.632 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:09.633 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:09.635 INFO:teuthology.orchestra.run.smithi115.stdout:2696 read 0x9207ba5 thru 0x920bfaf (0x440b bytes) 2024-09-17T22:42:09.635 INFO:teuthology.orchestra.run.smithi115.stdout:2697 punch from 0xc172b05 to 0xc17e79b, (0xbc96 bytes) 2024-09-17T22:42:09.639 INFO:teuthology.orchestra.run.smithi115.stdout:2699 punch from 0xd7cb7b9 to 0xd7d081d, (0x5064 bytes) 2024-09-17T22:42:09.639 INFO:teuthology.orchestra.run.smithi115.stdout:2700 write 0x122bba3 thru 0x1238d59 (0xd1b7 bytes) 2024-09-17T22:42:09.660 INFO:teuthology.orchestra.run.smithi115.stdout:2701 punch from 0x54c99c7 to 0x54cd1f3, (0x382c bytes) 2024-09-17T22:42:09.666 INFO:teuthology.orchestra.run.smithi115.stdout:2703 trunc from 0xed619a1 to 0x9c5d1d2 2024-09-17T22:42:09.680 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:09.676+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:09.694 INFO:teuthology.orchestra.run.smithi115.stdout:2704 punch from 0x6ad323f to 0x6ae29cd, (0xf78e bytes) 2024-09-17T22:42:09.695 INFO:teuthology.orchestra.run.smithi115.stdout:2706 write 0x1dfba83 thru 0x1e07438 (0xb9b6 bytes) 2024-09-17T22:42:09.711 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:09.711 INFO:teuthology.orchestra.run.smithi115.stdout:2710 writesame 0x10ad3b3 thru 0x10b1cdf data_size 0xd(0x492d bytes) 2024-09-17T22:42:09.714 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:09.714 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:09.723 INFO:teuthology.orchestra.run.smithi115.stdout:2711 punch from 0x72750fd to 0x727f650, (0xa553 bytes) 2024-09-17T22:42:09.723 INFO:teuthology.orchestra.run.smithi115.stdout:2714 read 0x2597c30 thru 0x259ca26 (0x4df7 bytes) 2024-09-17T22:42:09.723 INFO:teuthology.orchestra.run.smithi115.stdout:2715 punch from 0x56e1311 to 0x56ef920, (0xe60f bytes) 2024-09-17T22:42:09.724 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:42:09.734 INFO:teuthology.orchestra.run.smithi115.stdout:2717 writesame 0xe042656 thru 0xe0470c1 data_size 0xb(0x4a6c bytes) 2024-09-17T22:42:09.734 INFO:teuthology.orchestra.run.smithi115.stdout:2718 writesame 0xd0ea05b thru 0xd0f9ecc data_size 0x1(0xfe72 bytes) 2024-09-17T22:42:09.749 INFO:teuthology.orchestra.run.smithi115.stdout:2719 read 0x2a78d6d thru 0x2a80011 (0x72a5 bytes) 2024-09-17T22:42:09.750 INFO:teuthology.orchestra.run.smithi115.stdout:2720 read 0xb943033 thru 0xb94d1f6 (0xa1c4 bytes) 2024-09-17T22:42:09.750 INFO:teuthology.orchestra.run.smithi115.stdout:2722 write 0x46f96ae thru 0x4702c9f (0x95f2 bytes) 2024-09-17T22:42:09.760 INFO:teuthology.orchestra.run.smithi115.stdout:2723 writesame 0x5d59b35 thru 0x5d5d2ce data_size 0xb(0x379a bytes) 2024-09-17T22:42:09.773 INFO:teuthology.orchestra.run.smithi115.stdout:2726 write 0xa97625d thru 0xa97ad11 (0x4ab5 bytes) 2024-09-17T22:42:09.776 INFO:teuthology.orchestra.run.smithi115.stdout:2727 read 0xa91c675 thru 0xa928f33 (0xc8bf bytes) 2024-09-17T22:42:09.777 INFO:teuthology.orchestra.run.smithi115.stdout:2728 writesame 0x8a50d49 thru 0x8a5aafa data_size 0x5(0x9db2 bytes) 2024-09-17T22:42:09.780 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:09.777+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:09.797 INFO:teuthology.orchestra.run.smithi115.stdout:2730 trunc from 0xe0470c2 to 0x144a868 2024-09-17T22:42:09.831 INFO:teuthology.orchestra.run.smithi115.stdout:2731 punch from 0xc41165 to 0xc42d60, (0x1bfb bytes) 2024-09-17T22:42:09.831 INFO:teuthology.orchestra.run.smithi115.stdout:2735 punch from 0x12e638d to 0x12f288b, (0xc4fe bytes) 2024-09-17T22:42:09.831 INFO:teuthology.orchestra.run.smithi115.stdout:2736 read 0x12077d6 thru 0x120ba90 (0x42bb bytes) 2024-09-17T22:42:09.832 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:09.847 INFO:teuthology.orchestra.run.smithi115.stdout:2739 trunc from 0x144a868 to 0xcdc8c71 2024-09-17T22:42:09.853 INFO:teuthology.orchestra.run.smithi115.stdout:2741 writesame 0x3f56ad5 thru 0x3f5d4c7 data_size 0x1(0x69f3 bytes) 2024-09-17T22:42:09.870 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:09.870 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:09.874 INFO:teuthology.orchestra.run.smithi115.stdout:2742 trunc from 0xcdc8c71 to 0x9a5d30f 2024-09-17T22:42:09.880 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:09.878+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:09.882 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:42:09.882 INFO:teuthology.orchestra.run.smithi115.stdout:2743 read 0x451d283 thru 0x45291ef (0xbf6d bytes) 2024-09-17T22:42:09.882 INFO:teuthology.orchestra.run.smithi115.stdout:2746 read 0x6d3fac thru 0x6d6f82 (0x2fd7 bytes) 2024-09-17T22:42:09.890 INFO:teuthology.orchestra.run.smithi115.stdout:2747 write 0xc02ab1a thru 0xc034720 (0x9c07 bytes) 2024-09-17T22:42:09.901 INFO:teuthology.orchestra.run.smithi115.stdout:2748 write 0xd726169 thru 0xd72a79b (0x4633 bytes) 2024-09-17T22:42:09.904 INFO:teuthology.orchestra.run.smithi115.stdout:2749 write 0x46aafc5 thru 0x46b4d60 (0x9d9c bytes) 2024-09-17T22:42:09.908 INFO:teuthology.orchestra.run.smithi115.stdout:2751 punch from 0xd3f0cad to 0xd3fb014, (0xa367 bytes) 2024-09-17T22:42:09.908 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:09.908 INFO:teuthology.orchestra.run.smithi115.stdout:2753 write 0x7e364ad thru 0x7e377ab (0x12ff bytes) 2024-09-17T22:42:09.911 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:09.911 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:09.911 INFO:teuthology.orchestra.run.smithi115.stdout:2757 punch from 0x3bb3692 to 0x3bbc2b8, (0x8c26 bytes) 2024-09-17T22:42:09.911 INFO:teuthology.orchestra.run.smithi115.stdout:2761 write 0xb0836dc thru 0xb0904b5 (0xcdda bytes) 2024-09-17T22:42:09.915 INFO:teuthology.orchestra.run.smithi115.stdout:2763 read 0x71fe689 thru 0x7203f2c (0x58a4 bytes) 2024-09-17T22:42:09.915 INFO:teuthology.orchestra.run.smithi115.stdout:2764 read 0xcd5b112 thru 0xcd66aa7 (0xb996 bytes) 2024-09-17T22:42:09.918 INFO:teuthology.orchestra.run.smithi115.stdout:2765 trunc from 0xd72a79c to 0xade76a9 2024-09-17T22:42:09.941 INFO:teuthology.orchestra.run.smithi115.stdout:2766 trunc from 0xade76a9 to 0x3a95df3 2024-09-17T22:42:09.954 INFO:teuthology.orchestra.run.smithi115.stdout:2768 read 0x695d04 thru 0x69e7af (0x8aac bytes) 2024-09-17T22:42:09.962 INFO:teuthology.orchestra.run.smithi115.stdout:2769 writesame 0x3ffb6b7 thru 0x400148c data_size 0x1(0x5dd6 bytes) 2024-09-17T22:42:09.962 INFO:teuthology.orchestra.run.smithi115.stdout:2771 read 0x385a871 thru 0x385e73f (0x3ecf bytes) 2024-09-17T22:42:09.969 INFO:teuthology.orchestra.run.smithi115.stdout:2772 writesame 0x4199ed7 thru 0x419dc10 data_size 0x11(0x3d3a bytes) 2024-09-17T22:42:09.969 INFO:teuthology.orchestra.run.smithi115.stdout:2773 read 0x32b9d30 thru 0x32ba5fb (0x8cc bytes) 2024-09-17T22:42:09.969 INFO:teuthology.orchestra.run.smithi115.stdout:2775 writesame 0x194de86 thru 0x194e699 data_size 0x16(0x814 bytes) 2024-09-17T22:42:09.971 INFO:teuthology.orchestra.run.smithi115.stdout:2777 punch from 0x3c472ec to 0x3c48d18, (0x1a2c bytes) 2024-09-17T22:42:09.979 INFO:teuthology.orchestra.run.smithi115.stdout:2778 writesame 0x5c7da06 thru 0x5c871c3 data_size 0x1(0x97be bytes) 2024-09-17T22:42:09.981 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:09.978+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:09.989 INFO:teuthology.orchestra.run.smithi115.stdout:2779 write 0xa0308bc thru 0xa03becf (0xb614 bytes) 2024-09-17T22:42:09.992 INFO:teuthology.orchestra.run.smithi115.stdout:2780 read 0xd6786a thru 0xd7737a (0xfb11 bytes) 2024-09-17T22:42:09.992 INFO:teuthology.orchestra.run.smithi115.stdout:2781 write 0x72cf1f8 thru 0x72dd473 (0xe27c bytes) 2024-09-17T22:42:10.001 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:10.001 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:10.002 INFO:teuthology.orchestra.run.smithi115.stdout:2782 write 0xa3f2b54 thru 0xa3f9300 (0x67ad bytes) 2024-09-17T22:42:10.010 INFO:teuthology.orchestra.run.smithi115.stdout:2783 write 0xb6d2d11 thru 0xb6dafde (0x82ce bytes) 2024-09-17T22:42:10.011 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:42:10.013 INFO:teuthology.orchestra.run.smithi115.stdout:2784 read 0x7708035 thru 0x7712567 (0xa533 bytes) 2024-09-17T22:42:10.014 INFO:teuthology.orchestra.run.smithi115.stdout:2785 write 0x7686b8e thru 0x7687070 (0x4e3 bytes) 2024-09-17T22:42:10.016 INFO:teuthology.orchestra.run.smithi115.stdout:2786 read 0x93f53ee thru 0x93fad74 (0x5987 bytes) 2024-09-17T22:42:10.016 INFO:teuthology.orchestra.run.smithi115.stdout:2787 write 0x487bbc8 thru 0x4880b8e (0x4fc7 bytes) 2024-09-17T22:42:10.020 INFO:teuthology.orchestra.run.smithi115.stdout:2788 punch from 0x18a137 to 0x1973af, (0xd278 bytes) 2024-09-17T22:42:10.020 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:10.020 INFO:teuthology.orchestra.run.smithi115.stdout:2791 read 0x6c6a31b thru 0x6c6ed16 (0x49fc bytes) 2024-09-17T22:42:10.020 INFO:teuthology.orchestra.run.smithi115.stdout:2794 punch from 0x9848c23 to 0x98547d4, (0xbbb1 bytes) 2024-09-17T22:42:10.020 INFO:teuthology.orchestra.run.smithi115.stdout:2795 write 0x4513a48 thru 0x4516f49 (0x3502 bytes) 2024-09-17T22:42:10.023 INFO:teuthology.orchestra.run.smithi115.stdout:2796 punch from 0xf2a464 to 0xf2e54f, (0x40eb bytes) 2024-09-17T22:42:10.023 INFO:teuthology.orchestra.run.smithi115.stdout:2799 read 0x15d6ffd thru 0x15e458f (0xd593 bytes) 2024-09-17T22:42:10.023 INFO:teuthology.orchestra.run.smithi115.stdout:2800 write 0x24fcbf thru 0x2550f5 (0x5437 bytes) 2024-09-17T22:42:10.051 INFO:teuthology.orchestra.run.smithi115.stdout:2801 trunc from 0xb6dafdf to 0xd0f4b51 2024-09-17T22:42:10.056 INFO:teuthology.orchestra.run.smithi115.stdout:2802 read 0xa11a5f0 thru 0xa11e0dc (0x3aed bytes) 2024-09-17T22:42:10.056 INFO:teuthology.orchestra.run.smithi115.stdout:2803 read 0xc478f43 thru 0xc4886ba (0xf778 bytes) 2024-09-17T22:42:10.057 INFO:teuthology.orchestra.run.smithi115.stdout:2805 write 0x9e86c46 thru 0x9e8f568 (0x8923 bytes) 2024-09-17T22:42:10.059 INFO:teuthology.orchestra.run.smithi115.stdout:2806 write 0xad55423 thru 0xad574d7 (0x20b5 bytes) 2024-09-17T22:42:10.074 INFO:teuthology.orchestra.run.smithi115.stdout:2807 trunc from 0xd0f4b51 to 0x431522a 2024-09-17T22:42:10.082 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:10.079+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:10.085 INFO:teuthology.orchestra.run.smithi115.stdout:2808 read 0xd626c0 thru 0xd701a0 (0xdae1 bytes) 2024-09-17T22:42:10.097 INFO:teuthology.orchestra.run.smithi115.stdout:2809 trunc from 0x431522a to 0xd7b3833 2024-09-17T22:42:10.102 INFO:teuthology.orchestra.run.smithi115.stdout:2810 read 0x5a5d720 thru 0x5a5fe15 (0x26f6 bytes) 2024-09-17T22:42:10.102 INFO:teuthology.orchestra.run.smithi115.stdout:2811 write 0x7a2dfe7 thru 0x7a31415 (0x342f bytes) 2024-09-17T22:42:10.106 INFO:teuthology.orchestra.run.smithi115.stdout:2812 writesame 0x6ee04e3 thru 0x6ee2911 data_size 0x1(0x242f bytes) 2024-09-17T22:42:10.112 INFO:teuthology.orchestra.run.smithi115.stdout:2813 write 0x11b9395 thru 0x11c887d (0xf4e9 bytes) 2024-09-17T22:42:10.134 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:10.134 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:10.139 INFO:teuthology.orchestra.run.smithi115.stdout:2814 write 0xe9e9f46 thru 0xe9f9d31 (0xfdec bytes) 2024-09-17T22:42:10.144 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:42:10.144 INFO:teuthology.orchestra.run.smithi115.stdout:2816 punch from 0x54f602 to 0x5539e8, (0x43e6 bytes) 2024-09-17T22:42:10.144 INFO:teuthology.orchestra.run.smithi115.stdout:2817 write 0x92b8c91 thru 0x92bbb6b (0x2edb bytes) 2024-09-17T22:42:10.146 INFO:teuthology.orchestra.run.smithi115.stdout:2819 writesame 0x19268af thru 0x192f3a5 data_size 0x5(0x8af7 bytes) 2024-09-17T22:42:10.151 INFO:teuthology.orchestra.run.smithi115.stdout:2820 writesame 0x73f7144 thru 0x73fea05 data_size 0xd(0x78c2 bytes) 2024-09-17T22:42:10.155 INFO:teuthology.orchestra.run.smithi115.stdout:2821 trunc from 0xe9f9d32 to 0xe0d4059 2024-09-17T22:42:10.165 INFO:teuthology.orchestra.run.smithi115.stdout:2822 punch from 0x948555c to 0x94918d1, (0xc375 bytes) 2024-09-17T22:42:10.165 INFO:teuthology.orchestra.run.smithi115.stdout:2824 punch from 0x80432f3 to 0x80488f3, (0x5600 bytes) 2024-09-17T22:42:10.173 INFO:teuthology.orchestra.run.smithi115.stdout:2825 trunc from 0xe0d4059 to 0x73cd4f2 2024-09-17T22:42:10.183 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:10.180+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:10.184 INFO:teuthology.orchestra.run.smithi115.stdout:2826 read 0x451eb30 thru 0x45231c4 (0x4695 bytes) 2024-09-17T22:42:10.185 INFO:teuthology.orchestra.run.smithi115.stdout:2828 trunc from 0x73cd4f2 to 0x7be693c 2024-09-17T22:42:10.199 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is set 2024-09-17T22:42:10.207 INFO:teuthology.orchestra.run.smithi115.stdout:2829 write 0xcbce0f9 thru 0xcbd1792 (0x369a bytes) 2024-09-17T22:42:10.212 INFO:teuthology.orchestra.run.smithi115.stdout:2831 write 0xcb46bb6 thru 0xcb4b7fd (0x4c48 bytes) 2024-09-17T22:42:10.215 INFO:teuthology.orchestra.run.smithi115.stdout:2832 punch from 0xc2e0a3f to 0xc2e9ab6, (0x9077 bytes) 2024-09-17T22:42:10.215 INFO:teuthology.orchestra.run.smithi115.stdout:2834 write 0x6840388 thru 0x684d4ce (0xd147 bytes) 2024-09-17T22:42:10.218 INFO:teuthology.orchestra.run.smithi115.stdout:2837 read 0x44f60b9 thru 0x44fe68c (0x85d4 bytes) 2024-09-17T22:42:10.218 INFO:teuthology.orchestra.run.smithi115.stdout:2838 read 0x551d98c thru 0x5524855 (0x6eca bytes) 2024-09-17T22:42:10.225 INFO:teuthology.orchestra.run.smithi115.stdout:2839 write 0xe1e10b7 thru 0xe1ebd2a (0xac74 bytes) 2024-09-17T22:42:10.243 INFO:teuthology.orchestra.run.smithi115.stdout:2841 trunc from 0xe1ebd2b to 0x27ac843 2024-09-17T22:42:10.258 INFO:teuthology.orchestra.run.smithi115.stdout:2842 read 0x668557 thru 0x66c06c (0x3b16 bytes) 2024-09-17T22:42:10.258 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:10.258 INFO:teuthology.orchestra.run.smithi115.stdout:2844 read 0x763e71 thru 0x769584 (0x5714 bytes) 2024-09-17T22:42:10.258 INFO:teuthology.orchestra.run.smithi115.stdout:2845 write 0x17584ed thru 0x175e6a0 (0x61b4 bytes) 2024-09-17T22:42:10.266 INFO:teuthology.orchestra.run.smithi115.stdout:2847 write 0x769da11 thru 0x769fd43 (0x2333 bytes) 2024-09-17T22:42:10.269 INFO:teuthology.orchestra.run.smithi115.stdout:2849 punch from 0x29670fd to 0x29733e6, (0xc2e9 bytes) 2024-09-17T22:42:10.269 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:10.271 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:10.271 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:10.276 INFO:teuthology.orchestra.run.smithi115.stdout:2851 write 0x9220682 thru 0x9225c3b (0x55ba bytes) 2024-09-17T22:42:10.280 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:42:10.281 INFO:teuthology.orchestra.run.smithi115.stdout:2853 trunc from 0x9225c3c to 0x7be2005 2024-09-17T22:42:10.283 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:10.281+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:10.293 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:10.296 INFO:teuthology.orchestra.run.smithi115.stdout:2856 trunc from 0x7be2005 to 0x472d2b5 2024-09-17T22:42:10.308 INFO:teuthology.orchestra.run.smithi115.stdout:2857 read 0x340f048 thru 0x34196d1 (0xa68a bytes) 2024-09-17T22:42:10.317 INFO:teuthology.orchestra.run.smithi115.stdout:2858 writesame 0x749b4b7 thru 0x74a5a77 data_size 0x1(0xa5c1 bytes) 2024-09-17T22:42:10.317 INFO:teuthology.orchestra.run.smithi115.stdout:2859 punch from 0x4a41a0 to 0x4aeecf, (0xad2f bytes) 2024-09-17T22:42:10.317 INFO:teuthology.orchestra.run.smithi115.stdout:2860 write 0x560ed48 thru 0x561710a (0x83c3 bytes) 2024-09-17T22:42:10.323 INFO:teuthology.orchestra.run.smithi115.stdout:2863 write 0x3fccc77 thru 0x3fd40d8 (0x7462 bytes) 2024-09-17T22:42:10.329 INFO:teuthology.orchestra.run.smithi115.stdout:2864 read 0x14af38c thru 0x14b9b68 (0xa7dd bytes) 2024-09-17T22:42:10.335 INFO:teuthology.orchestra.run.smithi115.stdout:2865 trunc from 0x74a5a78 to 0xc2c49c8 2024-09-17T22:42:10.356 INFO:teuthology.orchestra.run.smithi115.stdout:2866 trunc from 0xc2c49c8 to 0x710433 2024-09-17T22:42:10.384 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:10.382+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:10.402 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:10.402 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:10.404 INFO:teuthology.orchestra.run.smithi115.stdout:2867 punch from 0x51aa73 to 0x51cf5a, (0x24e7 bytes) 2024-09-17T22:42:10.411 INFO:teuthology.orchestra.run.smithi115.stdout:2868 write 0x19538c1 thru 0x1958291 (0x49d1 bytes) 2024-09-17T22:42:10.412 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:42:10.437 INFO:teuthology.orchestra.run.smithi115.stdout:2869 write 0xda5b1c5 thru 0xda618c6 (0x6702 bytes) 2024-09-17T22:42:10.442 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:10.443 INFO:teuthology.orchestra.run.smithi115.stdout:2871 write 0x8d84583 thru 0x8d87770 (0x31ee bytes) 2024-09-17T22:42:10.446 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:10.446 INFO:teuthology.orchestra.run.smithi115.stdout:2873 read 0x50da2a3 thru 0x50e37a7 (0x9505 bytes) 2024-09-17T22:42:10.447 INFO:teuthology.orchestra.run.smithi115.stdout:2875 trunc from 0xda618c7 to 0xcde8751 2024-09-17T22:42:10.468 INFO:teuthology.orchestra.run.smithi115.stdout:2876 trunc from 0xcde8751 to 0x4850f3b 2024-09-17T22:42:10.479 INFO:teuthology.orchestra.run.smithi115.stdout:2877 punch from 0x430d06d to 0x4313e5f, (0x6df2 bytes) 2024-09-17T22:42:10.479 INFO:teuthology.orchestra.run.smithi115.stdout:2878 read 0x4435d6a thru 0x443a7a2 (0x4a39 bytes) 2024-09-17T22:42:10.485 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:10.483+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:10.488 INFO:teuthology.orchestra.run.smithi115.stdout:2879 trunc from 0x4850f3b to 0xc535ce8 2024-09-17T22:42:10.501 INFO:teuthology.orchestra.run.smithi115.stdout:2880 write 0xc9790a8 thru 0xc97c75e (0x36b7 bytes) 2024-09-17T22:42:10.504 INFO:teuthology.orchestra.run.smithi115.stdout:2881 writesame 0x9af28a8 thru 0x9af4845 data_size 0x13(0x1f9e bytes) 2024-09-17T22:42:10.507 INFO:teuthology.orchestra.run.smithi115.stdout:2883 read 0x69a1773 thru 0x69a31db (0x1a69 bytes) 2024-09-17T22:42:10.515 INFO:teuthology.orchestra.run.smithi115.stdout:2884 writesame 0xe6b92c7 thru 0xe6c1805 data_size 0xb(0x853f bytes) 2024-09-17T22:42:10.515 INFO:teuthology.orchestra.run.smithi115.stdout:2885 write 0x51eb288 thru 0x51f09db (0x5754 bytes) 2024-09-17T22:42:10.519 INFO:teuthology.orchestra.run.smithi115.stdout:2889 read 0xd4877b9 thru 0xd48b0a8 (0x38f0 bytes) 2024-09-17T22:42:10.519 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:10.520 INFO:teuthology.orchestra.run.smithi115.stdout:2892 write 0x1fd6819 thru 0x1fe334d (0xcb35 bytes) 2024-09-17T22:42:10.523 INFO:teuthology.orchestra.run.smithi115.stdout:2895 punch from 0x88d56d8 to 0x88e04e2, (0xae0a bytes) 2024-09-17T22:42:10.523 INFO:teuthology.orchestra.run.smithi115.stdout:2897 write 0x802ab9d thru 0x8033922 (0x8d86 bytes) 2024-09-17T22:42:10.527 INFO:teuthology.orchestra.run.smithi115.stdout:2899 clone 26 order 24 su 65536 sc 2 2024-09-17T22:42:10.552 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:10.552 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:10.561 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:42:10.605 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:10.603+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:10.668 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:10.668 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:10.674 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:42:10.687 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:10.685+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:10.788 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:10.786+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:10.815 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:10.816 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:10.824 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:42:10.862 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:42:10.862 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:42:10.862 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T22:42:10.863 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T22:42:10.906 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:10.904+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:10.927 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:10.927 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:10.936 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:42:10.990 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:10.987+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:11.040 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:11.040 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:11.049 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:42:11.091 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:11.088+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:11.153 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:42:11.153 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:11.153 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 67, 2024-09-17T22:42:11.153 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "551c5ed8-e790-485a-bc95-a9578a29275c", 2024-09-17T22:42:11.153 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-09-17T22:40:50.189230+0000", 2024-09-17T22:42:11.154 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-09-17T22:42:10.189579+0000", 2024-09-17T22:42:11.154 INFO:teuthology.orchestra.run.smithi115.stdout: "last_up_change": "2024-09-17T22:41:53.099515+0000", 2024-09-17T22:42:11.154 INFO:teuthology.orchestra.run.smithi115.stdout: "last_in_change": "2024-09-17T22:40:52.260526+0000", 2024-09-17T22:42:11.154 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T22:42:11.154 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_num": 5806080, 2024-09-17T22:42:11.154 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_set": [ 2024-09-17T22:42:11.154 INFO:teuthology.orchestra.run.smithi115.stdout: "nodeep-scrub", 2024-09-17T22:42:11.154 INFO:teuthology.orchestra.run.smithi115.stdout: "noscrub", 2024-09-17T22:42:11.154 INFO:teuthology.orchestra.run.smithi115.stdout: "pglog_hardlimit", 2024-09-17T22:42:11.154 INFO:teuthology.orchestra.run.smithi115.stdout: "purged_snapdirs", 2024-09-17T22:42:11.154 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_deletes", 2024-09-17T22:42:11.154 INFO:teuthology.orchestra.run.smithi115.stdout: "sortbitwise" 2024-09-17T22:42:11.154 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:11.154 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_version": 5, 2024-09-17T22:42:11.155 INFO:teuthology.orchestra.run.smithi115.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T22:42:11.155 INFO:teuthology.orchestra.run.smithi115.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T22:42:11.155 INFO:teuthology.orchestra.run.smithi115.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T22:42:11.155 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_snapshot": "", 2024-09-17T22:42:11.155 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_max": 3, 2024-09-17T22:42:11.155 INFO:teuthology.orchestra.run.smithi115.stdout: "max_osd": 8, 2024-09-17T22:42:11.155 INFO:teuthology.orchestra.run.smithi115.stdout: "require_min_compat_client": "luminous", 2024-09-17T22:42:11.155 INFO:teuthology.orchestra.run.smithi115.stdout: "min_compat_client": "luminous", 2024-09-17T22:42:11.155 INFO:teuthology.orchestra.run.smithi115.stdout: "require_osd_release": "squid", 2024-09-17T22:42:11.155 INFO:teuthology.orchestra.run.smithi115.stdout: "allow_crimson": false, 2024-09-17T22:42:11.155 INFO:teuthology.orchestra.run.smithi115.stdout: "pools": [ 2024-09-17T22:42:11.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.155 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 1, 2024-09-17T22:42:11.155 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_name": ".mgr", 2024-09-17T22:42:11.156 INFO:teuthology.orchestra.run.smithi115.stdout: "create_time": "2024-09-17T22:41:02.338356+0000", 2024-09-17T22:42:11.156 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-09-17T22:42:11.156 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_names": "hashpspool", 2024-09-17T22:42:11.156 INFO:teuthology.orchestra.run.smithi115.stdout: "type": 1, 2024-09-17T22:42:11.156 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 2, 2024-09-17T22:42:11.156 INFO:teuthology.orchestra.run.smithi115.stdout: "min_size": 1, 2024-09-17T22:42:11.156 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_rule": 0, 2024-09-17T22:42:11.156 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:42:11.156 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:42:11.156 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:42:11.156 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:42:11.156 INFO:teuthology.orchestra.run.smithi115.stdout: "is_stretch_pool": false, 2024-09-17T22:42:11.157 INFO:teuthology.orchestra.run.smithi115.stdout: "object_hash": 2, 2024-09-17T22:42:11.157 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:42:11.157 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num": 1, 2024-09-17T22:42:11.157 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num": 1, 2024-09-17T22:42:11.157 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num_target": 1, 2024-09-17T22:42:11.157 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_target": 1, 2024-09-17T22:42:11.157 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_pending": 1, 2024-09-17T22:42:11.157 INFO:teuthology.orchestra.run.smithi115.stdout: "last_pg_merge_meta": { 2024-09-17T22:42:11.157 INFO:teuthology.orchestra.run.smithi115.stdout: "source_pgid": "0.0", 2024-09-17T22:42:11.157 INFO:teuthology.orchestra.run.smithi115.stdout: "ready_epoch": 0, 2024-09-17T22:42:11.157 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_started": 0, 2024-09-17T22:42:11.157 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_clean": 0, 2024-09-17T22:42:11.157 INFO:teuthology.orchestra.run.smithi115.stdout: "source_version": "0'0", 2024-09-17T22:42:11.157 INFO:teuthology.orchestra.run.smithi115.stdout: "target_version": "0'0" 2024-09-17T22:42:11.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.158 INFO:teuthology.orchestra.run.smithi115.stdout: "last_change": "17", 2024-09-17T22:42:11.158 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend": "0", 2024-09-17T22:42:11.158 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:42:11.158 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:42:11.158 INFO:teuthology.orchestra.run.smithi115.stdout: "auid": 0, 2024-09-17T22:42:11.158 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:42:11.158 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_seq": 0, 2024-09-17T22:42:11.158 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_epoch": 0, 2024-09-17T22:42:11.158 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_snaps": [], 2024-09-17T22:42:11.158 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps": "[]", 2024-09-17T22:42:11.158 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_bytes": 0, 2024-09-17T22:42:11.158 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_objects": 0, 2024-09-17T22:42:11.158 INFO:teuthology.orchestra.run.smithi115.stdout: "tiers": [], 2024-09-17T22:42:11.159 INFO:teuthology.orchestra.run.smithi115.stdout: "tier_of": -1, 2024-09-17T22:42:11.159 INFO:teuthology.orchestra.run.smithi115.stdout: "read_tier": -1, 2024-09-17T22:42:11.159 INFO:teuthology.orchestra.run.smithi115.stdout: "write_tier": -1, 2024-09-17T22:42:11.159 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_mode": "none", 2024-09-17T22:42:11.159 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_bytes": 0, 2024-09-17T22:42:11.159 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_objects": 0, 2024-09-17T22:42:11.159 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:42:11.159 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:42:11.159 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:42:11.159 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_flush_age": 0, 2024-09-17T22:42:11.159 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_evict_age": 0, 2024-09-17T22:42:11.159 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profile": "", 2024-09-17T22:42:11.159 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_params": { 2024-09-17T22:42:11.159 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "none" 2024-09-17T22:42:11.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.160 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_period": 0, 2024-09-17T22:42:11.160 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_count": 0, 2024-09-17T22:42:11.160 INFO:teuthology.orchestra.run.smithi115.stdout: "use_gmt_hitset": true, 2024-09-17T22:42:11.160 INFO:teuthology.orchestra.run.smithi115.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:42:11.160 INFO:teuthology.orchestra.run.smithi115.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:42:11.160 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:42:11.160 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:42:11.160 INFO:teuthology.orchestra.run.smithi115.stdout: "grade_table": [], 2024-09-17T22:42:11.160 INFO:teuthology.orchestra.run.smithi115.stdout: "stripe_width": 0, 2024-09-17T22:42:11.160 INFO:teuthology.orchestra.run.smithi115.stdout: "expected_num_objects": 0, 2024-09-17T22:42:11.160 INFO:teuthology.orchestra.run.smithi115.stdout: "fast_read": false, 2024-09-17T22:42:11.160 INFO:teuthology.orchestra.run.smithi115.stdout: "options": { 2024-09-17T22:42:11.160 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_max": 32, 2024-09-17T22:42:11.161 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_min": 1 2024-09-17T22:42:11.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.161 INFO:teuthology.orchestra.run.smithi115.stdout: "application_metadata": { 2024-09-17T22:42:11.161 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": {} 2024-09-17T22:42:11.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.161 INFO:teuthology.orchestra.run.smithi115.stdout: "read_balance": { 2024-09-17T22:42:11.161 INFO:teuthology.orchestra.run.smithi115.stdout: "score_type": "Fair distribution", 2024-09-17T22:42:11.161 INFO:teuthology.orchestra.run.smithi115.stdout: "score_acting": 8, 2024-09-17T22:42:11.161 INFO:teuthology.orchestra.run.smithi115.stdout: "score_stable": 8, 2024-09-17T22:42:11.161 INFO:teuthology.orchestra.run.smithi115.stdout: "optimal_score": 0.25, 2024-09-17T22:42:11.161 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_acting": 2, 2024-09-17T22:42:11.161 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_stable": 2, 2024-09-17T22:42:11.161 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity_weighted": 1, 2024-09-17T22:42:11.161 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity": 1, 2024-09-17T22:42:11.162 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity_weighted": 1 2024-09-17T22:42:11.162 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.162 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-17T22:42:11.162 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_name": "rbd", 2024-09-17T22:42:11.162 INFO:teuthology.orchestra.run.smithi115.stdout: "create_time": "2024-09-17T22:41:08.097861+0000", 2024-09-17T22:42:11.162 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 8193, 2024-09-17T22:42:11.162 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T22:42:11.162 INFO:teuthology.orchestra.run.smithi115.stdout: "type": 1, 2024-09-17T22:42:11.162 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 2, 2024-09-17T22:42:11.162 INFO:teuthology.orchestra.run.smithi115.stdout: "min_size": 1, 2024-09-17T22:42:11.162 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_rule": 0, 2024-09-17T22:42:11.162 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:42:11.163 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:42:11.163 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:42:11.163 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:42:11.163 INFO:teuthology.orchestra.run.smithi115.stdout: "is_stretch_pool": false, 2024-09-17T22:42:11.163 INFO:teuthology.orchestra.run.smithi115.stdout: "object_hash": 2, 2024-09-17T22:42:11.163 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:42:11.163 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num": 8, 2024-09-17T22:42:11.163 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num": 8, 2024-09-17T22:42:11.163 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num_target": 8, 2024-09-17T22:42:11.163 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_target": 8, 2024-09-17T22:42:11.163 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_pending": 8, 2024-09-17T22:42:11.163 INFO:teuthology.orchestra.run.smithi115.stdout: "last_pg_merge_meta": { 2024-09-17T22:42:11.163 INFO:teuthology.orchestra.run.smithi115.stdout: "source_pgid": "0.0", 2024-09-17T22:42:11.163 INFO:teuthology.orchestra.run.smithi115.stdout: "ready_epoch": 0, 2024-09-17T22:42:11.163 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_started": 0, 2024-09-17T22:42:11.164 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_clean": 0, 2024-09-17T22:42:11.164 INFO:teuthology.orchestra.run.smithi115.stdout: "source_version": "0'0", 2024-09-17T22:42:11.164 INFO:teuthology.orchestra.run.smithi115.stdout: "target_version": "0'0" 2024-09-17T22:42:11.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.164 INFO:teuthology.orchestra.run.smithi115.stdout: "last_change": "21", 2024-09-17T22:42:11.164 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend": "0", 2024-09-17T22:42:11.164 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:42:11.164 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:42:11.164 INFO:teuthology.orchestra.run.smithi115.stdout: "auid": 0, 2024-09-17T22:42:11.164 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:42:11.164 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_seq": 2, 2024-09-17T22:42:11.164 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_epoch": 21, 2024-09-17T22:42:11.164 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_snaps": [], 2024-09-17T22:42:11.165 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps": "[]", 2024-09-17T22:42:11.165 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_bytes": 0, 2024-09-17T22:42:11.165 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_objects": 0, 2024-09-17T22:42:11.165 INFO:teuthology.orchestra.run.smithi115.stdout: "tiers": [], 2024-09-17T22:42:11.165 INFO:teuthology.orchestra.run.smithi115.stdout: "tier_of": -1, 2024-09-17T22:42:11.165 INFO:teuthology.orchestra.run.smithi115.stdout: "read_tier": -1, 2024-09-17T22:42:11.165 INFO:teuthology.orchestra.run.smithi115.stdout: "write_tier": -1, 2024-09-17T22:42:11.165 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_mode": "none", 2024-09-17T22:42:11.165 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_bytes": 0, 2024-09-17T22:42:11.165 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_objects": 0, 2024-09-17T22:42:11.165 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:42:11.165 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:42:11.165 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:42:11.165 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_flush_age": 0, 2024-09-17T22:42:11.166 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_evict_age": 0, 2024-09-17T22:42:11.166 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profile": "", 2024-09-17T22:42:11.166 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_params": { 2024-09-17T22:42:11.166 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "none" 2024-09-17T22:42:11.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.166 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_period": 0, 2024-09-17T22:42:11.166 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_count": 0, 2024-09-17T22:42:11.166 INFO:teuthology.orchestra.run.smithi115.stdout: "use_gmt_hitset": true, 2024-09-17T22:42:11.166 INFO:teuthology.orchestra.run.smithi115.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:42:11.166 INFO:teuthology.orchestra.run.smithi115.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:42:11.166 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:42:11.166 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:42:11.166 INFO:teuthology.orchestra.run.smithi115.stdout: "grade_table": [], 2024-09-17T22:42:11.166 INFO:teuthology.orchestra.run.smithi115.stdout: "stripe_width": 0, 2024-09-17T22:42:11.166 INFO:teuthology.orchestra.run.smithi115.stdout: "expected_num_objects": 0, 2024-09-17T22:42:11.167 INFO:teuthology.orchestra.run.smithi115.stdout: "fast_read": false, 2024-09-17T22:42:11.167 INFO:teuthology.orchestra.run.smithi115.stdout: "options": {}, 2024-09-17T22:42:11.167 INFO:teuthology.orchestra.run.smithi115.stdout: "application_metadata": { 2024-09-17T22:42:11.167 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd": {} 2024-09-17T22:42:11.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.167 INFO:teuthology.orchestra.run.smithi115.stdout: "read_balance": { 2024-09-17T22:42:11.167 INFO:teuthology.orchestra.run.smithi115.stdout: "score_type": "Fair distribution", 2024-09-17T22:42:11.167 INFO:teuthology.orchestra.run.smithi115.stdout: "score_acting": 5, 2024-09-17T22:42:11.167 INFO:teuthology.orchestra.run.smithi115.stdout: "score_stable": 5, 2024-09-17T22:42:11.167 INFO:teuthology.orchestra.run.smithi115.stdout: "optimal_score": 0.75, 2024-09-17T22:42:11.167 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_acting": 3.75, 2024-09-17T22:42:11.167 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_stable": 3.75, 2024-09-17T22:42:11.167 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity_weighted": 1, 2024-09-17T22:42:11.167 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity": 1, 2024-09-17T22:42:11.168 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity_weighted": 1 2024-09-17T22:42:11.168 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.168 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 3, 2024-09-17T22:42:11.168 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_name": "pool_client.0", 2024-09-17T22:42:11.168 INFO:teuthology.orchestra.run.smithi115.stdout: "create_time": "2024-09-17T22:41:17.501542+0000", 2024-09-17T22:42:11.168 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 8193, 2024-09-17T22:42:11.168 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T22:42:11.168 INFO:teuthology.orchestra.run.smithi115.stdout: "type": 1, 2024-09-17T22:42:11.168 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 2, 2024-09-17T22:42:11.168 INFO:teuthology.orchestra.run.smithi115.stdout: "min_size": 1, 2024-09-17T22:42:11.168 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_rule": 0, 2024-09-17T22:42:11.168 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:42:11.169 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:42:11.169 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:42:11.169 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:42:11.169 INFO:teuthology.orchestra.run.smithi115.stdout: "is_stretch_pool": false, 2024-09-17T22:42:11.169 INFO:teuthology.orchestra.run.smithi115.stdout: "object_hash": 2, 2024-09-17T22:42:11.169 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:42:11.169 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num": 32, 2024-09-17T22:42:11.169 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num": 32, 2024-09-17T22:42:11.169 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num_target": 32, 2024-09-17T22:42:11.169 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_target": 32, 2024-09-17T22:42:11.169 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_pending": 32, 2024-09-17T22:42:11.169 INFO:teuthology.orchestra.run.smithi115.stdout: "last_pg_merge_meta": { 2024-09-17T22:42:11.169 INFO:teuthology.orchestra.run.smithi115.stdout: "source_pgid": "0.0", 2024-09-17T22:42:11.169 INFO:teuthology.orchestra.run.smithi115.stdout: "ready_epoch": 0, 2024-09-17T22:42:11.169 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_started": 0, 2024-09-17T22:42:11.170 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_clean": 0, 2024-09-17T22:42:11.170 INFO:teuthology.orchestra.run.smithi115.stdout: "source_version": "0'0", 2024-09-17T22:42:11.170 INFO:teuthology.orchestra.run.smithi115.stdout: "target_version": "0'0" 2024-09-17T22:42:11.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.170 INFO:teuthology.orchestra.run.smithi115.stdout: "last_change": "66", 2024-09-17T22:42:11.170 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend": "0", 2024-09-17T22:42:11.170 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:42:11.170 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:42:11.170 INFO:teuthology.orchestra.run.smithi115.stdout: "auid": 0, 2024-09-17T22:42:11.170 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:42:11.170 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_seq": 27, 2024-09-17T22:42:11.170 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_epoch": 66, 2024-09-17T22:42:11.170 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_snaps": [], 2024-09-17T22:42:11.171 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps": "[]", 2024-09-17T22:42:11.171 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_bytes": 0, 2024-09-17T22:42:11.171 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_objects": 0, 2024-09-17T22:42:11.171 INFO:teuthology.orchestra.run.smithi115.stdout: "tiers": [], 2024-09-17T22:42:11.171 INFO:teuthology.orchestra.run.smithi115.stdout: "tier_of": -1, 2024-09-17T22:42:11.171 INFO:teuthology.orchestra.run.smithi115.stdout: "read_tier": -1, 2024-09-17T22:42:11.171 INFO:teuthology.orchestra.run.smithi115.stdout: "write_tier": -1, 2024-09-17T22:42:11.171 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_mode": "none", 2024-09-17T22:42:11.171 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_bytes": 0, 2024-09-17T22:42:11.171 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_objects": 0, 2024-09-17T22:42:11.171 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:42:11.171 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:42:11.171 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:42:11.171 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_flush_age": 0, 2024-09-17T22:42:11.172 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_evict_age": 0, 2024-09-17T22:42:11.172 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profile": "", 2024-09-17T22:42:11.172 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_params": { 2024-09-17T22:42:11.172 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "none" 2024-09-17T22:42:11.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.172 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_period": 0, 2024-09-17T22:42:11.172 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_count": 0, 2024-09-17T22:42:11.172 INFO:teuthology.orchestra.run.smithi115.stdout: "use_gmt_hitset": true, 2024-09-17T22:42:11.172 INFO:teuthology.orchestra.run.smithi115.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:42:11.172 INFO:teuthology.orchestra.run.smithi115.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:42:11.172 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:42:11.172 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:42:11.172 INFO:teuthology.orchestra.run.smithi115.stdout: "grade_table": [], 2024-09-17T22:42:11.172 INFO:teuthology.orchestra.run.smithi115.stdout: "stripe_width": 0, 2024-09-17T22:42:11.173 INFO:teuthology.orchestra.run.smithi115.stdout: "expected_num_objects": 0, 2024-09-17T22:42:11.173 INFO:teuthology.orchestra.run.smithi115.stdout: "fast_read": false, 2024-09-17T22:42:11.173 INFO:teuthology.orchestra.run.smithi115.stdout: "options": {}, 2024-09-17T22:42:11.173 INFO:teuthology.orchestra.run.smithi115.stdout: "application_metadata": { 2024-09-17T22:42:11.173 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd": {} 2024-09-17T22:42:11.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.173 INFO:teuthology.orchestra.run.smithi115.stdout: "read_balance": { 2024-09-17T22:42:11.173 INFO:teuthology.orchestra.run.smithi115.stdout: "score_type": "Fair distribution", 2024-09-17T22:42:11.173 INFO:teuthology.orchestra.run.smithi115.stdout: "score_acting": 1.7400000095367432, 2024-09-17T22:42:11.173 INFO:teuthology.orchestra.run.smithi115.stdout: "score_stable": 1.7400000095367432, 2024-09-17T22:42:11.173 INFO:teuthology.orchestra.run.smithi115.stdout: "optimal_score": 0.87999999523162842, 2024-09-17T22:42:11.173 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-17T22:42:11.173 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-17T22:42:11.173 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity_weighted": 1, 2024-09-17T22:42:11.174 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity": 1, 2024-09-17T22:42:11.174 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity_weighted": 1 2024-09-17T22:42:11.174 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.174 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.174 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:11.174 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:42:11.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.174 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 0, 2024-09-17T22:42:11.174 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "3bd5c11f-e7c2-4968-839a-2799c00d93a6", 2024-09-17T22:42:11.174 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 0, 2024-09-17T22:42:11.174 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:11.174 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:11.174 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:11.174 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:11.175 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:11.175 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:11.175 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 23, 2024-09-17T22:42:11.175 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 51, 2024-09-17T22:42:11.175 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:11.175 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:11.175 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.175 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.175 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6824", 2024-09-17T22:42:11.175 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:11.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.175 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.176 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6825", 2024-09-17T22:42:11.176 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:11.176 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.176 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.176 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:11.176 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.176 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.176 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6826", 2024-09-17T22:42:11.176 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:11.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.177 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.177 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6827", 2024-09-17T22:42:11.177 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:11.177 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.177 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.177 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:11.177 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.177 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.178 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6830", 2024-09-17T22:42:11.178 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:11.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.178 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.178 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6831", 2024-09-17T22:42:11.178 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:11.178 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.178 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.178 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:11.178 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.178 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6828", 2024-09-17T22:42:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6829", 2024-09-17T22:42:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6825/2782271783", 2024-09-17T22:42:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6827/2782271783", 2024-09-17T22:42:11.179 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6831/2782271783", 2024-09-17T22:42:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6829/2782271783", 2024-09-17T22:42:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "exists" 2024-09-17T22:42:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 1, 2024-09-17T22:42:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "02918203-cf2e-49fc-b994-743b5d81b4a1", 2024-09-17T22:42:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:11.180 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 51, 2024-09-17T22:42:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6816", 2024-09-17T22:42:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.181 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6817", 2024-09-17T22:42:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6818", 2024-09-17T22:42:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.182 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6819", 2024-09-17T22:42:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6822", 2024-09-17T22:42:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.183 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6823", 2024-09-17T22:42:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6820", 2024-09-17T22:42:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.184 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6821", 2024-09-17T22:42:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6817/1941780390", 2024-09-17T22:42:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6819/1941780390", 2024-09-17T22:42:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6823/1941780390", 2024-09-17T22:42:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6821/1941780390", 2024-09-17T22:42:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 2, 2024-09-17T22:42:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "22ee34ef-522e-4121-85b7-38bbafeca5de", 2024-09-17T22:42:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 13, 2024-09-17T22:42:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 51, 2024-09-17T22:42:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:11.186 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6808", 2024-09-17T22:42:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6809", 2024-09-17T22:42:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:11.187 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6810", 2024-09-17T22:42:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6811", 2024-09-17T22:42:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.188 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6814", 2024-09-17T22:42:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6815", 2024-09-17T22:42:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6812", 2024-09-17T22:42:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6813", 2024-09-17T22:42:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6809/689014698", 2024-09-17T22:42:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6811/689014698", 2024-09-17T22:42:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6815/689014698", 2024-09-17T22:42:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6813/689014698", 2024-09-17T22:42:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 3, 2024-09-17T22:42:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "5d464921-5696-43c5-b89d-430a54ddefb5", 2024-09-17T22:42:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:11.191 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 13, 2024-09-17T22:42:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 54, 2024-09-17T22:42:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6800", 2024-09-17T22:42:11.192 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6801", 2024-09-17T22:42:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.193 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6802", 2024-09-17T22:42:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6803", 2024-09-17T22:42:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6806", 2024-09-17T22:42:11.194 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6807", 2024-09-17T22:42:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6804", 2024-09-17T22:42:11.195 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6805", 2024-09-17T22:42:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6801/453192890", 2024-09-17T22:42:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6803/453192890", 2024-09-17T22:42:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6807/453192890", 2024-09-17T22:42:11.196 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6805/453192890", 2024-09-17T22:42:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 4, 2024-09-17T22:42:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "4cb6973a-63ae-4f48-af94-37610aee3c06", 2024-09-17T22:42:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:11.197 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 51, 2024-09-17T22:42:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6800", 2024-09-17T22:42:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6801", 2024-09-17T22:42:11.198 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6802", 2024-09-17T22:42:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6803", 2024-09-17T22:42:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:11.199 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6806", 2024-09-17T22:42:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6807", 2024-09-17T22:42:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:11.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6804", 2024-09-17T22:42:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6805", 2024-09-17T22:42:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:11.201 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6801/380388860", 2024-09-17T22:42:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6803/380388860", 2024-09-17T22:42:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6807/380388860", 2024-09-17T22:42:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6805/380388860", 2024-09-17T22:42:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 5, 2024-09-17T22:42:11.202 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "ef9fe11b-b719-48f4-9c6c-e1b1b355acdc", 2024-09-17T22:42:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 23, 2024-09-17T22:42:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.203 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6808", 2024-09-17T22:42:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6809", 2024-09-17T22:42:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6810", 2024-09-17T22:42:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6811", 2024-09-17T22:42:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:11.205 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6814", 2024-09-17T22:42:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6815", 2024-09-17T22:42:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:11.206 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6812", 2024-09-17T22:42:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6813", 2024-09-17T22:42:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6809/943488949", 2024-09-17T22:42:11.207 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6811/943488949", 2024-09-17T22:42:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6815/943488949", 2024-09-17T22:42:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6813/943488949", 2024-09-17T22:42:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 6, 2024-09-17T22:42:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "e4045aec-22c1-401b-bfbf-64f115bc32af", 2024-09-17T22:42:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:11.208 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 54, 2024-09-17T22:42:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6816", 2024-09-17T22:42:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6817", 2024-09-17T22:42:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6818", 2024-09-17T22:42:11.210 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6819", 2024-09-17T22:42:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6822", 2024-09-17T22:42:11.211 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6823", 2024-09-17T22:42:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6820", 2024-09-17T22:42:11.212 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6821", 2024-09-17T22:42:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6817/1873501077", 2024-09-17T22:42:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6819/1873501077", 2024-09-17T22:42:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6823/1873501077", 2024-09-17T22:42:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6821/1873501077", 2024-09-17T22:42:11.213 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 7, 2024-09-17T22:42:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "4a05a5d7-8281-4572-a9d0-6095ee5e168b", 2024-09-17T22:42:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:11.214 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 54, 2024-09-17T22:42:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6824", 2024-09-17T22:42:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6825", 2024-09-17T22:42:11.215 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6826", 2024-09-17T22:42:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.216 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6827", 2024-09-17T22:42:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6830", 2024-09-17T22:42:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.217 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6831", 2024-09-17T22:42:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6828", 2024-09-17T22:42:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:11.218 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6829", 2024-09-17T22:42:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6825/3609357666", 2024-09-17T22:42:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6827/3609357666", 2024-09-17T22:42:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6831/3609357666", 2024-09-17T22:42:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6829/3609357666", 2024-09-17T22:42:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:11.219 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_xinfo": [ 2024-09-17T22:42:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 0, 2024-09-17T22:42:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:59.351758+0000", 2024-09-17T22:42:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 51 2024-09-17T22:42:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 1, 2024-09-17T22:42:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:11.220 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:59.058915+0000", 2024-09-17T22:42:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 2, 2024-09-17T22:42:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:11.221 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.922383+0000", 2024-09-17T22:42:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 3, 2024-09-17T22:42:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.802379+0000", 2024-09-17T22:42:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 4, 2024-09-17T22:42:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.881681+0000", 2024-09-17T22:42:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 5, 2024-09-17T22:42:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:11.223 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.616649+0000", 2024-09-17T22:42:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 6, 2024-09-17T22:42:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.589555+0000", 2024-09-17T22:42:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:11.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 7, 2024-09-17T22:42:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:59.047783+0000", 2024-09-17T22:42:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_upmap": [], 2024-09-17T22:42:11.225 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_upmap_items": [ 2024-09-17T22:42:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.12", 2024-09-17T22:42:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: "mappings": [ 2024-09-17T22:42:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: "from": 3, 2024-09-17T22:42:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: "to": 0 2024-09-17T22:42:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.16", 2024-09-17T22:42:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: "mappings": [ 2024-09-17T22:42:11.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: "from": 2, 2024-09-17T22:42:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: "to": 0 2024-09-17T22:42:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.1d", 2024-09-17T22:42:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: "mappings": [ 2024-09-17T22:42:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: "from": 3, 2024-09-17T22:42:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: "to": 7 2024-09-17T22:42:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.227 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_upmap_primaries": [], 2024-09-17T22:42:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_temp": [ 2024-09-17T22:42:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.12", 2024-09-17T22:42:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:42:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-09-17T22:42:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: 3 2024-09-17T22:42:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.16", 2024-09-17T22:42:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:42:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-09-17T22:42:11.228 INFO:teuthology.orchestra.run.smithi115.stdout: 2 2024-09-17T22:42:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.1d", 2024-09-17T22:42:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:42:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-09-17T22:42:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: 3 2024-09-17T22:42:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_temp": [], 2024-09-17T22:42:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: "blocklist": {}, 2024-09-17T22:42:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: "range_blocklist": {}, 2024-09-17T22:42:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profiles": { 2024-09-17T22:42:11.229 INFO:teuthology.orchestra.run.smithi115.stdout: "default": { 2024-09-17T22:42:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "crush-failure-domain": "osd", 2024-09-17T22:42:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "k": "2", 2024-09-17T22:42:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "m": "1", 2024-09-17T22:42:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "plugin": "jerasure", 2024-09-17T22:42:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "technique": "reed_sol_van" 2024-09-17T22:42:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps_queue": [], 2024-09-17T22:42:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "new_removed_snaps": [], 2024-09-17T22:42:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "new_purged_snaps": [], 2024-09-17T22:42:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_node_flags": {}, 2024-09-17T22:42:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "device_class_flags": {}, 2024-09-17T22:42:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": { 2024-09-17T22:42:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode_enabled": false, 2024-09-17T22:42:11.230 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_bucket_count": 0, 2024-09-17T22:42:11.231 INFO:teuthology.orchestra.run.smithi115.stdout: "degraded_stretch_mode": 0, 2024-09-17T22:42:11.231 INFO:teuthology.orchestra.run.smithi115.stdout: "recovering_stretch_mode": 0, 2024-09-17T22:42:11.231 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode_bucket": 0 2024-09-17T22:42:11.231 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:11.231 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:11.232 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:11.233 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:11.233 INFO:tasks.thrashosds.thrasher:j is {'epoch': 67, 'fsid': '551c5ed8-e790-485a-bc95-a9578a29275c', 'created': '2024-09-17T22:40:50.189230+0000', 'modified': '2024-09-17T22:42:10.189579+0000', 'last_up_change': '2024-09-17T22:41:53.099515+0000', 'last_in_change': '2024-09-17T22:40:52.260526+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T22:41:02.338356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T22:41:08.097861+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5, 'score_stable': 5, 'optimal_score': 0.75, 'raw_score_acting': 3.75, 'raw_score_stable': 3.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T22:41:17.501542+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '3bd5c11f-e7c2-4968-839a-2799c00d93a6', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 51, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6824', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6825', 'nonce': 2782271783}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 2782271783}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 2782271783}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 2782271783}]}, 'public_addr': '172.21.15.115:6825/2782271783', 'cluster_addr': '172.21.15.115:6827/2782271783', 'heartbeat_back_addr': '172.21.15.115:6831/2782271783', 'heartbeat_front_addr': '172.21.15.115:6829/2782271783', 'state': ['exists']}, {'osd': 1, 'uuid': '02918203-cf2e-49fc-b994-743b5d81b4a1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 1941780390}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 1941780390}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 1941780390}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 1941780390}]}, 'public_addr': '172.21.15.115:6817/1941780390', 'cluster_addr': '172.21.15.115:6819/1941780390', 'heartbeat_back_addr': '172.21.15.115:6823/1941780390', 'heartbeat_front_addr': '172.21.15.115:6821/1941780390', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '22ee34ef-522e-4121-85b7-38bbafeca5de', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 689014698}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 689014698}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 689014698}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 689014698}]}, 'public_addr': '172.21.15.115:6809/689014698', 'cluster_addr': '172.21.15.115:6811/689014698', 'heartbeat_back_addr': '172.21.15.115:6815/689014698', 'heartbeat_front_addr': '172.21.15.115:6813/689014698', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5d464921-5696-43c5-b89d-430a54ddefb5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 453192890}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 453192890}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 453192890}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 453192890}]}, 'public_addr': '172.21.15.115:6801/453192890', 'cluster_addr': '172.21.15.115:6803/453192890', 'heartbeat_back_addr': '172.21.15.115:6807/453192890', 'heartbeat_front_addr': '172.21.15.115:6805/453192890', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '4cb6973a-63ae-4f48-af94-37610aee3c06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6800', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6801', 'nonce': 380388860}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6802', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6803', 'nonce': 380388860}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6806', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6807', 'nonce': 380388860}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6804', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6805', 'nonce': 380388860}]}, 'public_addr': '172.21.15.129:6801/380388860', 'cluster_addr': '172.21.15.129:6803/380388860', 'heartbeat_back_addr': '172.21.15.129:6807/380388860', 'heartbeat_front_addr': '172.21.15.129:6805/380388860', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ef9fe11b-b719-48f4-9c6c-e1b1b355acdc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6808', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6809', 'nonce': 943488949}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6810', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6811', 'nonce': 943488949}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6814', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6815', 'nonce': 943488949}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6812', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6813', 'nonce': 943488949}]}, 'public_addr': '172.21.15.129:6809/943488949', 'cluster_addr': '172.21.15.129:6811/943488949', 'heartbeat_back_addr': '172.21.15.129:6815/943488949', 'heartbeat_front_addr': '172.21.15.129:6813/943488949', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e4045aec-22c1-401b-bfbf-64f115bc32af', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6816', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6817', 'nonce': 1873501077}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6818', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6819', 'nonce': 1873501077}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6822', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6823', 'nonce': 1873501077}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6820', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6821', 'nonce': 1873501077}]}, 'public_addr': '172.21.15.129:6817/1873501077', 'cluster_addr': '172.21.15.129:6819/1873501077', 'heartbeat_back_addr': '172.21.15.129:6823/1873501077', 'heartbeat_front_addr': '172.21.15.129:6821/1873501077', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '4a05a5d7-8281-4572-a9d0-6095ee5e168b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6824', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6825', 'nonce': 3609357666}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6826', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6827', 'nonce': 3609357666}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6830', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6831', 'nonce': 3609357666}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6828', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6829', 'nonce': 3609357666}]}, 'public_addr': '172.21.15.129:6825/3609357666', 'cluster_addr': '172.21.15.129:6827/3609357666', 'heartbeat_back_addr': '172.21.15.129:6831/3609357666', 'heartbeat_front_addr': '172.21.15.129:6829/3609357666', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:59.351758+0000', 'dead_epoch': 51}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:59.058915+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.922383+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.802379+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.881681+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.616649+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.589555+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:59.047783+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.12', 'mappings': [{'from': 3, 'to': 0}]}, {'pgid': '3.16', 'mappings': [{'from': 2, 'to': 0}]}, {'pgid': '3.1d', 'mappings': [{'from': 3, 'to': 7}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.12', 'osds': [6, 3]}, {'pgid': '3.16', 'osds': [3, 2]}, {'pgid': '3.1d', 'osds': [7, 3]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T22:42:11.234 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:42:11.234 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:11.229+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:11.234 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:42:11.267 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone25 from 0xe6c1806 (overlap 0x710433) to 0x388219 2024-09-17T22:42:11.333 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:11.331+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:11.357 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:11.357 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:11.366 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:42:11.375 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-17T22:42:11.435 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:11.432+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:11.535 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:11.533+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:11.550 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:11.550 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:11.563 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:42:11.611 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:42:11.612 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:42:11.626 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-09-17T22:42:11.204105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":954786010,"num_objects":1355,"num_object_clones":313,"num_object_copies":2710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":355,"num_objects_misplaced":121,"num_objects_unfound":0,"num_objects_dirty":1355,"num_whiteouts":720,"num_read":207059,"num_read_kb":538546,"num_write":7635,"num_write_kb":145233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3161143,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6735,"ondisk_log_size":6735,"up":69,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":303304,"kb_used_data":28792,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754671416,"statfs":{"total":773094113280,"available":772783529984,"internally_reserved":0,"allocated":29483008,"data_stored":186588528,"data_compressed":6283017,"data_compressed_allocated":16994304,"data_compressed_original":180275049,"omap_allocated":12750,"internal_metadata":280874546},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49927181,"num_objects":125,"num_object_clones":7,"num_object_copies":250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":9,"num_objects_unfound":0,"num_objects_dirty":125,"num_whiteouts":71,"num_read":3881,"num_read_kb":23525,"num_write":761,"num_write_kb":22282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023846"},"pg_stats":[{"pgid":"3.1f","version":"61'275","reported_seq":6170,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T22:42:06.176818+0000","last_change":"2024-09-17T22:41:21.453392+0000","last_active":"2024-09-17T22:42:06.176818+0000","last_peered":"2024-09-17T22:42:06.176818+0000","last_clean":"2024-09-17T22:42:06.176818+0000","last_became_active":"2024-09-17T22:41:19.443658+0000","last_became_peered":"2024-09-17T22:41:19.443658+0000","last_unstale":"2024-09-17T22:42:06.176818+0000","last_undegraded":"2024-09-17T22:42:06.176818+0000","last_fullsized":"2024-09-17T22:42:06.176818+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:27.266882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047335399999999999,"stat_sum":{"num_bytes":31923746,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":13,"num_read":6726,"num_read_kb":12131,"num_write":320,"num_write_kb":2268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"63'127","reported_seq":5644,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T22:42:06.323650+0000","last_change":"2024-09-17T22:41:21.453587+0000","last_active":"2024-09-17T22:42:06.323650+0000","last_peered":"2024-09-17T22:42:06.323650+0000","last_clean":"2024-09-17T22:42:06.323650+0000","last_became_active":"2024-09-17T22:41:19.449357+0000","last_became_peered":"2024-09-17T22:41:19.449357+0000","last_unstale":"2024-09-17T22:42:06.323650+0000","last_undegraded":"2024-09-17T22:42:06.323650+0000","last_fullsized":"2024-09-17T22:42:06.323650+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:18:21.986416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996099999999995,"stat_sum":{"num_bytes":52593321,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":21,"num_read":5538,"num_read_kb":13074,"num_write":121,"num_write_kb":5202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"63'100","reported_seq":396,"reported_epoch":63,"state":"active+clean+remapped","last_fresh":"2024-09-17T22:42:06.315384+0000","last_change":"2024-09-17T22:41:57.183293+0000","last_active":"2024-09-17T22:42:06.315384+0000","last_peered":"2024-09-17T22:42:06.315384+0000","last_clean":"2024-09-17T22:42:06.315384+0000","last_became_active":"2024-09-17T22:41:57.133365+0000","last_became_peered":"2024-09-17T22:41:57.133365+0000","last_unstale":"2024-09-17T22:42:06.315384+0000","last_undegraded":"2024-09-17T22:42:06.315384+0000","last_fullsized":"2024-09-17T22:42:06.315384+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:28:42.891481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000382498,"stat_sum":{"num_bytes":28005367,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":28,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":14,"num_read":14142,"num_read_kb":40484,"num_write":86,"num_write_kb":4210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3161143,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[{"shards":"3,7","objects":28}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"63'112","reported_seq":1057,"reported_epoch":64,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:07.180424+0000","last_change":"2024-09-17T22:41:54.113405+0000","last_active":"2024-09-17T22:42:07.180424+0000","last_peered":"2024-09-17T22:42:07.180424+0000","last_clean":"2024-09-17T22:41:52.379845+0000","last_became_active":"2024-09-17T22:41:54.113405+0000","last_became_peered":"2024-09-17T22:41:54.113405+0000","last_unstale":"2024-09-17T22:42:07.180424+0000","last_undegraded":"2024-09-17T22:41:54.112692+0000","last_fullsized":"2024-09-17T22:41:54.112591+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:55:17.582500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001451078,"stat_sum":{"num_bytes":8912322,"num_objects":33,"num_object_clones":11,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":18,"num_read":844,"num_read_kb":2989,"num_write":98,"num_write_kb":4274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":33}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"63'105","reported_seq":7055,"reported_epoch":63,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:06.328809+0000","last_change":"2024-09-17T22:41:54.118027+0000","last_active":"2024-09-17T22:42:06.328809+0000","last_peered":"2024-09-17T22:42:06.328809+0000","last_clean":"2024-09-17T22:41:52.368521+0000","last_became_active":"2024-09-17T22:41:54.118027+0000","last_became_peered":"2024-09-17T22:41:54.118027+0000","last_unstale":"2024-09-17T22:42:06.328809+0000","last_undegraded":"2024-09-17T22:41:54.116518+0000","last_fullsized":"2024-09-17T22:41:54.116411+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:38:44.478612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055813,"stat_sum":{"num_bytes":24531964,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":19,"num_read":6878,"num_read_kb":16169,"num_write":86,"num_write_kb":2634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":36}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"64'200","reported_seq":1874,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T22:42:07.304842+0000","last_change":"2024-09-17T22:41:21.446945+0000","last_active":"2024-09-17T22:42:07.304842+0000","last_peered":"2024-09-17T22:42:07.304842+0000","last_clean":"2024-09-17T22:42:07.304842+0000","last_became_active":"2024-09-17T22:41:19.446541+0000","last_became_peered":"2024-09-17T22:41:19.446541+0000","last_unstale":"2024-09-17T22:42:07.304842+0000","last_undegraded":"2024-09-17T22:42:07.304842+0000","last_fullsized":"2024-09-17T22:42:07.304842+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:36:58.913459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001630631,"stat_sum":{"num_bytes":32473976,"num_objects":55,"num_object_clones":9,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":35,"num_read":2129,"num_read_kb":10509,"num_write":198,"num_write_kb":3426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"63'146","reported_seq":5589,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T22:42:06.260685+0000","last_change":"2024-09-17T22:41:21.448965+0000","last_active":"2024-09-17T22:42:06.260685+0000","last_peered":"2024-09-17T22:42:06.260685+0000","last_clean":"2024-09-17T22:42:06.260685+0000","last_became_active":"2024-09-17T22:41:19.449940+0000","last_became_peered":"2024-09-17T22:41:19.449940+0000","last_unstale":"2024-09-17T22:42:06.260685+0000","last_undegraded":"2024-09-17T22:42:06.260685+0000","last_fullsized":"2024-09-17T22:42:06.260685+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:38.539217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011385119999999999,"stat_sum":{"num_bytes":31006981,"num_objects":44,"num_object_clones":11,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":20,"num_read":5487,"num_read_kb":18276,"num_write":137,"num_write_kb":7546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"63'130","reported_seq":2108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T22:42:06.315007+0000","last_change":"2024-09-17T22:41:21.453701+0000","last_active":"2024-09-17T22:42:06.315007+0000","last_peered":"2024-09-17T22:42:06.315007+0000","last_clean":"2024-09-17T22:42:06.315007+0000","last_became_active":"2024-09-17T22:41:19.446287+0000","last_became_peered":"2024-09-17T22:41:19.446287+0000","last_unstale":"2024-09-17T22:42:06.315007+0000","last_undegraded":"2024-09-17T22:42:06.315007+0000","last_fullsized":"2024-09-17T22:42:06.315007+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:00:41.050727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067184299999999998,"stat_sum":{"num_bytes":74716059,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":19,"num_read":2373,"num_read_kb":13558,"num_write":110,"num_write_kb":6588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"63'166","reported_seq":1108,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T22:42:06.320202+0000","last_change":"2024-09-17T22:41:21.453344+0000","last_active":"2024-09-17T22:42:06.320202+0000","last_peered":"2024-09-17T22:42:06.320202+0000","last_clean":"2024-09-17T22:42:06.320202+0000","last_became_active":"2024-09-17T22:41:19.445923+0000","last_became_peered":"2024-09-17T22:41:19.445923+0000","last_unstale":"2024-09-17T22:42:06.320202+0000","last_undegraded":"2024-09-17T22:42:06.320202+0000","last_fullsized":"2024-09-17T22:42:06.320202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:46:21.493934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043560199999999999,"stat_sum":{"num_bytes":15020697,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":23,"num_read":1200,"num_read_kb":5788,"num_write":194,"num_write_kb":3455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"66'340","reported_seq":21452,"reported_epoch":67,"state":"active+clean+remapped","last_fresh":"2024-09-17T22:42:10.197109+0000","last_change":"2024-09-17T22:41:57.132629+0000","last_active":"2024-09-17T22:42:10.197109+0000","last_peered":"2024-09-17T22:42:10.197109+0000","last_clean":"2024-09-17T22:42:10.197109+0000","last_became_active":"2024-09-17T22:41:57.132018+0000","last_became_peered":"2024-09-17T22:41:57.132018+0000","last_unstale":"2024-09-17T22:42:10.197109+0000","last_undegraded":"2024-09-17T22:42:10.197109+0000","last_fullsized":"2024-09-17T22:42:10.197109+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:03.287602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000275164,"stat_sum":{"num_bytes":47298696,"num_objects":58,"num_object_clones":17,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":58,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":31,"num_read":22312,"num_read_kb":59404,"num_write":433,"num_write_kb":9090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":58}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"63'181","reported_seq":5865,"reported_epoch":63,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:06.291488+0000","last_change":"2024-09-17T22:41:54.119655+0000","last_active":"2024-09-17T22:42:06.291488+0000","last_peered":"2024-09-17T22:42:06.291488+0000","last_clean":"2024-09-17T22:41:52.245886+0000","last_became_active":"2024-09-17T22:41:54.119655+0000","last_became_peered":"2024-09-17T22:41:54.119655+0000","last_unstale":"2024-09-17T22:42:06.291488+0000","last_undegraded":"2024-09-17T22:41:54.119161+0000","last_fullsized":"2024-09-17T22:41:54.119071+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:06:43.774619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041055799999999999,"stat_sum":{"num_bytes":58442445,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":21,"num_read":6090,"num_read_kb":11416,"num_write":136,"num_write_kb":2689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":36}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"66'401","reported_seq":7011,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-17T22:42:10.197533+0000","last_change":"2024-09-17T22:41:21.446210+0000","last_active":"2024-09-17T22:42:10.197533+0000","last_peered":"2024-09-17T22:42:10.197533+0000","last_clean":"2024-09-17T22:42:10.197533+0000","last_became_active":"2024-09-17T22:41:19.451631+0000","last_became_peered":"2024-09-17T22:41:19.451631+0000","last_unstale":"2024-09-17T22:42:10.197533+0000","last_undegraded":"2024-09-17T22:42:10.197533+0000","last_fullsized":"2024-09-17T22:42:10.197533+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:23:20.074594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081193699999999999,"stat_sum":{"num_bytes":21610776,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":17,"num_read":7957,"num_read_kb":13263,"num_write":586,"num_write_kb":14717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":113,"reported_epoch":67,"state":"active+undersized","last_fresh":"2024-09-17T22:42:10.197892+0000","last_change":"2024-09-17T22:41:54.118987+0000","last_active":"2024-09-17T22:42:10.197892+0000","last_peered":"2024-09-17T22:42:10.197892+0000","last_clean":"2024-09-17T22:41:52.098057+0000","last_became_active":"2024-09-17T22:41:54.118987+0000","last_became_peered":"2024-09-17T22:41:54.118987+0000","last_unstale":"2024-09-17T22:42:10.197892+0000","last_undegraded":"2024-09-17T22:42:10.197892+0000","last_fullsized":"2024-09-17T22:41:54.114557+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.941682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"63'186","reported_seq":1407,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T22:42:06.346064+0000","last_change":"2024-09-17T22:41:21.448698+0000","last_active":"2024-09-17T22:42:06.346064+0000","last_peered":"2024-09-17T22:42:06.346064+0000","last_clean":"2024-09-17T22:42:06.346064+0000","last_became_active":"2024-09-17T22:41:19.448966+0000","last_became_peered":"2024-09-17T22:41:19.448966+0000","last_unstale":"2024-09-17T22:42:06.346064+0000","last_undegraded":"2024-09-17T22:42:06.346064+0000","last_fullsized":"2024-09-17T22:42:06.346064+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:57.669430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089570000000000003,"stat_sum":{"num_bytes":15484594,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":20,"num_read":1566,"num_read_kb":11696,"num_write":238,"num_write_kb":2896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":105,"reported_epoch":63,"state":"active+undersized","last_fresh":"2024-09-17T22:42:06.174638+0000","last_change":"2024-09-17T22:41:54.118378+0000","last_active":"2024-09-17T22:42:06.174638+0000","last_peered":"2024-09-17T22:42:06.174638+0000","last_clean":"2024-09-17T22:41:52.097696+0000","last_became_active":"2024-09-17T22:41:54.118378+0000","last_became_peered":"2024-09-17T22:41:54.118378+0000","last_unstale":"2024-09-17T22:42:06.174638+0000","last_undegraded":"2024-09-17T22:42:06.174638+0000","last_fullsized":"2024-09-17T22:41:54.116433+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:48:40.138581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"63'211","reported_seq":894,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T22:42:06.334035+0000","last_change":"2024-09-17T22:41:21.448596+0000","last_active":"2024-09-17T22:42:06.334035+0000","last_peered":"2024-09-17T22:42:06.334035+0000","last_clean":"2024-09-17T22:42:06.334035+0000","last_became_active":"2024-09-17T22:41:19.451637+0000","last_became_peered":"2024-09-17T22:41:19.451637+0000","last_unstale":"2024-09-17T22:42:06.334035+0000","last_undegraded":"2024-09-17T22:42:06.334035+0000","last_fullsized":"2024-09-17T22:42:06.334035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:36:14.665645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085737499999999996,"stat_sum":{"num_bytes":43142865,"num_objects":50,"num_object_clones":16,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":28,"num_read":1098,"num_read_kb":6589,"num_write":260,"num_write_kb":4278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":111,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-17T22:42:10.197789+0000","last_change":"2024-09-17T22:41:11.397753+0000","last_active":"2024-09-17T22:42:10.197789+0000","last_peered":"2024-09-17T22:42:10.197789+0000","last_clean":"2024-09-17T22:42:10.197789+0000","last_became_active":"2024-09-17T22:41:09.387651+0000","last_became_peered":"2024-09-17T22:41:09.387651+0000","last_unstale":"2024-09-17T22:42:10.197789+0000","last_undegraded":"2024-09-17T22:42:10.197789+0000","last_fullsized":"2024-09-17T22:42:10.197789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:14:30.915047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"63'96","reported_seq":579,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T22:42:06.332964+0000","last_change":"2024-09-17T22:41:21.466363+0000","last_active":"2024-09-17T22:42:06.332964+0000","last_peered":"2024-09-17T22:42:06.332964+0000","last_clean":"2024-09-17T22:42:06.332964+0000","last_became_active":"2024-09-17T22:41:19.444893+0000","last_became_peered":"2024-09-17T22:41:19.444893+0000","last_unstale":"2024-09-17T22:42:06.332964+0000","last_undegraded":"2024-09-17T22:42:06.332964+0000","last_fullsized":"2024-09-17T22:42:06.332964+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:51:10.726867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053617200000000002,"stat_sum":{"num_bytes":20943414,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":17,"num_read":405,"num_read_kb":2628,"num_write":89,"num_write_kb":1664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":103,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T22:42:06.176682+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:42:06.176682+0000","last_peered":"2024-09-17T22:42:06.176682+0000","last_clean":"2024-09-17T22:42:06.176682+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:42:06.176682+0000","last_undegraded":"2024-09-17T22:42:06.176682+0000","last_fullsized":"2024-09-17T22:42:06.176682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"66'211","reported_seq":14899,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-17T22:42:10.202773+0000","last_change":"2024-09-17T22:41:21.516349+0000","last_active":"2024-09-17T22:42:10.202773+0000","last_peered":"2024-09-17T22:42:10.202773+0000","last_clean":"2024-09-17T22:42:10.202773+0000","last_became_active":"2024-09-17T22:41:19.451723+0000","last_became_peered":"2024-09-17T22:41:19.451723+0000","last_unstale":"2024-09-17T22:42:10.202773+0000","last_undegraded":"2024-09-17T22:42:10.202773+0000","last_fullsized":"2024-09-17T22:42:10.202773+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:43:42.451765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040033199999999999,"stat_sum":{"num_bytes":40486490,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":17,"num_read":15104,"num_read_kb":27585,"num_write":267,"num_write_kb":3377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"66'427","reported_seq":10127,"reported_epoch":67,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:10.202758+0000","last_change":"2024-09-17T22:41:54.119266+0000","last_active":"2024-09-17T22:42:10.202758+0000","last_peered":"2024-09-17T22:42:10.202758+0000","last_clean":"2024-09-17T22:41:52.262287+0000","last_became_active":"2024-09-17T22:41:54.119266+0000","last_became_peered":"2024-09-17T22:41:54.119266+0000","last_unstale":"2024-09-17T22:42:10.202758+0000","last_undegraded":"2024-09-17T22:41:54.116326+0000","last_fullsized":"2024-09-17T22:41:54.116242+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:59:04.155627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063898899999999996,"stat_sum":{"num_bytes":32468495,"num_objects":66,"num_object_clones":14,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":66,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":39,"num_read":12842,"num_read_kb":25498,"num_write":480,"num_write_kb":4924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":66}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":113,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-17T22:42:10.197792+0000","last_change":"2024-09-17T22:41:11.397488+0000","last_active":"2024-09-17T22:42:10.197792+0000","last_peered":"2024-09-17T22:42:10.197792+0000","last_clean":"2024-09-17T22:42:10.197792+0000","last_became_active":"2024-09-17T22:41:09.651390+0000","last_became_peered":"2024-09-17T22:41:09.651390+0000","last_unstale":"2024-09-17T22:42:10.197792+0000","last_undegraded":"2024-09-17T22:42:10.197792+0000","last_fullsized":"2024-09-17T22:42:10.197792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:00.598921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"61'163","reported_seq":3129,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T22:42:06.176742+0000","last_change":"2024-09-17T22:41:21.453941+0000","last_active":"2024-09-17T22:42:06.176742+0000","last_peered":"2024-09-17T22:42:06.176742+0000","last_clean":"2024-09-17T22:42:06.176742+0000","last_became_active":"2024-09-17T22:41:19.444314+0000","last_became_peered":"2024-09-17T22:41:19.444314+0000","last_unstale":"2024-09-17T22:42:06.176742+0000","last_undegraded":"2024-09-17T22:42:06.176742+0000","last_fullsized":"2024-09-17T22:42:06.176742+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:34:42.693333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085427099999999998,"stat_sum":{"num_bytes":29345411,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":18,"num_read":3398,"num_read_kb":14247,"num_write":165,"num_write_kb":5763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":103,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T22:42:06.230710+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:42:06.230710+0000","last_peered":"2024-09-17T22:42:06.230710+0000","last_clean":"2024-09-17T22:42:06.230710+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:42:06.230710+0000","last_undegraded":"2024-09-17T22:42:06.230710+0000","last_fullsized":"2024-09-17T22:42:06.230710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"63'216","reported_seq":6208,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T22:42:06.335870+0000","last_change":"2024-09-17T22:41:21.466094+0000","last_active":"2024-09-17T22:42:06.335870+0000","last_peered":"2024-09-17T22:42:06.335870+0000","last_clean":"2024-09-17T22:42:06.335870+0000","last_became_active":"2024-09-17T22:41:19.449791+0000","last_became_peered":"2024-09-17T22:41:19.449791+0000","last_unstale":"2024-09-17T22:42:06.335870+0000","last_undegraded":"2024-09-17T22:42:06.335870+0000","last_fullsized":"2024-09-17T22:42:06.335870+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:31.090448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042775300000000001,"stat_sum":{"num_bytes":11841786,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":18,"num_read":6545,"num_read_kb":5003,"num_write":286,"num_write_kb":1562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":111,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-17T22:42:10.197836+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:42:10.197836+0000","last_peered":"2024-09-17T22:42:10.197836+0000","last_clean":"2024-09-17T22:42:10.197836+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:42:10.197836+0000","last_undegraded":"2024-09-17T22:42:10.197836+0000","last_fullsized":"2024-09-17T22:42:10.197836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"62'311","reported_seq":6623,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T22:42:06.345687+0000","last_change":"2024-09-17T22:41:21.448409+0000","last_active":"2024-09-17T22:42:06.345687+0000","last_peered":"2024-09-17T22:42:06.345687+0000","last_clean":"2024-09-17T22:42:06.345687+0000","last_became_active":"2024-09-17T22:41:19.449279+0000","last_became_peered":"2024-09-17T22:41:19.449279+0000","last_unstale":"2024-09-17T22:42:06.345687+0000","last_undegraded":"2024-09-17T22:42:06.345687+0000","last_fullsized":"2024-09-17T22:42:06.345687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:58:02.312906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075217299999999997,"stat_sum":{"num_bytes":19741982,"num_objects":46,"num_object_clones":13,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":24,"num_read":7854,"num_read_kb":15661,"num_write":394,"num_write_kb":1320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":112,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-17T22:42:10.197419+0000","last_change":"2024-09-17T22:41:11.398108+0000","last_active":"2024-09-17T22:42:10.197419+0000","last_peered":"2024-09-17T22:42:10.197419+0000","last_clean":"2024-09-17T22:42:10.197419+0000","last_became_active":"2024-09-17T22:41:09.388184+0000","last_became_peered":"2024-09-17T22:41:09.388184+0000","last_unstale":"2024-09-17T22:42:10.197419+0000","last_undegraded":"2024-09-17T22:42:10.197419+0000","last_fullsized":"2024-09-17T22:42:10.197419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:43.506329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":256,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T22:42:07.180273+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:42:07.180273+0000","last_peered":"2024-09-17T22:42:07.180273+0000","last_clean":"2024-09-17T22:42:07.180273+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:42:07.180273+0000","last_undegraded":"2024-09-17T22:42:07.180273+0000","last_fullsized":"2024-09-17T22:42:07.180273+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"63'113","reported_seq":6818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T22:42:06.174639+0000","last_change":"2024-09-17T22:41:21.448156+0000","last_active":"2024-09-17T22:42:06.174639+0000","last_peered":"2024-09-17T22:42:06.174639+0000","last_clean":"2024-09-17T22:42:06.174639+0000","last_became_active":"2024-09-17T22:41:19.448956+0000","last_became_peered":"2024-09-17T22:41:19.448956+0000","last_unstale":"2024-09-17T22:42:06.174639+0000","last_undegraded":"2024-09-17T22:42:06.174639+0000","last_fullsized":"2024-09-17T22:42:06.174639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:02:42.910967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055851200000000005,"stat_sum":{"num_bytes":18366933,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":21,"num_read":6729,"num_read_kb":18612,"num_write":109,"num_write_kb":3316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"66'465","reported_seq":13712,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-17T22:42:10.197425+0000","last_change":"2024-09-17T22:41:21.445421+0000","last_active":"2024-09-17T22:42:10.197425+0000","last_peered":"2024-09-17T22:42:10.197425+0000","last_clean":"2024-09-17T22:42:10.197425+0000","last_became_active":"2024-09-17T22:41:19.449003+0000","last_became_peered":"2024-09-17T22:41:19.449003+0000","last_unstale":"2024-09-17T22:42:10.197425+0000","last_undegraded":"2024-09-17T22:42:10.197425+0000","last_fullsized":"2024-09-17T22:42:10.197425+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:02:40.113983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051465400000000002,"stat_sum":{"num_bytes":13847910,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":21,"num_read":15162,"num_read_kb":36412,"num_write":691,"num_write_kb":6043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"66'252","reported_seq":7362,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-17T22:42:10.197710+0000","last_change":"2024-09-17T22:41:21.445728+0000","last_active":"2024-09-17T22:42:10.197710+0000","last_peered":"2024-09-17T22:42:10.197710+0000","last_clean":"2024-09-17T22:42:10.197710+0000","last_became_active":"2024-09-17T22:41:19.448011+0000","last_became_peered":"2024-09-17T22:41:19.448011+0000","last_unstale":"2024-09-17T22:42:10.197710+0000","last_undegraded":"2024-09-17T22:42:10.197710+0000","last_fullsized":"2024-09-17T22:42:10.197710+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:46:35.899795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068494200000000002,"stat_sum":{"num_bytes":41986812,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":26,"num_read":7483,"num_read_kb":29685,"num_write":285,"num_write_kb":5614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"61'95","reported_seq":760,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T22:42:06.230803+0000","last_change":"2024-09-17T22:41:21.462738+0000","last_active":"2024-09-17T22:42:06.230803+0000","last_peered":"2024-09-17T22:42:06.230803+0000","last_clean":"2024-09-17T22:42:06.230803+0000","last_became_active":"2024-09-17T22:41:19.447215+0000","last_became_peered":"2024-09-17T22:41:19.447215+0000","last_unstale":"2024-09-17T22:42:06.230803+0000","last_undegraded":"2024-09-17T22:42:06.230803+0000","last_fullsized":"2024-09-17T22:42:06.230803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:13:36.780887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042703000000000001,"stat_sum":{"num_bytes":23904450,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":19,"num_read":594,"num_read_kb":6420,"num_write":85,"num_write_kb":2497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"63'155","reported_seq":20710,"reported_epoch":63,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:06.332350+0000","last_change":"2024-09-17T22:41:54.119555+0000","last_active":"2024-09-17T22:42:06.332350+0000","last_peered":"2024-09-17T22:42:06.332350+0000","last_clean":"2024-09-17T22:41:52.249181+0000","last_became_active":"2024-09-17T22:41:54.119555+0000","last_became_peered":"2024-09-17T22:41:54.119555+0000","last_unstale":"2024-09-17T22:42:06.332350+0000","last_undegraded":"2024-09-17T22:41:54.118897+0000","last_fullsized":"2024-09-17T22:41:54.118785+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:23:37.447949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000395712,"stat_sum":{"num_bytes":76272766,"num_objects":42,"num_object_clones":9,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":23,"num_read":21049,"num_read_kb":58099,"num_write":137,"num_write_kb":11552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":42}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"66'122","reported_seq":1054,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-17T22:42:10.197704+0000","last_change":"2024-09-17T22:41:21.446562+0000","last_active":"2024-09-17T22:42:10.197704+0000","last_peered":"2024-09-17T22:42:10.197704+0000","last_clean":"2024-09-17T22:42:10.197704+0000","last_became_active":"2024-09-17T22:41:19.447631+0000","last_became_peered":"2024-09-17T22:41:19.447631+0000","last_unstale":"2024-09-17T22:42:10.197704+0000","last_undegraded":"2024-09-17T22:42:10.197704+0000","last_fullsized":"2024-09-17T22:42:10.197704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:51:28.811215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000402657,"stat_sum":{"num_bytes":20685899,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":21,"num_read":855,"num_read_kb":2866,"num_write":103,"num_write_kb":3271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"66'168","reported_seq":7820,"reported_epoch":67,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:10.202970+0000","last_change":"2024-09-17T22:41:54.119016+0000","last_active":"2024-09-17T22:42:10.202970+0000","last_peered":"2024-09-17T22:42:10.202970+0000","last_clean":"2024-09-17T22:41:52.398679+0000","last_became_active":"2024-09-17T22:41:54.119016+0000","last_became_peered":"2024-09-17T22:41:54.119016+0000","last_unstale":"2024-09-17T22:42:10.202970+0000","last_undegraded":"2024-09-17T22:41:54.116346+0000","last_fullsized":"2024-09-17T22:41:54.116297+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:07:05.547322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038538199999999998,"stat_sum":{"num_bytes":22056164,"num_objects":51,"num_object_clones":12,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":51,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":28,"num_read":8179,"num_read_kb":19858,"num_write":125,"num_write_kb":8631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":51}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"63'129","reported_seq":1042,"reported_epoch":64,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:07.180616+0000","last_change":"2024-09-17T22:41:54.113116+0000","last_active":"2024-09-17T22:42:07.180616+0000","last_peered":"2024-09-17T22:42:07.180616+0000","last_clean":"2024-09-17T22:41:52.346319+0000","last_became_active":"2024-09-17T22:41:54.113116+0000","last_became_peered":"2024-09-17T22:41:54.113116+0000","last_unstale":"2024-09-17T22:42:07.180616+0000","last_undegraded":"2024-09-17T22:41:54.112692+0000","last_fullsized":"2024-09-17T22:41:54.112614+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:38:39.735951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000602379,"stat_sum":{"num_bytes":19240832,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":29,"num_read":867,"num_read_kb":2844,"num_write":110,"num_write_kb":1748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":48}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"63'206","reported_seq":1404,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T22:42:07.180159+0000","last_change":"2024-09-17T22:41:21.446576+0000","last_active":"2024-09-17T22:42:07.180159+0000","last_peered":"2024-09-17T22:42:07.180159+0000","last_clean":"2024-09-17T22:42:07.180159+0000","last_became_active":"2024-09-17T22:41:19.447419+0000","last_became_peered":"2024-09-17T22:41:19.447419+0000","last_unstale":"2024-09-17T22:42:07.180159+0000","last_undegraded":"2024-09-17T22:42:07.180159+0000","last_fullsized":"2024-09-17T22:42:07.180159+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:17:15.266264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001865071,"stat_sum":{"num_bytes":9530337,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":24,"num_read":2562,"num_read_kb":7596,"num_write":178,"num_write_kb":2567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"67'346","reported_seq":2431,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-17T22:42:10.327710+0000","last_change":"2024-09-17T22:41:21.445226+0000","last_active":"2024-09-17T22:42:10.327710+0000","last_peered":"2024-09-17T22:42:10.327710+0000","last_clean":"2024-09-17T22:42:10.327710+0000","last_became_active":"2024-09-17T22:41:19.454151+0000","last_became_peered":"2024-09-17T22:41:19.454151+0000","last_unstale":"2024-09-17T22:42:10.327710+0000","last_undegraded":"2024-09-17T22:42:10.327710+0000","last_fullsized":"2024-09-17T22:42:10.327710+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:30.375649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050545600000000005,"stat_sum":{"num_bytes":18114938,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":29,"num_read":3694,"num_read_kb":9055,"num_write":422,"num_write_kb":1820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"62'180","reported_seq":6251,"reported_epoch":63,"state":"active+clean+remapped","last_fresh":"2024-09-17T22:42:06.338992+0000","last_change":"2024-09-17T22:41:57.132756+0000","last_active":"2024-09-17T22:42:06.338992+0000","last_peered":"2024-09-17T22:42:06.338992+0000","last_clean":"2024-09-17T22:42:06.338992+0000","last_became_active":"2024-09-17T22:41:57.131666+0000","last_became_peered":"2024-09-17T22:41:57.131666+0000","last_unstale":"2024-09-17T22:42:06.338992+0000","last_undegraded":"2024-09-17T22:42:06.338992+0000","last_fullsized":"2024-09-17T22:42:06.338992+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:41:22.503910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048487299999999998,"stat_sum":{"num_bytes":37634303,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":35,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":18,"num_read":7019,"num_read_kb":15803,"num_write":168,"num_write_kb":2167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6,3],"avail_no_missing":["6","3"],"object_location_counts":[{"shards":"3,6","objects":35}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"65'280","reported_seq":1784,"reported_epoch":67,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:10.197894+0000","last_change":"2024-09-17T22:41:54.118972+0000","last_active":"2024-09-17T22:42:10.197894+0000","last_peered":"2024-09-17T22:42:10.197894+0000","last_clean":"2024-09-17T22:41:52.267782+0000","last_became_active":"2024-09-17T22:41:54.118972+0000","last_became_peered":"2024-09-17T22:41:54.118972+0000","last_unstale":"2024-09-17T22:42:10.197894+0000","last_undegraded":"2024-09-17T22:41:54.114427+0000","last_fullsized":"2024-09-17T22:41:54.114366+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:23:49.284051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062146200000000003,"stat_sum":{"num_bytes":12562892,"num_objects":43,"num_object_clones":6,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":29,"num_read":2262,"num_read_kb":5228,"num_write":377,"num_write_kb":1706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":43}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":954195623,"num_objects":1351,"num_object_clones":313,"num_object_copies":2702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":355,"num_objects_misplaced":121,"num_objects_unfound":0,"num_objects_dirty":1351,"num_whiteouts":720,"num_read":206943,"num_read_kb":538446,"num_write":7474,"num_write_kb":142815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3161143,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24551424,"data_stored":191755879,"data_compressed":6518745,"data_compressed_allocated":17702912,"data_compressed_original":187975529,"omap_allocated":0,"internal_metadata":0},"log_size":6615,"ondisk_log_size":6615,"up":53,"acting":56,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":14,"acting":14,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38444,"kb_used_data":3096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333396,"statfs":{"total":96636764160,"available":96597397504,"internally_reserved":0,"allocated":3170304,"data_stored":18611959,"data_compressed":547918,"data_compressed_allocated":1728512,"data_compressed_original":17917737,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542156,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34156,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337684,"statfs":{"total":96636764160,"available":96601788416,"internally_reserved":0,"allocated":3043328,"data_stored":23926169,"data_compressed":437861,"data_compressed_allocated":1896448,"data_compressed_original":23315877,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542159,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34728,"kb_used_data":3540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337112,"statfs":{"total":96636764160,"available":96601202688,"internally_reserved":0,"allocated":3624960,"data_stored":28074344,"data_compressed":805159,"data_compressed_allocated":2326528,"data_compressed_original":27437866,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34656,"kb_used_data":3464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337184,"statfs":{"total":96636764160,"available":96601276416,"internally_reserved":0,"allocated":3547136,"data_stored":20562556,"data_compressed":873744,"data_compressed_allocated":1990656,"data_compressed_original":19777196,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542159,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39244,"kb_used_data":3896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332596,"statfs":{"total":96636764160,"available":96596578304,"internally_reserved":0,"allocated":3989504,"data_stored":25961422,"data_compressed":973165,"data_compressed_allocated":2461696,"data_compressed_original":25228292,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38132,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333708,"statfs":{"total":96636764160,"available":96597716992,"internally_reserved":0,"allocated":2797568,"data_stored":15947509,"data_compressed":645142,"data_compressed_allocated":1335296,"data_compressed_original":15232865,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44292,"kb_used_data":4784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327548,"statfs":{"total":96636764160,"available":96591409152,"internally_reserved":0,"allocated":4898816,"data_stored":27483569,"data_compressed":1048513,"data_compressed_allocated":2830336,"data_compressed_original":26390904,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39652,"kb_used_data":4308,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332188,"statfs":{"total":96636764160,"available":96596160512,"internally_reserved":0,"allocated":4411392,"data_stored":26021000,"data_compressed":951515,"data_compressed_allocated":2424832,"data_compressed_original":24974312,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2514944,"data_stored":23924031,"data_compressed":438892,"data_compressed_allocated":1921024,"data_compressed_original":23549349,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2441216,"data_stored":17780745,"data_compressed":565983,"data_compressed_allocated":1720320,"data_compressed_original":17413929,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3731456,"data_stored":25779872,"data_compressed":1010406,"data_compressed_allocated":2490368,"data_compressed_original":25068520,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4370432,"data_stored":29306693,"data_compressed":1139866,"data_compressed_allocated":3047424,"data_compressed_original":28549496,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2236416,"data_stored":17724082,"data_compressed":663818,"data_compressed_allocated":1482752,"data_compressed_original":17330017,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3284992,"data_stored":26099074,"data_compressed":968767,"data_compressed_allocated":2490368,"data_compressed_original":25703428,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2985984,"data_stored":22339057,"data_compressed":913958,"data_compressed_allocated":2150400,"data_compressed_original":21874348,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2985984,"data_stored":28802325,"data_compressed":817055,"data_compressed_allocated":2400256,"data_compressed_original":28486442,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:42:11.628 INFO:tasks.thrashosds.thrasher:Setting 2.6 to [2, 3, 1, 7] 2024-09-17T22:42:11.629 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.6', '2', '3', '1', '7'] 2024-09-17T22:42:11.629 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.6 2 3 1 7 2024-09-17T22:42:11.633 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:42:11.647 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:11.645+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:11.738 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:11.735+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:11.751 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:11.751 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:11.763 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:42:11.776 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:11.797 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:11.797 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:11.814 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:11.815 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:11.832 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:11.832 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:11.839 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:11.836+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:11.849 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:11.850 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:11.867 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:11.867 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:11.884 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:11.884 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:11.901 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:11.901 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:11.904 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:11.904 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:11.910 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:11.910 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:11.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:11.911 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:11.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.911 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188678 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 4530176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.911 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.826734+0000", 2024-09-17T22:42:11.911 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.080779154, 2024-09-17T22:42:11.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02614153, 2024-09-17T22:42:11.911 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.911 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.911 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.911 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.911 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.912 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.912 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188678 2024-09-17T22:42:11.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.912 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826734+0000", 2024-09-17T22:42:11.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826734+0000", 2024-09-17T22:42:11.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826735+0000", 2024-09-17T22:42:11.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:42:11.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826737+0000", 2024-09-17T22:42:11.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1069999999999999e-06 2024-09-17T22:42:11.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826743+0000", 2024-09-17T22:42:11.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4910000000000002e-06 2024-09-17T22:42:11.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826748+0000", 2024-09-17T22:42:11.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.741e-06 2024-09-17T22:42:11.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.852667+0000", 2024-09-17T22:42:11.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025918632 2024-09-17T22:42:11.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.852738+0000", 2024-09-17T22:42:11.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1155000000000006e-05 2024-09-17T22:42:11.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.852876+0000", 2024-09-17T22:42:11.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013796699999999999 2024-09-17T22:42:11.915 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.915 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.915 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.915 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188679 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 4620288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.915 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.826754+0000", 2024-09-17T22:42:11.915 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.080759161, 2024-09-17T22:42:11.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026368841000000001, 2024-09-17T22:42:11.916 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.916 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.916 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.916 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.916 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.916 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.916 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188679 2024-09-17T22:42:11.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.916 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826754+0000", 2024-09-17T22:42:11.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826754+0000", 2024-09-17T22:42:11.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826755+0000", 2024-09-17T22:42:11.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9999999999999998e-07 2024-09-17T22:42:11.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826757+0000", 2024-09-17T22:42:11.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1600000000000001e-06 2024-09-17T22:42:11.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826763+0000", 2024-09-17T22:42:11.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1360000000000002e-06 2024-09-17T22:42:11.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826768+0000", 2024-09-17T22:42:11.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6439999999999999e-06 2024-09-17T22:42:11.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.852910+0000", 2024-09-17T22:42:11.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026142873 2024-09-17T22:42:11.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.852985+0000", 2024-09-17T22:42:11.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4698999999999997e-05 2024-09-17T22:42:11.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.853123+0000", 2024-09-17T22:42:11.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00013782900000000001 2024-09-17T22:42:11.920 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.920 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.920 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.920 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188680 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 4710400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.920 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.826797+0000", 2024-09-17T22:42:11.920 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.0807166019999999, 2024-09-17T22:42:11.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026517866000000001, 2024-09-17T22:42:11.920 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.920 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.920 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.920 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.920 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.920 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.921 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188680 2024-09-17T22:42:11.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.921 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826797+0000", 2024-09-17T22:42:11.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826797+0000", 2024-09-17T22:42:11.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826797+0000", 2024-09-17T22:42:11.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5700000000000002e-07 2024-09-17T22:42:11.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826800+0000", 2024-09-17T22:42:11.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7669999999999999e-06 2024-09-17T22:42:11.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826808+0000", 2024-09-17T22:42:11.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4599999999999997e-06 2024-09-17T22:42:11.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826813+0000", 2024-09-17T22:42:11.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0730000000000004e-06 2024-09-17T22:42:11.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.853167+0000", 2024-09-17T22:42:11.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026354122000000001 2024-09-17T22:42:11.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.853226+0000", 2024-09-17T22:42:11.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9258000000000001e-05 2024-09-17T22:42:11.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.853315+0000", 2024-09-17T22:42:11.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.8628999999999995e-05 2024-09-17T22:42:11.924 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.924 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.924 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.924 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188681 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 4800512~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.924 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.826820+0000", 2024-09-17T22:42:11.924 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.0806934400000001, 2024-09-17T22:42:11.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026664772, 2024-09-17T22:42:11.925 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.925 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.925 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.925 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188681 2024-09-17T22:42:11.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.925 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826820+0000", 2024-09-17T22:42:11.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826820+0000", 2024-09-17T22:42:11.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826820+0000", 2024-09-17T22:42:11.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7800000000000002e-07 2024-09-17T22:42:11.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826823+0000", 2024-09-17T22:42:11.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:42:11.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826829+0000", 2024-09-17T22:42:11.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5239999999999997e-06 2024-09-17T22:42:11.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826834+0000", 2024-09-17T22:42:11.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9180000000000002e-06 2024-09-17T22:42:11.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.853349+0000", 2024-09-17T22:42:11.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026515113 2024-09-17T22:42:11.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.853400+0000", 2024-09-17T22:42:11.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0726999999999997e-05 2024-09-17T22:42:11.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.853485+0000", 2024-09-17T22:42:11.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.488e-05 2024-09-17T22:42:11.928 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.928 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.928 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.929 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188682 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 4890624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.929 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.826845+0000", 2024-09-17T22:42:11.929 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.0806685379999998, 2024-09-17T22:42:11.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026778743000000001, 2024-09-17T22:42:11.929 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.929 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.929 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.929 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.929 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.929 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.929 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188682 2024-09-17T22:42:11.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.930 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826845+0000", 2024-09-17T22:42:11.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826845+0000", 2024-09-17T22:42:11.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826845+0000", 2024-09-17T22:42:11.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1600000000000002e-07 2024-09-17T22:42:11.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826847+0000", 2024-09-17T22:42:11.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0540000000000002e-06 2024-09-17T22:42:11.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826853+0000", 2024-09-17T22:42:11.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9150000000000001e-06 2024-09-17T22:42:11.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826859+0000", 2024-09-17T22:42:11.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.037e-06 2024-09-17T22:42:11.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.853501+0000", 2024-09-17T22:42:11.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026642088000000001 2024-09-17T22:42:11.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.853545+0000", 2024-09-17T22:42:11.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4035000000000001e-05 2024-09-17T22:42:11.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.853624+0000", 2024-09-17T22:42:11.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8197999999999999e-05 2024-09-17T22:42:11.933 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.933 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.933 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.933 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188683 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 4980736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.933 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.826866+0000", 2024-09-17T22:42:11.933 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.0806474320000001, 2024-09-17T22:42:11.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026912587000000002, 2024-09-17T22:42:11.933 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.933 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.933 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.933 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.933 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.934 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.934 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188683 2024-09-17T22:42:11.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.934 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826866+0000", 2024-09-17T22:42:11.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826866+0000", 2024-09-17T22:42:11.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826866+0000", 2024-09-17T22:42:11.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4499999999999997e-07 2024-09-17T22:42:11.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826868+0000", 2024-09-17T22:42:11.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1009999999999999e-06 2024-09-17T22:42:11.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826875+0000", 2024-09-17T22:42:11.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4189999999999999e-06 2024-09-17T22:42:11.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826880+0000", 2024-09-17T22:42:11.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8289999999999998e-06 2024-09-17T22:42:11.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.853649+0000", 2024-09-17T22:42:11.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026769438 2024-09-17T22:42:11.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.853697+0000", 2024-09-17T22:42:11.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8204000000000001e-05 2024-09-17T22:42:11.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.853778+0000", 2024-09-17T22:42:11.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1150999999999996e-05 2024-09-17T22:42:11.937 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.937 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.937 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.937 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188684 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5070848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.937 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.826889+0000", 2024-09-17T22:42:11.937 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.0806240420000002, 2024-09-17T22:42:11.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027045797, 2024-09-17T22:42:11.937 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.937 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.937 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.938 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.938 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.938 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.938 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188684 2024-09-17T22:42:11.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.938 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826889+0000", 2024-09-17T22:42:11.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826889+0000", 2024-09-17T22:42:11.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826890+0000", 2024-09-17T22:42:11.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:42:11.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826892+0000", 2024-09-17T22:42:11.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.295e-06 2024-09-17T22:42:11.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826898+0000", 2024-09-17T22:42:11.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3419999999999996e-06 2024-09-17T22:42:11.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826903+0000", 2024-09-17T22:42:11.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0860000000000001e-06 2024-09-17T22:42:11.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.853806+0000", 2024-09-17T22:42:11.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.026902071999999999 2024-09-17T22:42:11.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.853859+0000", 2024-09-17T22:42:11.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3430000000000002e-05 2024-09-17T22:42:11.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.853935+0000", 2024-09-17T22:42:11.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6124000000000005e-05 2024-09-17T22:42:11.941 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.941 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.941 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.941 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188685 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5160960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.942 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.826910+0000", 2024-09-17T22:42:11.942 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.0806032480000001, 2024-09-17T22:42:11.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027180129000000001, 2024-09-17T22:42:11.942 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.942 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.942 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.942 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188685 2024-09-17T22:42:11.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.942 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826910+0000", 2024-09-17T22:42:11.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826910+0000", 2024-09-17T22:42:11.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826910+0000", 2024-09-17T22:42:11.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.96e-07 2024-09-17T22:42:11.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826913+0000", 2024-09-17T22:42:11.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0420000000000001e-06 2024-09-17T22:42:11.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826918+0000", 2024-09-17T22:42:11.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5559999999999998e-06 2024-09-17T22:42:11.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826923+0000", 2024-09-17T22:42:11.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6310000000000002e-06 2024-09-17T22:42:11.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.853963+0000", 2024-09-17T22:42:11.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027040044999999999 2024-09-17T22:42:11.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854010+0000", 2024-09-17T22:42:11.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7409e-05 2024-09-17T22:42:11.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854090+0000", 2024-09-17T22:42:11.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0049999999999994e-05 2024-09-17T22:42:11.946 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.946 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.946 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.946 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188686 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5251072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.946 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.826929+0000", 2024-09-17T22:42:11.946 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.0805843739999998, 2024-09-17T22:42:11.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02729736, 2024-09-17T22:42:11.946 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.946 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.946 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.946 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.946 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.946 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.947 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188686 2024-09-17T22:42:11.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.947 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826929+0000", 2024-09-17T22:42:11.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826929+0000", 2024-09-17T22:42:11.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826929+0000", 2024-09-17T22:42:11.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:42:11.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826932+0000", 2024-09-17T22:42:11.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1050000000000002e-06 2024-09-17T22:42:11.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826940+0000", 2024-09-17T22:42:11.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0339999999999997e-06 2024-09-17T22:42:11.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826945+0000", 2024-09-17T22:42:11.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3190000000000002e-06 2024-09-17T22:42:11.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854105+0000", 2024-09-17T22:42:11.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027160217 2024-09-17T22:42:11.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854146+0000", 2024-09-17T22:42:11.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1264999999999999e-05 2024-09-17T22:42:11.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854226+0000", 2024-09-17T22:42:11.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9975999999999995e-05 2024-09-17T22:42:11.950 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.950 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.950 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.950 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188687 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5341184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.950 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.826952+0000", 2024-09-17T22:42:11.950 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.0805618109999999, 2024-09-17T22:42:11.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027428010999999999, 2024-09-17T22:42:11.950 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.950 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.951 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.951 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.951 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.951 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.951 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188687 2024-09-17T22:42:11.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.951 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826952+0000", 2024-09-17T22:42:11.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826952+0000", 2024-09-17T22:42:11.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826952+0000", 2024-09-17T22:42:11.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1899999999999998e-07 2024-09-17T22:42:11.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826954+0000", 2024-09-17T22:42:11.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9e-06 2024-09-17T22:42:11.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826961+0000", 2024-09-17T22:42:11.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2729999999999997e-06 2024-09-17T22:42:11.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826966+0000", 2024-09-17T22:42:11.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2549999999999997e-06 2024-09-17T22:42:11.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854255+0000", 2024-09-17T22:42:11.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027288302 2024-09-17T22:42:11.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854301+0000", 2024-09-17T22:42:11.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6152000000000001e-05 2024-09-17T22:42:11.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854380+0000", 2024-09-17T22:42:11.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8809999999999999e-05 2024-09-17T22:42:11.954 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.954 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.954 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.954 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188688 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5431296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.954 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.826974+0000", 2024-09-17T22:42:11.955 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.0805393319999999, 2024-09-17T22:42:11.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027557710999999999, 2024-09-17T22:42:11.955 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.955 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.955 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.955 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188688 2024-09-17T22:42:11.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.955 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826974+0000", 2024-09-17T22:42:11.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826974+0000", 2024-09-17T22:42:11.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826974+0000", 2024-09-17T22:42:11.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4900000000000001e-07 2024-09-17T22:42:11.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826977+0000", 2024-09-17T22:42:11.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1629999999999999e-06 2024-09-17T22:42:11.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826984+0000", 2024-09-17T22:42:11.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5959999999999998e-06 2024-09-17T22:42:11.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826990+0000", 2024-09-17T22:42:11.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7629999999999997e-06 2024-09-17T22:42:11.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854411+0000", 2024-09-17T22:42:11.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027420542999999999 2024-09-17T22:42:11.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854457+0000", 2024-09-17T22:42:11.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6344999999999998e-05 2024-09-17T22:42:11.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854532+0000", 2024-09-17T22:42:11.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4851999999999994e-05 2024-09-17T22:42:11.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.959 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.959 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.959 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188689 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5521408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.959 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.826996+0000", 2024-09-17T22:42:11.959 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.0805176159999998, 2024-09-17T22:42:11.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027689002000000001, 2024-09-17T22:42:11.959 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.959 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.959 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.959 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188689 2024-09-17T22:42:11.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.960 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826996+0000", 2024-09-17T22:42:11.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826996+0000", 2024-09-17T22:42:11.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826996+0000", 2024-09-17T22:42:11.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1300000000000001e-07 2024-09-17T22:42:11.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.826998+0000", 2024-09-17T22:42:11.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.877e-06 2024-09-17T22:42:11.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827005+0000", 2024-09-17T22:42:11.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9040000000000003e-06 2024-09-17T22:42:11.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827011+0000", 2024-09-17T22:42:11.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.392e-06 2024-09-17T22:42:11.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854558+0000", 2024-09-17T22:42:11.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027546217000000001 2024-09-17T22:42:11.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854606+0000", 2024-09-17T22:42:11.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8068999999999997e-05 2024-09-17T22:42:11.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854685+0000", 2024-09-17T22:42:11.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9129999999999996e-05 2024-09-17T22:42:11.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.963 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.963 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188690 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.963 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827019+0000", 2024-09-17T22:42:11.963 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.0804948190000001, 2024-09-17T22:42:11.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027809461000000001, 2024-09-17T22:42:11.963 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.963 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.963 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.964 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188690 2024-09-17T22:42:11.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.964 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827019+0000", 2024-09-17T22:42:11.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827019+0000", 2024-09-17T22:42:11.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827019+0000", 2024-09-17T22:42:11.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0499999999999999e-07 2024-09-17T22:42:11.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827021+0000", 2024-09-17T22:42:11.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0880000000000002e-06 2024-09-17T22:42:11.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827029+0000", 2024-09-17T22:42:11.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7409999999999992e-06 2024-09-17T22:42:11.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827034+0000", 2024-09-17T22:42:11.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7790000000000003e-06 2024-09-17T22:42:11.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854712+0000", 2024-09-17T22:42:11.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027678154999999999 2024-09-17T22:42:11.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854752+0000", 2024-09-17T22:42:11.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0772999999999997e-05 2024-09-17T22:42:11.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854828+0000", 2024-09-17T22:42:11.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5519999999999995e-05 2024-09-17T22:42:11.967 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.967 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.967 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.968 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188691 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5701632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.968 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827040+0000", 2024-09-17T22:42:11.968 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.0804732370000001, 2024-09-17T22:42:11.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027946036, 2024-09-17T22:42:11.968 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.968 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.968 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.968 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188691 2024-09-17T22:42:11.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.969 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827040+0000", 2024-09-17T22:42:11.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827040+0000", 2024-09-17T22:42:11.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827040+0000", 2024-09-17T22:42:11.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:42:11.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827042+0000", 2024-09-17T22:42:11.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9099999999999999e-06 2024-09-17T22:42:11.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827049+0000", 2024-09-17T22:42:11.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0319999999999998e-06 2024-09-17T22:42:11.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827054+0000", 2024-09-17T22:42:11.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9520000000000002e-06 2024-09-17T22:42:11.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854858+0000", 2024-09-17T22:42:11.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027803148999999999 2024-09-17T22:42:11.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854906+0000", 2024-09-17T22:42:11.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8958000000000001e-05 2024-09-17T22:42:11.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.854986+0000", 2024-09-17T22:42:11.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9654999999999995e-05 2024-09-17T22:42:11.972 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.972 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.972 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.972 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188692 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5791744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.972 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827061+0000", 2024-09-17T22:42:11.972 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.0804526640000001, 2024-09-17T22:42:11.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028071307, 2024-09-17T22:42:11.972 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.972 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.972 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.972 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.972 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.973 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188692 2024-09-17T22:42:11.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.973 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827061+0000", 2024-09-17T22:42:11.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827061+0000", 2024-09-17T22:42:11.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827061+0000", 2024-09-17T22:42:11.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0699999999999998e-07 2024-09-17T22:42:11.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827063+0000", 2024-09-17T22:42:11.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0530000000000001e-06 2024-09-17T22:42:11.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827070+0000", 2024-09-17T22:42:11.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3409999999999997e-06 2024-09-17T22:42:11.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827075+0000", 2024-09-17T22:42:11.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6890000000000003e-06 2024-09-17T22:42:11.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855014+0000", 2024-09-17T22:42:11.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027938705000000001 2024-09-17T22:42:11.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855054+0000", 2024-09-17T22:42:11.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9928999999999997e-05 2024-09-17T22:42:11.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855132+0000", 2024-09-17T22:42:11.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8182999999999993e-05 2024-09-17T22:42:11.976 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.976 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.976 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.976 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188693 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5881856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.976 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:11.976 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.0804264209999999, 2024-09-17T22:42:11.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028193223, 2024-09-17T22:42:11.976 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.977 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.977 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.977 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188693 2024-09-17T22:42:11.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.977 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:11.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:11.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:11.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:42:11.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827090+0000", 2024-09-17T22:42:11.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2170000000000001e-06 2024-09-17T22:42:11.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827098+0000", 2024-09-17T22:42:11.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1510000000000004e-06 2024-09-17T22:42:11.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827103+0000", 2024-09-17T22:42:11.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0839999999999999e-06 2024-09-17T22:42:11.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855158+0000", 2024-09-17T22:42:11.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028055217 2024-09-17T22:42:11.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855201+0000", 2024-09-17T22:42:11.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3272000000000002e-05 2024-09-17T22:42:11.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855280+0000", 2024-09-17T22:42:11.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8845999999999997e-05 2024-09-17T22:42:11.980 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.980 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.981 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.981 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188694 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5971968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.981 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:11.981 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.0804025340000001, 2024-09-17T22:42:11.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028312278, 2024-09-17T22:42:11.981 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.981 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.981 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.981 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.981 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.981 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.981 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188694 2024-09-17T22:42:11.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.982 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:11.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:11.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:11.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:42:11.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827114+0000", 2024-09-17T22:42:11.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3029999999999998e-06 2024-09-17T22:42:11.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827121+0000", 2024-09-17T22:42:11.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3699999999999997e-06 2024-09-17T22:42:11.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827126+0000", 2024-09-17T22:42:11.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7350000000000004e-06 2024-09-17T22:42:11.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855307+0000", 2024-09-17T22:42:11.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028180851999999999 2024-09-17T22:42:11.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855346+0000", 2024-09-17T22:42:11.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9274999999999999e-05 2024-09-17T22:42:11.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855423+0000", 2024-09-17T22:42:11.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7294999999999998e-05 2024-09-17T22:42:11.985 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.985 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.985 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.985 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188695 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.985 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:11.985 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.080381756, 2024-09-17T22:42:11.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028435876999999998, 2024-09-17T22:42:11.985 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.985 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.986 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.986 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188695 2024-09-17T22:42:11.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.986 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:11.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:11.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:11.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:42:11.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827138+0000", 2024-09-17T22:42:11.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2580000000000001e-06 2024-09-17T22:42:11.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827145+0000", 2024-09-17T22:42:11.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0129999999999996e-06 2024-09-17T22:42:11.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827150+0000", 2024-09-17T22:42:11.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3749999999999996e-06 2024-09-17T22:42:11.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855447+0000", 2024-09-17T22:42:11.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028297224999999999 2024-09-17T22:42:11.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855487+0000", 2024-09-17T22:42:11.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0454000000000003e-05 2024-09-17T22:42:11.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855567+0000", 2024-09-17T22:42:11.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0149999999999997e-05 2024-09-17T22:42:11.989 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.989 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.989 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.989 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188696 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.990 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:11.990 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.0803579719999998, 2024-09-17T22:42:11.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028543570000000001, 2024-09-17T22:42:11.990 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.990 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.990 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.990 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.990 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.990 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.990 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188696 2024-09-17T22:42:11.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.990 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:11.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:11.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827156+0000", 2024-09-17T22:42:11.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T22:42:11.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827158+0000", 2024-09-17T22:42:11.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.875e-06 2024-09-17T22:42:11.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827164+0000", 2024-09-17T22:42:11.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7950000000000002e-06 2024-09-17T22:42:11.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827169+0000", 2024-09-17T22:42:11.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2880000000000003e-06 2024-09-17T22:42:11.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855582+0000", 2024-09-17T22:42:11.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028412929 2024-09-17T22:42:11.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855624+0000", 2024-09-17T22:42:11.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2484000000000002e-05 2024-09-17T22:42:11.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855699+0000", 2024-09-17T22:42:11.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4869000000000004e-05 2024-09-17T22:42:11.995 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.995 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.995 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.995 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188697 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6242304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:11.995 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:11.995 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 3.0803264430000001, 2024-09-17T22:42:11.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028658598, 2024-09-17T22:42:11.995 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:11.995 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:11.995 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:11.995 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:11.995 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:11.996 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:11.996 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188697 2024-09-17T22:42:11.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.996 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:11.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:11.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:11.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:11.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:11.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:11.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:11.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:11.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8800000000000003e-07 2024-09-17T22:42:11.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:11.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827190+0000", 2024-09-17T22:42:11.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.351e-06 2024-09-17T22:42:11.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:11.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827198+0000", 2024-09-17T22:42:11.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.909e-06 2024-09-17T22:42:11.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:11.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827203+0000", 2024-09-17T22:42:11.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4020000000000001e-06 2024-09-17T22:42:11.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:11.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855720+0000", 2024-09-17T22:42:11.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028516909999999999 2024-09-17T22:42:11.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:11.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855762+0000", 2024-09-17T22:42:11.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2184999999999997e-05 2024-09-17T22:42:11.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:11.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:11.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:11.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855845+0000", 2024-09-17T22:42:11.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3353e-05 2024-09-17T22:42:11.999 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.999 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.999 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.999 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:11.999 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:11.999 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:12.001 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:42:12.001 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:11.998+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:12.048 INFO:teuthology.orchestra.run.smithi115.stdout:2900 trunc from 0xe6c1806 to 0x4d5227b 2024-09-17T22:42:12.064 INFO:teuthology.orchestra.run.smithi115.stdout:2901 punch from 0x361905c to 0x3625aa1, (0xca45 bytes) 2024-09-17T22:42:12.064 INFO:teuthology.orchestra.run.smithi115.stdout:2902 write 0x211f24c thru 0x212d5ba (0xe36f bytes) 2024-09-17T22:42:12.069 INFO:teuthology.orchestra.run.smithi115.stdout:2903 punch from 0x3b81d4 to 0x3bb1e5, (0x3011 bytes) 2024-09-17T22:42:12.069 INFO:teuthology.orchestra.run.smithi115.stdout:2904 punch from 0x2652f62 to 0x2657acb, (0x4b69 bytes) 2024-09-17T22:42:12.070 INFO:teuthology.orchestra.run.smithi115.stdout:2906 trunc from 0x4d5227b to 0x407a089 2024-09-17T22:42:12.077 INFO:teuthology.orchestra.run.smithi115.stdout:2907 write 0x209ad81 thru 0x209c973 (0x1bf3 bytes) 2024-09-17T22:42:12.079 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:12.079 INFO:teuthology.orchestra.run.smithi115.stdout:2910 read 0x2288204 thru 0x22936f3 (0xb4f0 bytes) 2024-09-17T22:42:12.079 INFO:teuthology.orchestra.run.smithi115.stdout:2911 read 0x13e8e2f thru 0x13ef645 (0x6817 bytes) 2024-09-17T22:42:12.097 INFO:teuthology.orchestra.run.smithi115.stdout:2912 write 0xd007ff7 thru 0xd01136a (0x9374 bytes) 2024-09-17T22:42:12.098 INFO:teuthology.orchestra.run.smithi115.stdout:2913 write 0x1c64dfc thru 0x1c66c6e (0x1e73 bytes) 2024-09-17T22:42:12.101 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:12.099+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:12.117 INFO:teuthology.orchestra.run.smithi115.stdout:2914 punch from 0x8d85287 to 0x8d8dc94, (0x8a0d bytes) 2024-09-17T22:42:12.117 INFO:teuthology.orchestra.run.smithi115.stdout:2915 writesame 0x39ddcaa thru 0x39ec8ca data_size 0x1(0xec21 bytes) 2024-09-17T22:42:12.140 INFO:teuthology.orchestra.run.smithi115.stdout:2918 writesame 0x65bd19d thru 0x65be3d8 data_size 0x6(0x123c bytes) 2024-09-17T22:42:12.144 INFO:teuthology.orchestra.run.smithi115.stdout:2919 write 0x374b534 thru 0x37578b2 (0xc37f bytes) 2024-09-17T22:42:12.145 INFO:teuthology.orchestra.run.smithi115.stdout:2920 read 0x89e65f2 thru 0x89f52fe (0xed0d bytes) 2024-09-17T22:42:12.153 INFO:teuthology.orchestra.run.smithi115.stdout:2921 trunc from 0xd01136b to 0x696a4e0 2024-09-17T22:42:12.157 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:12.157 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:12.157 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:12.157 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:12.167 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:42:12.176 INFO:teuthology.orchestra.run.smithi115.stdout:2922 writesame 0xafc20e7 thru 0xafd02e5 data_size 0x69(0xe1ff bytes) 2024-09-17T22:42:12.184 INFO:teuthology.orchestra.run.smithi115.stdout:2923 write 0xd437abe thru 0xd438187 (0x6ca bytes) 2024-09-17T22:42:12.188 INFO:teuthology.orchestra.run.smithi115.stdout:2925 write 0xb5622eb thru 0xb5672f5 (0x500b bytes) 2024-09-17T22:42:12.201 INFO:teuthology.orchestra.run.smithi115.stdout:2930 write 0xedeeed6 thru 0xedf1433 (0x255e bytes) 2024-09-17T22:42:12.207 INFO:teuthology.orchestra.run.smithi115.stdout:2931 punch from 0xe5a0fc0 to 0xe5a5046, (0x4086 bytes) 2024-09-17T22:42:12.207 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:12.207 INFO:teuthology.orchestra.run.smithi115.stdout:2933 read 0xa6d2d54 thru 0xa6dd933 (0xabe0 bytes) 2024-09-17T22:42:12.207 INFO:teuthology.orchestra.run.smithi115.stdout:2934 writesame 0x8e36a9d thru 0x8e36abf data_size 0x5(0x23 bytes) 2024-09-17T22:42:12.209 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:12.206+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:12.209 INFO:teuthology.orchestra.run.smithi115.stderr:set 2.6 pg_upmap_items mapping to [2->3,1->7] 2024-09-17T22:42:12.211 INFO:teuthology.orchestra.run.smithi115.stdout:2935 write 0xacf40fd thru 0xacfb4f0 (0x73f4 bytes) 2024-09-17T22:42:12.217 INFO:teuthology.orchestra.run.smithi115.stdout:2936 read 0x31c9d65 thru 0x31cbd7c (0x2018 bytes) 2024-09-17T22:42:12.218 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:12.218 INFO:teuthology.orchestra.run.smithi115.stdout:2938 punch from 0xe8ff726 to 0xe90e47f, (0xed59 bytes) 2024-09-17T22:42:12.218 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:12.218 INFO:teuthology.orchestra.run.smithi115.stdout:2941 write 0x9617efc thru 0x96218b7 (0x99bc bytes) 2024-09-17T22:42:12.218 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:42:12.237 INFO:teuthology.orchestra.run.smithi115.stdout:2942 trunc from 0xedf1434 to 0x14ed63f 2024-09-17T22:42:12.285 INFO:teuthology.orchestra.run.smithi115.stdout:2943 write 0xcd363da thru 0xcd401ae (0x9dd5 bytes) 2024-09-17T22:42:12.287 INFO:teuthology.orchestra.run.smithi115.stdout:2944 punch from 0xa062453 to 0xa068ead, (0x6a5a bytes) 2024-09-17T22:42:12.288 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:12.288 INFO:teuthology.orchestra.run.smithi115.stdout:2946 writesame 0x2df313b thru 0x2dfffba data_size 0x80(0xce80 bytes) 2024-09-17T22:42:12.301 INFO:teuthology.orchestra.run.smithi115.stdout:2947 trunc from 0xcd401af to 0x4114242 2024-09-17T22:42:12.303 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:12.300+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:12.311 INFO:teuthology.orchestra.run.smithi115.stdout:2948 write 0x2365f8c thru 0x2368306 (0x237b bytes) 2024-09-17T22:42:12.321 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:12.321 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:12.321 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:12.322 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:12.322 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:12.323 INFO:teuthology.orchestra.run.smithi115.stdout:2949 write 0x6bcb4a7 thru 0x6bd8cd9 (0xd833 bytes) 2024-09-17T22:42:12.328 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:12.328 INFO:teuthology.orchestra.run.smithi115.stdout:2952 punch from 0x484ad97 to 0x48557c1, (0xaa2a bytes) 2024-09-17T22:42:12.328 INFO:teuthology.orchestra.run.smithi115.stdout:2953 writesame 0x509a9e8 thru 0x50a5531 data_size 0x2(0xab4a bytes) 2024-09-17T22:42:12.331 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:42:12.345 INFO:teuthology.orchestra.run.smithi115.stdout:2955 writesame 0x958976b thru 0x958c059 data_size 0x7(0x28ef bytes) 2024-09-17T22:42:12.345 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:12.346 INFO:teuthology.orchestra.run.smithi115.stdout:2958 write 0x139d9f2 thru 0x13a167f (0x3c8e bytes) 2024-09-17T22:42:12.347 INFO:teuthology.orchestra.run.smithi115.stdout:2959 punch from 0x7382f2 to 0x747bea, (0xf8f8 bytes) 2024-09-17T22:42:12.351 INFO:teuthology.orchestra.run.smithi115.stdout:2960 trunc from 0x958c05a to 0x606ada5 2024-09-17T22:42:12.357 INFO:teuthology.orchestra.run.smithi115.stdout:2961 write 0xd1adda thru 0xd1b002 (0x229 bytes) 2024-09-17T22:42:12.358 INFO:teuthology.orchestra.run.smithi115.stdout:2962 read 0x403604 thru 0x410b0d (0xd50a bytes) 2024-09-17T22:42:12.361 INFO:teuthology.orchestra.run.smithi115.stdout:2964 trunc from 0x606ada5 to 0x738bdd8 2024-09-17T22:42:12.366 INFO:teuthology.orchestra.run.smithi115.stdout:2965 punch from 0x545ec6e to 0x5468033, (0x93c5 bytes) 2024-09-17T22:42:12.376 INFO:teuthology.orchestra.run.smithi115.stdout:2966 writesame 0xb0d11e3 thru 0xb0d7c6c data_size 0xd(0x6a8a bytes) 2024-09-17T22:42:12.379 INFO:teuthology.orchestra.run.smithi115.stdout:2968 trunc from 0xb0d7c6d to 0xe0e2e96 2024-09-17T22:42:12.385 INFO:teuthology.orchestra.run.smithi115.stdout:2970 write 0xa4731d3 thru 0xa47dac1 (0xa8ef bytes) 2024-09-17T22:42:12.388 INFO:teuthology.orchestra.run.smithi115.stdout:2971 read 0xca8c7cc thru 0xca93e01 (0x7636 bytes) 2024-09-17T22:42:12.388 INFO:teuthology.orchestra.run.smithi115.stdout:2973 read 0xbde85ee thru 0xbdf638c (0xdd9f bytes) 2024-09-17T22:42:12.389 INFO:teuthology.orchestra.run.smithi115.stdout:2974 read 0x983356 thru 0x9883bf (0x506a bytes) 2024-09-17T22:42:12.389 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:12.389 INFO:teuthology.orchestra.run.smithi115.stdout:2976 write 0xd8f337d thru 0xd8fb660 (0x82e4 bytes) 2024-09-17T22:42:12.402 INFO:teuthology.orchestra.run.smithi115.stdout:2977 trunc from 0xe0e2e96 to 0x409e8de 2024-09-17T22:42:12.403 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:12.401+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:12.416 INFO:teuthology.orchestra.run.smithi115.stdout:2978 punch from 0x502b1f to 0x510f06, (0xe3e7 bytes) 2024-09-17T22:42:12.416 INFO:teuthology.orchestra.run.smithi115.stdout:2979 punch from 0x21a21e to 0x220884, (0x6666 bytes) 2024-09-17T22:42:12.421 INFO:teuthology.orchestra.run.smithi115.stdout:2980 trunc from 0x409e8de to 0x83079b9 2024-09-17T22:42:12.426 INFO:teuthology.orchestra.run.smithi115.stdout:2982 write 0x1266fcb thru 0x127409f (0xd0d5 bytes) 2024-09-17T22:42:12.436 INFO:teuthology.orchestra.run.smithi115.stdout:2986 trunc from 0x83079b9 to 0xeba1224 2024-09-17T22:42:12.442 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:12.442 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:12.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:12.442 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:12.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.442 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36437 3.b 3:d64eaf12:::rbd_data.115f86d8437d.000000000000001c:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:12.442 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:42:12.442 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.723605790999997, 2024-09-17T22:42:12.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0068296390000000002, 2024-09-17T22:42:12.442 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.442 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.442 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:12.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:12.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:12.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.443 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36437 2024-09-17T22:42:12.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.443 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:42:12.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715034+0000", 2024-09-17T22:42:12.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715035+0000", 2024-09-17T22:42:12.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.288e-06 2024-09-17T22:42:12.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715041+0000", 2024-09-17T22:42:12.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.643e-06 2024-09-17T22:42:12.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715051+0000", 2024-09-17T22:42:12.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0597e-05 2024-09-17T22:42:12.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715056+0000", 2024-09-17T22:42:12.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1379999999999998e-06 2024-09-17T22:42:12.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715101+0000", 2024-09-17T22:42:12.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4433e-05 2024-09-17T22:42:12.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.715187+0000", 2024-09-17T22:42:12.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6148000000000003e-05 2024-09-17T22:42:12.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:42:12.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.716264+0000", 2024-09-17T22:42:12.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010770179999999999 2024-09-17T22:42:12.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:12.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721735+0000", 2024-09-17T22:42:12.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0054709019999999997 2024-09-17T22:42:12.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:42:12.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721754+0000", 2024-09-17T22:42:12.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9133000000000001e-05 2024-09-17T22:42:12.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:12.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721807+0000", 2024-09-17T22:42:12.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2831999999999998e-05 2024-09-17T22:42:12.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.721863+0000", 2024-09-17T22:42:12.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6507e-05 2024-09-17T22:42:12.449 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.449 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.449 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.449 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36744 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 196608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:12.449 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:42:12.449 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.703601648999999, 2024-09-17T22:42:12.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063082559999999999, 2024-09-17T22:42:12.450 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.450 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.450 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:12.450 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:12.450 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:12.450 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.450 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36744 2024-09-17T22:42:12.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.450 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:42:12.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:42:12.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735038+0000", 2024-09-17T22:42:12.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:42:12.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735040+0000", 2024-09-17T22:42:12.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9630000000000001e-06 2024-09-17T22:42:12.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735047+0000", 2024-09-17T22:42:12.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3439999999999997e-06 2024-09-17T22:42:12.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735052+0000", 2024-09-17T22:42:12.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9250000000000003e-06 2024-09-17T22:42:12.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741147+0000", 2024-09-17T22:42:12.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0060954210000000002 2024-09-17T22:42:12.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741243+0000", 2024-09-17T22:42:12.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5674999999999995e-05 2024-09-17T22:42:12.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741346+0000", 2024-09-17T22:42:12.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103522 2024-09-17T22:42:12.453 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.454 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.454 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.454 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36745 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:12.454 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:42:12.454 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.703581665000002, 2024-09-17T22:42:12.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065186740000000003, 2024-09-17T22:42:12.454 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.454 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.454 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:12.454 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:12.454 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:12.455 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.455 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36745 2024-09-17T22:42:12.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.455 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:42:12.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:42:12.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735058+0000", 2024-09-17T22:42:12.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2500000000000001e-07 2024-09-17T22:42:12.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735061+0000", 2024-09-17T22:42:12.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.824e-06 2024-09-17T22:42:12.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735068+0000", 2024-09-17T22:42:12.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8580000000000002e-06 2024-09-17T22:42:12.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735074+0000", 2024-09-17T22:42:12.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5829999999999997e-06 2024-09-17T22:42:12.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741379+0000", 2024-09-17T22:42:12.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063051909999999999 2024-09-17T22:42:12.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741472+0000", 2024-09-17T22:42:12.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3238000000000004e-05 2024-09-17T22:42:12.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741577+0000", 2024-09-17T22:42:12.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000104555 2024-09-17T22:42:12.458 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.459 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.459 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.459 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36746 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:12.459 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:42:12.459 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.703558698999998, 2024-09-17T22:42:12.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067275240000000004, 2024-09-17T22:42:12.459 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.459 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.460 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:12.460 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:12.460 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:12.460 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.460 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36746 2024-09-17T22:42:12.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.460 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:42:12.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:42:12.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735081+0000", 2024-09-17T22:42:12.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.03e-07 2024-09-17T22:42:12.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735083+0000", 2024-09-17T22:42:12.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9470000000000002e-06 2024-09-17T22:42:12.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735090+0000", 2024-09-17T22:42:12.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7800000000000003e-06 2024-09-17T22:42:12.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735095+0000", 2024-09-17T22:42:12.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8500000000000002e-06 2024-09-17T22:42:12.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741608+0000", 2024-09-17T22:42:12.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065131809999999998 2024-09-17T22:42:12.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741703+0000", 2024-09-17T22:42:12.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4914999999999995e-05 2024-09-17T22:42:12.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741808+0000", 2024-09-17T22:42:12.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000105448 2024-09-17T22:42:12.463 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.463 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.463 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.464 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36747 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:12.464 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:42:12.464 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.703538146, 2024-09-17T22:42:12.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069468790000000004, 2024-09-17T22:42:12.464 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.464 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.464 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:12.464 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:12.464 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:12.464 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.464 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36747 2024-09-17T22:42:12.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.465 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:42:12.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735101+0000", 2024-09-17T22:42:12.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735102+0000", 2024-09-17T22:42:12.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:42:12.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735104+0000", 2024-09-17T22:42:12.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7400000000000001e-06 2024-09-17T22:42:12.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735111+0000", 2024-09-17T22:42:12.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4189999999999996e-06 2024-09-17T22:42:12.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735116+0000", 2024-09-17T22:42:12.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0860000000000001e-06 2024-09-17T22:42:12.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741847+0000", 2024-09-17T22:42:12.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067306859999999996 2024-09-17T22:42:12.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.741942+0000", 2024-09-17T22:42:12.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5180999999999996e-05 2024-09-17T22:42:12.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742048+0000", 2024-09-17T22:42:12.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106387 2024-09-17T22:42:12.468 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.468 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.468 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.468 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36748 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 233472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:12.468 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:42:12.468 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.703516825000001, 2024-09-17T22:42:12.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071864119999999997, 2024-09-17T22:42:12.469 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.469 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.469 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:12.469 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:12.469 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:12.469 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.469 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36748 2024-09-17T22:42:12.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.469 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:42:12.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:42:12.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735123+0000", 2024-09-17T22:42:12.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.34e-07 2024-09-17T22:42:12.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735125+0000", 2024-09-17T22:42:12.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0020000000000001e-06 2024-09-17T22:42:12.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735132+0000", 2024-09-17T22:42:12.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3489999999999997e-06 2024-09-17T22:42:12.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735138+0000", 2024-09-17T22:42:12.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9709999999999998e-06 2024-09-17T22:42:12.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742087+0000", 2024-09-17T22:42:12.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069480669999999996 2024-09-17T22:42:12.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742188+0000", 2024-09-17T22:42:12.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101279 2024-09-17T22:42:12.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742309+0000", 2024-09-17T22:42:12.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012131 2024-09-17T22:42:12.473 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.473 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.473 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.473 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36749 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:12.473 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:42:12.473 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.703494138000003, 2024-09-17T22:42:12.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0074112570000000001, 2024-09-17T22:42:12.474 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.474 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.474 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:12.474 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:12.474 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:12.474 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.474 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36749 2024-09-17T22:42:12.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.474 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:42:12.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735145+0000", 2024-09-17T22:42:12.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735146+0000", 2024-09-17T22:42:12.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9200000000000002e-07 2024-09-17T22:42:12.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735158+0000", 2024-09-17T22:42:12.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2508e-05 2024-09-17T22:42:12.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735166+0000", 2024-09-17T22:42:12.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6290000000000001e-06 2024-09-17T22:42:12.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735174+0000", 2024-09-17T22:42:12.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1000000000000004e-06 2024-09-17T22:42:12.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742354+0000", 2024-09-17T22:42:12.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071795660000000001 2024-09-17T22:42:12.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742454+0000", 2024-09-17T22:42:12.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100408 2024-09-17T22:42:12.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742557+0000", 2024-09-17T22:42:12.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102654 2024-09-17T22:42:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36750 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 247808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:42:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.703457012000001, 2024-09-17T22:42:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0077413890000000004, 2024-09-17T22:42:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.478 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36750 2024-09-17T22:42:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:42:12.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:42:12.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735183+0000", 2024-09-17T22:42:12.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2399999999999998e-07 2024-09-17T22:42:12.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735185+0000", 2024-09-17T22:42:12.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1710000000000001e-06 2024-09-17T22:42:12.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735192+0000", 2024-09-17T22:42:12.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.81e-06 2024-09-17T22:42:12.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735197+0000", 2024-09-17T22:42:12.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9080000000000003e-06 2024-09-17T22:42:12.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742587+0000", 2024-09-17T22:42:12.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.007390359 2024-09-17T22:42:12.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742694+0000", 2024-09-17T22:42:12.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106901 2024-09-17T22:42:12.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742924+0000", 2024-09-17T22:42:12.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00022961600000000001 2024-09-17T22:42:12.482 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.482 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.483 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.483 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:36751 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 253952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:12.483 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:42:12.483 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.703435038000002, 2024-09-17T22:42:12.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0079273859999999998, 2024-09-17T22:42:12.483 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.483 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.483 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:12.483 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:12.483 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:12.483 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.484 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 36751 2024-09-17T22:42:12.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.484 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:42:12.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:42:12.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735205+0000", 2024-09-17T22:42:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1600000000000001e-07 2024-09-17T22:42:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735207+0000", 2024-09-17T22:42:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1909999999999999e-06 2024-09-17T22:42:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735214+0000", 2024-09-17T22:42:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9269999999999999e-06 2024-09-17T22:42:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.735220+0000", 2024-09-17T22:42:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3739999999999997e-06 2024-09-17T22:42:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.742957+0000", 2024-09-17T22:42:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0077371699999999998 2024-09-17T22:42:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.743037+0000", 2024-09-17T22:42:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0370000000000005e-05 2024-09-17T22:42:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.743132+0000", 2024-09-17T22:42:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4838000000000002e-05 2024-09-17T22:42:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37659 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:12.487 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:42:12.488 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.688867395000003, 2024-09-17T22:42:12.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063144200000000003, 2024-09-17T22:42:12.488 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.488 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.488 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:12.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:12.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:12.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.488 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37659 2024-09-17T22:42:12.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.488 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:42:12.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749772+0000", 2024-09-17T22:42:12.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749773+0000", 2024-09-17T22:42:12.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0600000000000001e-07 2024-09-17T22:42:12.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749774+0000", 2024-09-17T22:42:12.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:42:12.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749781+0000", 2024-09-17T22:42:12.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5740000000000001e-06 2024-09-17T22:42:12.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749786+0000", 2024-09-17T22:42:12.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9099999999999996e-06 2024-09-17T22:42:12.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755889+0000", 2024-09-17T22:42:12.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0061034260000000003 2024-09-17T22:42:12.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.755972+0000", 2024-09-17T22:42:12.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.2795999999999998e-05 2024-09-17T22:42:12.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756087+0000", 2024-09-17T22:42:12.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000114418 2024-09-17T22:42:12.492 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.492 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.492 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.492 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37660 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:12.492 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:42:12.492 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.688837556999999, 2024-09-17T22:42:12.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065587800000000002, 2024-09-17T22:42:12.492 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.493 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.493 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:12.493 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:12.493 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:12.493 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.493 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37660 2024-09-17T22:42:12.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.493 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:42:12.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:42:12.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749802+0000", 2024-09-17T22:42:12.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:42:12.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749805+0000", 2024-09-17T22:42:12.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1399999999999998e-06 2024-09-17T22:42:12.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749812+0000", 2024-09-17T22:42:12.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4270000000000002e-06 2024-09-17T22:42:12.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749819+0000", 2024-09-17T22:42:12.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.759e-06 2024-09-17T22:42:12.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756153+0000", 2024-09-17T22:42:12.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0063345809999999997 2024-09-17T22:42:12.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756257+0000", 2024-09-17T22:42:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103583 2024-09-17T22:42:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756361+0000", 2024-09-17T22:42:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103853 2024-09-17T22:42:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.497 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.498 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.498 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37661 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:12.498 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:42:12.498 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.688808963, 2024-09-17T22:42:12.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067442300000000004, 2024-09-17T22:42:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37661 2024-09-17T22:42:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:42:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:42:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749831+0000", 2024-09-17T22:42:12.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9399999999999995e-07 2024-09-17T22:42:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749835+0000", 2024-09-17T22:42:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1049999999999997e-06 2024-09-17T22:42:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749855+0000", 2024-09-17T22:42:12.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9343000000000001e-05 2024-09-17T22:42:12.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749860+0000", 2024-09-17T22:42:12.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4310000000000002e-06 2024-09-17T22:42:12.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756395+0000", 2024-09-17T22:42:12.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065349900000000001 2024-09-17T22:42:12.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756479+0000", 2024-09-17T22:42:12.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4030999999999996e-05 2024-09-17T22:42:12.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756575+0000", 2024-09-17T22:42:12.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5835999999999995e-05 2024-09-17T22:42:12.506 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.507 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.507 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.507 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37662 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:12.508 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:42:12.508 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.688755157000003, 2024-09-17T22:42:12.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006927295, 2024-09-17T22:42:12.508 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.508 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.508 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:12.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:12.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:12.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.509 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37662 2024-09-17T22:42:12.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.509 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:42:12.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749884+0000", 2024-09-17T22:42:12.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749885+0000", 2024-09-17T22:42:12.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2699999999999999e-07 2024-09-17T22:42:12.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749888+0000", 2024-09-17T22:42:12.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.6599999999999999e-06 2024-09-17T22:42:12.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749895+0000", 2024-09-17T22:42:12.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4200000000000001e-06 2024-09-17T22:42:12.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749901+0000", 2024-09-17T22:42:12.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5090000000000001e-06 2024-09-17T22:42:12.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756605+0000", 2024-09-17T22:42:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067040629999999997 2024-09-17T22:42:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756691+0000", 2024-09-17T22:42:12.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5846000000000004e-05 2024-09-17T22:42:12.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756812+0000", 2024-09-17T22:42:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012116999999999999 2024-09-17T22:42:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37663 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 249856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:42:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.688730947000003, 2024-09-17T22:42:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071811440000000004, 2024-09-17T22:42:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37663 2024-09-17T22:42:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:42:12.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:42:12.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749909+0000", 2024-09-17T22:42:12.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:42:12.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749911+0000", 2024-09-17T22:42:12.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.035e-06 2024-09-17T22:42:12.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749917+0000", 2024-09-17T22:42:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4370000000000004e-06 2024-09-17T22:42:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749922+0000", 2024-09-17T22:42:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6249999999999998e-06 2024-09-17T22:42:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756845+0000", 2024-09-17T22:42:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0069227280000000004 2024-09-17T22:42:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.756994+0000", 2024-09-17T22:42:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000149068 2024-09-17T22:42:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757090+0000", 2024-09-17T22:42:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5851000000000001e-05 2024-09-17T22:42:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:37664 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 256000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:42:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.688710102000002, 2024-09-17T22:42:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.007384743, 2024-09-17T22:42:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 37664 2024-09-17T22:42:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:42:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749929+0000", 2024-09-17T22:42:12.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749930+0000", 2024-09-17T22:42:12.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:42:12.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749933+0000", 2024-09-17T22:42:12.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8430000000000001e-06 2024-09-17T22:42:12.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749939+0000", 2024-09-17T22:42:12.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:42:12.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.749944+0000", 2024-09-17T22:42:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9230000000000001e-06 2024-09-17T22:42:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757124+0000", 2024-09-17T22:42:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0071800020000000004 2024-09-17T22:42:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757212+0000", 2024-09-17T22:42:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7383e-05 2024-09-17T22:42:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.757314+0000", 2024-09-17T22:42:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010247399999999999 2024-09-17T22:42:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:39363 3.b 3:d322b826:::rbd_data.115fe0c30ef6.000000000000000e:5 [read 260096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:42:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 42.646721763000002, 2024-09-17T22:42:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.006226245, 2024-09-17T22:42:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 39363 2024-09-17T22:42:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:42:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:42:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791918+0000", 2024-09-17T22:42:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3500000000000002e-07 2024-09-17T22:42:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791920+0000", 2024-09-17T22:42:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.869e-06 2024-09-17T22:42:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791926+0000", 2024-09-17T22:42:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8490000000000002e-06 2024-09-17T22:42:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.791931+0000", 2024-09-17T22:42:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4269999999999999e-06 2024-09-17T22:42:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.797912+0000", 2024-09-17T22:42:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0059808819999999999 2024-09-17T22:42:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.798020+0000", 2024-09-17T22:42:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010783800000000001 2024-09-17T22:42:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:29.798144+0000", 2024-09-17T22:42:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00012404500000000001 2024-09-17T22:42:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4447.0:51869 3.6 e35/23 3:6037e35f:::rbd_object_map.115fe9ac8535:head v 35'71, mlcod=35'66)", 2024-09-17T22:42:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:42:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 38.661399463000002, 2024-09-17T22:42:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0066208669999999999, 2024-09-17T22:42:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:42:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777240+0000", 2024-09-17T22:42:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777241+0000", 2024-09-17T22:42:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4000000000000001e-07 2024-09-17T22:42:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777246+0000", 2024-09-17T22:42:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2789999999999998e-06 2024-09-17T22:42:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777255+0000", 2024-09-17T22:42:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6209999999999993e-06 2024-09-17T22:42:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777259+0000", 2024-09-17T22:42:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7650000000000001e-06 2024-09-17T22:42:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.545 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777298+0000", 2024-09-17T22:42:12.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8797999999999997e-05 2024-09-17T22:42:12.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.545 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.777311+0000", 2024-09-17T22:42:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2364999999999999e-05 2024-09-17T22:42:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.783814+0000", 2024-09-17T22:42:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0065039169999999997 2024-09-17T22:42:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:33.783861+0000", 2024-09-17T22:42:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6482e-05 2024-09-17T22:42:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:105336 3.b 3:d3549b8d:::rbd_data.115ff9e8d63c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-17T22:42:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:43.787529+0000", 2024-09-17T22:42:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.651110158000002, 2024-09-17T22:42:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0099864259999999996, 2024-09-17T22:42:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 105336 2024-09-17T22:42:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.787529+0000", 2024-09-17T22:42:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.787529+0000", 2024-09-17T22:42:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.787531+0000", 2024-09-17T22:42:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9759999999999998e-06 2024-09-17T22:42:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788120+0000", 2024-09-17T22:42:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00058899799999999997 2024-09-17T22:42:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788138+0000", 2024-09-17T22:42:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7393000000000001e-05 2024-09-17T22:42:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788146+0000", 2024-09-17T22:42:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8220000000000007e-06 2024-09-17T22:42:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788199+0000", 2024-09-17T22:42:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3718000000000003e-05 2024-09-17T22:42:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.788316+0000", 2024-09-17T22:42:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000116999 2024-09-17T22:42:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:42:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.791608+0000", 2024-09-17T22:42:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003291188 2024-09-17T22:42:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797379+0000", 2024-09-17T22:42:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0057716829999999997 2024-09-17T22:42:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:42:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797401+0000", 2024-09-17T22:42:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1729000000000002e-05 2024-09-17T22:42:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797459+0000", 2024-09-17T22:42:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8115000000000002e-05 2024-09-17T22:42:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:43.797516+0000", 2024-09-17T22:42:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6805000000000002e-05 2024-09-17T22:42:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop(client.4447.0:177615 3.0 e57/23 3:036f4184:::rbd_data.115f31bbb4b1.000000000000001c:head v 57'151, mlcod=55'148)", 2024-09-17T22:42:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:59.581018+0000", 2024-09-17T22:42:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 12.857621755, 2024-09-17T22:42:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0067345199999999999, 2024-09-17T22:42:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581018+0000", 2024-09-17T22:42:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581018+0000", 2024-09-17T22:42:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581018+0000", 2024-09-17T22:42:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4499999999999997e-07 2024-09-17T22:42:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581024+0000", 2024-09-17T22:42:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9630000000000003e-06 2024-09-17T22:42:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581036+0000", 2024-09-17T22:42:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1287999999999999e-05 2024-09-17T22:42:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581044+0000", 2024-09-17T22:42:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8220000000000007e-06 2024-09-17T22:42:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581094+0000", 2024-09-17T22:42:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0475e-05 2024-09-17T22:42:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.581112+0000", 2024-09-17T22:42:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7932e-05 2024-09-17T22:42:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.587716+0000", 2024-09-17T22:42:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0066042879999999998 2024-09-17T22:42:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:59.587752+0000", 2024-09-17T22:42:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6106999999999998e-05 2024-09-17T22:42:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:178091 3.b 3:d1597d64:::rbd_data.115f31bbb4b1.000000000000006b:head [set-alloc-hint object_size 2097152 write_size 2097152,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-17T22:42:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:00.078810+0000", 2024-09-17T22:42:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 12.359829649, 2024-09-17T22:42:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.017828758, 2024-09-17T22:42:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 178091 2024-09-17T22:42:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078810+0000", 2024-09-17T22:42:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078810+0000", 2024-09-17T22:42:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078811+0000", 2024-09-17T22:42:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.176e-06 2024-09-17T22:42:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078817+0000", 2024-09-17T22:42:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6540000000000004e-06 2024-09-17T22:42:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078828+0000", 2024-09-17T22:42:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0875e-05 2024-09-17T22:42:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078833+0000", 2024-09-17T22:42:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6099999999999997e-06 2024-09-17T22:42:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078884+0000", 2024-09-17T22:42:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.1013000000000001e-05 2024-09-17T22:42:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.078933+0000", 2024-09-17T22:42:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8380000000000001e-05 2024-09-17T22:42:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:42:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.081828+0000", 2024-09-17T22:42:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0028952219999999998 2024-09-17T22:42:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.096563+0000", 2024-09-17T22:42:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.014734860000000001 2024-09-17T22:42:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:42:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.096575+0000", 2024-09-17T22:42:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2126e-05 2024-09-17T22:42:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.096608+0000", 2024-09-17T22:42:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2836000000000002e-05 2024-09-17T22:42:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:00.096639+0000", 2024-09-17T22:42:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1006000000000001e-05 2024-09-17T22:42:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.567 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.567 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:12.567 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:12.567 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:12.570 INFO:teuthology.orchestra.run.smithi115.stdout:2987 punch from 0xab792c3 to 0xab862b7, (0xcff4 bytes) 2024-09-17T22:42:12.570 INFO:teuthology.orchestra.run.smithi115.stdout:2989 write 0x925b685 thru 0x9265b3b (0xa4b7 bytes) 2024-09-17T22:42:12.570 INFO:teuthology.orchestra.run.smithi115.stdout:2990 write 0x5bfb7da thru 0x5c06f14 (0xb73b bytes) 2024-09-17T22:42:12.570 INFO:teuthology.orchestra.run.smithi115.stdout:2991 write 0x72a65f2 thru 0x72ac4ee (0x5efd bytes) 2024-09-17T22:42:12.571 INFO:teuthology.orchestra.run.smithi115.stdout:2996 trunc from 0xeba1224 to 0x2b7909e 2024-09-17T22:42:12.571 INFO:teuthology.orchestra.run.smithi115.stdout:2997 clone 27 order 25 su 524288 sc 14 2024-09-17T22:42:12.571 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:12.566+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:12.572 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:42:12.670 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:12.667+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:12.676 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:12.676 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:12.676 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:12.676 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:12.685 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:42:12.770 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:12.768+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:12.788 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:12.788 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:12.788 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:12.788 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:12.788 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:12.797 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:42:12.871 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:12.869+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:12.901 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:12.901 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:12.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:12.901 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:12.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.902 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:42:12.902 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:12.902 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.539100320000003, 2024-09-17T22:42:12.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.249158453, 2024-09-17T22:42:12.902 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:12.902 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:12.902 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:12.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:12.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:42:12.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:42:12.903 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-17T22:42:12.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.903 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:12.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:12.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:12.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:12.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:12.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:12.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358680+0000", 2024-09-17T22:42:12.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.592e-06 2024-09-17T22:42:12.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:12.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358688+0000", 2024-09-17T22:42:12.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1610000000000003e-06 2024-09-17T22:42:12.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:12.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358697+0000", 2024-09-17T22:42:12.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2580000000000001e-06 2024-09-17T22:42:12.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358706+0000", 2024-09-17T22:42:12.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1640000000000007e-06 2024-09-17T22:42:12.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:12.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358766+0000", 2024-09-17T22:42:12.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9893999999999999e-05 2024-09-17T22:42:12.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:42:12.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358778+0000", 2024-09-17T22:42:12.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1148999999999999e-05 2024-09-17T22:42:12.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:12.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606347+0000", 2024-09-17T22:42:12.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.247569971 2024-09-17T22:42:12.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:12.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606533+0000", 2024-09-17T22:42:12.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018599400000000001 2024-09-17T22:42:12.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:42:12.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606752+0000", 2024-09-17T22:42:12.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000218373 2024-09-17T22:42:12.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:42:12.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607455+0000", 2024-09-17T22:42:12.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070311799999999997 2024-09-17T22:42:12.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:12.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607735+0000", 2024-09-17T22:42:12.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027972300000000001 2024-09-17T22:42:12.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:42:12.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607793+0000", 2024-09-17T22:42:12.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8529999999999997e-05 2024-09-17T22:42:12.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:12.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607837+0000", 2024-09-17T22:42:12.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3526000000000003e-05 2024-09-17T22:42:12.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.910 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:12.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.910 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186749 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 11960320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:12.910 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:12.911 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.4744589980000002, 2024-09-17T22:42:12.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029226166000000001, 2024-09-17T22:42:12.911 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:12.911 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:12.911 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:12.911 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:12.911 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:12.911 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.911 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186749 2024-09-17T22:42:12.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.911 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:12.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:12.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:12.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:12.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:12.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:12.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:12.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-09-17T22:42:12.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:12.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423328+0000", 2024-09-17T22:42:12.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6790000000000005e-06 2024-09-17T22:42:12.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:12.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423334+0000", 2024-09-17T22:42:12.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0009999999999998e-06 2024-09-17T22:42:12.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423338+0000", 2024-09-17T22:42:12.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6419999999999998e-06 2024-09-17T22:42:12.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:12.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452464+0000", 2024-09-17T22:42:12.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029125389000000002 2024-09-17T22:42:12.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:12.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452501+0000", 2024-09-17T22:42:12.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7656e-05 2024-09-17T22:42:12.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:12.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452546+0000", 2024-09-17T22:42:12.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4468999999999997e-05 2024-09-17T22:42:12.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.915 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:12.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.916 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186750 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12050432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:12.916 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:12.916 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.4744332079999998, 2024-09-17T22:42:12.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029307863999999999, 2024-09-17T22:42:12.916 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:12.916 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:12.916 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:12.916 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:12.916 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:12.916 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.916 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186750 2024-09-17T22:42:12.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.917 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:12.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:12.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:12.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:12.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:12.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423346+0000", 2024-09-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.84e-07 2024-09-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423348+0000", 2024-09-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7540000000000001e-06 2024-09-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:12.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423353+0000", 2024-09-17T22:42:12.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:42:12.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423362+0000", 2024-09-17T22:42:12.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9220000000000003e-06 2024-09-17T22:42:12.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:12.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452571+0000", 2024-09-17T22:42:12.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029209174000000001 2024-09-17T22:42:12.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:12.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452612+0000", 2024-09-17T22:42:12.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1363999999999999e-05 2024-09-17T22:42:12.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:12.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452653+0000", 2024-09-17T22:42:12.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0893999999999998e-05 2024-09-17T22:42:12.922 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.922 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:12.922 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186751 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12140544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.4744107179999997, 2024-09-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029387488, 2024-09-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:12.923 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:12.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.924 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186751 2024-09-17T22:42:12.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.924 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:12.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:12.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:12.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:12.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:12.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:12.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:12.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1399999999999997e-07 2024-09-17T22:42:12.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:12.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423375+0000", 2024-09-17T22:42:12.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8889999999999996e-06 2024-09-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423381+0000", 2024-09-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5690000000000004e-06 2024-09-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423386+0000", 2024-09-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9559999999999996e-06 2024-09-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452679+0000", 2024-09-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029292820000000001 2024-09-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452718+0000", 2024-09-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8961000000000002e-05 2024-09-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:12.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452755+0000", 2024-09-17T22:42:12.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7879000000000001e-05 2024-09-17T22:42:12.928 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.928 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:12.928 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.928 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186752 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12230656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:12.928 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:12.928 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.4743794670000003, 2024-09-17T22:42:12.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029463269, 2024-09-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186752 2024-09-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:12.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:12.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:12.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:12.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:12.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423400+0000", 2024-09-17T22:42:12.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:42:12.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:12.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423401+0000", 2024-09-17T22:42:12.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3599999999999999e-06 2024-09-17T22:42:12.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:12.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423404+0000", 2024-09-17T22:42:12.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5999999999999998e-06 2024-09-17T22:42:12.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423407+0000", 2024-09-17T22:42:12.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6869999999999999e-06 2024-09-17T22:42:12.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:12.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452782+0000", 2024-09-17T22:42:12.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029375107000000001 2024-09-17T22:42:12.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:12.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452821+0000", 2024-09-17T22:42:12.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9122999999999998e-05 2024-09-17T22:42:12.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:12.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452862+0000", 2024-09-17T22:42:12.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1019000000000001e-05 2024-09-17T22:42:12.934 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.934 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:12.934 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.934 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186753 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:12.935 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:12.935 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.4743674450000004, 2024-09-17T22:42:12.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029553297999999999, 2024-09-17T22:42:12.935 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:12.935 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:12.936 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:12.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:12.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:12.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.936 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186753 2024-09-17T22:42:12.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.937 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:12.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:12.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:12.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:12.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:12.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:12.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:12.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.98e-07 2024-09-17T22:42:12.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:12.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423412+0000", 2024-09-17T22:42:12.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8700000000000004e-07 2024-09-17T22:42:12.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:12.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423415+0000", 2024-09-17T22:42:12.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:42:12.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423418+0000", 2024-09-17T22:42:12.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.283e-06 2024-09-17T22:42:12.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:12.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452893+0000", 2024-09-17T22:42:12.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029475881999999998 2024-09-17T22:42:12.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:12.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452928+0000", 2024-09-17T22:42:12.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4109000000000002e-05 2024-09-17T22:42:12.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:12.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452964+0000", 2024-09-17T22:42:12.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6937000000000002e-05 2024-09-17T22:42:12.943 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.943 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:12.944 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.944 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186754 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12410880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:12.944 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:12.944 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.4743583080000002, 2024-09-17T22:42:12.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029656021000000001, 2024-09-17T22:42:12.945 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:12.945 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:12.945 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:12.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:12.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:12.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.946 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186754 2024-09-17T22:42:12.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.946 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:12.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:12.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:12.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:12.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:12.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:12.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423421+0000", 2024-09-17T22:42:12.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9999999999999999e-07 2024-09-17T22:42:12.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:12.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423425+0000", 2024-09-17T22:42:12.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6519999999999997e-06 2024-09-17T22:42:12.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:12.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423428+0000", 2024-09-17T22:42:12.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1370000000000002e-06 2024-09-17T22:42:12.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423431+0000", 2024-09-17T22:42:12.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4090000000000001e-06 2024-09-17T22:42:12.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:12.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452992+0000", 2024-09-17T22:42:12.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029561545000000002 2024-09-17T22:42:12.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:12.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453031+0000", 2024-09-17T22:42:12.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8834000000000001e-05 2024-09-17T22:42:12.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:12.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453076+0000", 2024-09-17T22:42:12.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5244000000000003e-05 2024-09-17T22:42:12.953 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.953 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:12.953 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.954 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186755 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12500992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:12.954 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:12.954 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.474345027, 2024-09-17T22:42:12.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029745649999999998, 2024-09-17T22:42:12.954 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:12.954 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:12.954 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:12.955 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:12.955 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:12.955 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.955 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186755 2024-09-17T22:42:12.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.955 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:12.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:12.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:12.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.17e-07 2024-09-17T22:42:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423435+0000", 2024-09-17T22:42:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:42:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423438+0000", 2024-09-17T22:42:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8210000000000002e-06 2024-09-17T22:42:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423443+0000", 2024-09-17T22:42:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1070000000000004e-06 2024-09-17T22:42:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453103+0000", 2024-09-17T22:42:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029660399 2024-09-17T22:42:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453141+0000", 2024-09-17T22:42:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.748e-05 2024-09-17T22:42:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453179+0000", 2024-09-17T22:42:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8671000000000003e-05 2024-09-17T22:42:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187439 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8437760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.3569079300000002, 2024-09-17T22:42:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029278048000000001, 2024-09-17T22:42:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187439 2024-09-17T22:42:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6199999999999999e-07 2024-09-17T22:42:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540872+0000", 2024-09-17T22:42:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.09e-07 2024-09-17T22:42:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540875+0000", 2024-09-17T22:42:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8990000000000001e-06 2024-09-17T22:42:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540877+0000", 2024-09-17T22:42:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1789999999999998e-06 2024-09-17T22:42:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570074+0000", 2024-09-17T22:42:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029197358 2024-09-17T22:42:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570111+0000", 2024-09-17T22:42:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6170000000000001e-05 2024-09-17T22:42:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570149+0000", 2024-09-17T22:42:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8170999999999998e-05 2024-09-17T22:42:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187440 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8527872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.3568924600000001, 2024-09-17T22:42:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029359758999999999, 2024-09-17T22:42:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187440 2024-09-17T22:42:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2700000000000001e-07 2024-09-17T22:42:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540888+0000", 2024-09-17T22:42:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-17T22:42:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540893+0000", 2024-09-17T22:42:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9110000000000001e-06 2024-09-17T22:42:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540895+0000", 2024-09-17T22:42:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.712e-06 2024-09-17T22:42:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570168+0000", 2024-09-17T22:42:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029273114999999999 2024-09-17T22:42:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570208+0000", 2024-09-17T22:42:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9227999999999998e-05 2024-09-17T22:42:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570246+0000", 2024-09-17T22:42:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8253000000000001e-05 2024-09-17T22:42:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187441 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8617984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.3568794960000004, 2024-09-17T22:42:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029442211999999999, 2024-09-17T22:42:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187441 2024-09-17T22:42:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:42:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540900+0000", 2024-09-17T22:42:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7999999999999993e-07 2024-09-17T22:42:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540903+0000", 2024-09-17T22:42:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9129999999999999e-06 2024-09-17T22:42:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540909+0000", 2024-09-17T22:42:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4269999999999999e-06 2024-09-17T22:42:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570267+0000", 2024-09-17T22:42:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029358308 2024-09-17T22:42:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570304+0000", 2024-09-17T22:42:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6860999999999998e-05 2024-09-17T22:42:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570341+0000", 2024-09-17T22:42:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7509000000000003e-05 2024-09-17T22:42:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187442 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8708096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.3568667889999997, 2024-09-17T22:42:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029522606999999999, 2024-09-17T22:42:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187442 2024-09-17T22:42:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.22e-07 2024-09-17T22:42:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540917+0000", 2024-09-17T22:42:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999998e-06 2024-09-17T22:42:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540920+0000", 2024-09-17T22:42:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4989999999999999e-06 2024-09-17T22:42:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540923+0000", 2024-09-17T22:42:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3870000000000002e-06 2024-09-17T22:42:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570361+0000", 2024-09-17T22:42:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029438345000000001 2024-09-17T22:42:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570397+0000", 2024-09-17T22:42:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5429000000000002e-05 2024-09-17T22:42:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570434+0000", 2024-09-17T22:42:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7824999999999998e-05 2024-09-17T22:42:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187443 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8798208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.3568528620000002, 2024-09-17T22:42:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029603640000000001, 2024-09-17T22:42:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187443 2024-09-17T22:42:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.05e-07 2024-09-17T22:42:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540927+0000", 2024-09-17T22:42:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:42:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540930+0000", 2024-09-17T22:42:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9229999999999998e-06 2024-09-17T22:42:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540932+0000", 2024-09-17T22:42:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.294e-06 2024-09-17T22:42:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570454+0000", 2024-09-17T22:42:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029522230999999999 2024-09-17T22:42:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570491+0000", 2024-09-17T22:42:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6835999999999998e-05 2024-09-17T22:42:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570529+0000", 2024-09-17T22:42:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8167999999999998e-05 2024-09-17T22:42:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187444 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8888320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.3568433879999997, 2024-09-17T22:42:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029687584999999999, 2024-09-17T22:42:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187444 2024-09-17T22:42:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-09-17T22:42:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540936+0000", 2024-09-17T22:42:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3900000000000003e-07 2024-09-17T22:42:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540939+0000", 2024-09-17T22:42:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.994e-06 2024-09-17T22:42:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540942+0000", 2024-09-17T22:42:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1739999999999999e-06 2024-09-17T22:42:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570549+0000", 2024-09-17T22:42:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029607652000000002 2024-09-17T22:42:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570585+0000", 2024-09-17T22:42:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5775000000000002e-05 2024-09-17T22:42:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:12.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570623+0000", 2024-09-17T22:42:12.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7839999999999997e-05 2024-09-17T22:42:12.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.993 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:12.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187445 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8978432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.3568344999999997, 2024-09-17T22:42:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029772837999999999, 2024-09-17T22:42:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187445 2024-09-17T22:42:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.996 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:12.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:12.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:12.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:12.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:12.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:42:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540945+0000", 2024-09-17T22:42:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3699999999999999e-07 2024-09-17T22:42:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540948+0000", 2024-09-17T22:42:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.898e-06 2024-09-17T22:42:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540950+0000", 2024-09-17T22:42:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2249999999999999e-06 2024-09-17T22:42:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570643+0000", 2024-09-17T22:42:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029692463999999998 2024-09-17T22:42:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570680+0000", 2024-09-17T22:42:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6772e-05 2024-09-17T22:42:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570717+0000", 2024-09-17T22:42:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7438999999999998e-05 2024-09-17T22:42:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187446 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9068544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.3568257619999997, 2024-09-17T22:42:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029856321000000002, 2024-09-17T22:42:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187446 2024-09-17T22:42:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:13.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:42:13.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540958+0000", 2024-09-17T22:42:13.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3449999999999996e-06 2024-09-17T22:42:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540962+0000", 2024-09-17T22:42:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3529999999999999e-06 2024-09-17T22:42:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540964+0000", 2024-09-17T22:42:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3810000000000002e-06 2024-09-17T22:42:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570737+0000", 2024-09-17T22:42:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029772824 2024-09-17T22:42:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570770+0000", 2024-09-17T22:42:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2835e-05 2024-09-17T22:42:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570809+0000", 2024-09-17T22:42:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9379999999999999e-05 2024-09-17T22:42:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.014 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187447 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9158656~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:13.014 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:13.014 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.3568117940000004, 2024-09-17T22:42:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02993262, 2024-09-17T22:42:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.016 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.016 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187447 2024-09-17T22:42:13.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.016 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-09-17T22:42:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540968+0000", 2024-09-17T22:42:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6500000000000008e-07 2024-09-17T22:42:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540971+0000", 2024-09-17T22:42:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9869999999999999e-06 2024-09-17T22:42:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540973+0000", 2024-09-17T22:42:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3120000000000001e-06 2024-09-17T22:42:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570829+0000", 2024-09-17T22:42:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029855742000000001 2024-09-17T22:42:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570861+0000", 2024-09-17T22:42:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2152e-05 2024-09-17T22:42:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570899+0000", 2024-09-17T22:42:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8241999999999997e-05 2024-09-17T22:42:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187448 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9248768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.3568027640000002, 2024-09-17T22:42:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030014865000000002, 2024-09-17T22:42:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187448 2024-09-17T22:42:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9600000000000001e-07 2024-09-17T22:42:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540977+0000", 2024-09-17T22:42:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.16e-07 2024-09-17T22:42:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540980+0000", 2024-09-17T22:42:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.892e-06 2024-09-17T22:42:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540982+0000", 2024-09-17T22:42:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3410000000000001e-06 2024-09-17T22:42:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570919+0000", 2024-09-17T22:42:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029936800999999999 2024-09-17T22:42:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570953+0000", 2024-09-17T22:42:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3939000000000002e-05 2024-09-17T22:42:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570991+0000", 2024-09-17T22:42:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7780000000000001e-05 2024-09-17T22:42:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187449 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9338880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.3567935850000001, 2024-09-17T22:42:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030110238000000001, 2024-09-17T22:42:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187449 2024-09-17T22:42:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1500000000000001e-07 2024-09-17T22:42:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540986+0000", 2024-09-17T22:42:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9500000000000001e-07 2024-09-17T22:42:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540989+0000", 2024-09-17T22:42:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7949999999999999e-06 2024-09-17T22:42:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540994+0000", 2024-09-17T22:42:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0359999999999997e-06 2024-09-17T22:42:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571011+0000", 2024-09-17T22:42:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030016915000000002 2024-09-17T22:42:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571056+0000", 2024-09-17T22:42:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5457999999999997e-05 2024-09-17T22:42:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571095+0000", 2024-09-17T22:42:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8924000000000002e-05 2024-09-17T22:42:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187450 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9428992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.3567819380000001, 2024-09-17T22:42:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030188204999999999, 2024-09-17T22:42:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187450 2024-09-17T22:42:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.16e-07 2024-09-17T22:42:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540998+0000", 2024-09-17T22:42:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9000000000000005e-07 2024-09-17T22:42:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.541001+0000", 2024-09-17T22:42:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0680000000000001e-06 2024-09-17T22:42:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.541006+0000", 2024-09-17T22:42:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4720000000000003e-06 2024-09-17T22:42:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571110+0000", 2024-09-17T22:42:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030103431 2024-09-17T22:42:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571147+0000", 2024-09-17T22:42:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6748999999999997e-05 2024-09-17T22:42:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571185+0000", 2024-09-17T22:42:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8278999999999997e-05 2024-09-17T22:42:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.036 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.036 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.036 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.036 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.036 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:13.037 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:42:13.038 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:13.035+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:13.139 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:13.136+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:13.141 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:13.142 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:13.142 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:13.142 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:13.150 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:42:13.210 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is unset 2024-09-17T22:42:13.239 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:13.236+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:13.287 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone26 from 0x2b7909e (overlap 0x14ed63f) to 0x217bd3 2024-09-17T22:42:13.303 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:13.303 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:13.304 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:13.304 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:13.304 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:13.313 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:42:13.356 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:13.354+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:13.366 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-17T22:42:13.395 INFO:teuthology.orchestra.run.smithi115.stdout:2998 writesame 0xacad76d thru 0xacb3a94 data_size 0x4c(0x6328 bytes) 2024-09-17T22:42:13.399 INFO:teuthology.orchestra.run.smithi115.stdout:2999 write 0x537a22a thru 0x537a34f (0x126 bytes) 2024-09-17T22:42:13.404 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:13.404 INFO:teuthology.orchestra.run.smithi115.stdout:3002 write 0x36519f1 thru 0x36618b3 (0xfec3 bytes) 2024-09-17T22:42:13.408 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:13.408 INFO:teuthology.orchestra.run.smithi115.stdout:3004 read 0x9ce0cfd thru 0x9ce3d8b (0x308f bytes) 2024-09-17T22:42:13.408 INFO:teuthology.orchestra.run.smithi115.stdout:3005 write 0x2da154 thru 0x2e6efc (0xcda9 bytes) 2024-09-17T22:42:13.413 INFO:teuthology.orchestra.run.smithi115.stdout:3006 write 0x3dc7e05 thru 0x3dcddcf (0x5fcb bytes) 2024-09-17T22:42:13.418 INFO:teuthology.orchestra.run.smithi115.stdout:3007 punch from 0x16e3880 to 0x16f35e6, (0xfd66 bytes) 2024-09-17T22:42:13.418 INFO:teuthology.orchestra.run.smithi115.stdout:3009 read 0x74bd26d thru 0x74c8382 (0xb116 bytes) 2024-09-17T22:42:13.418 INFO:teuthology.orchestra.run.smithi115.stdout:3010 read 0x26d6288 thru 0x26d8e48 (0x2bc1 bytes) 2024-09-17T22:42:13.418 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:13.419 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:13.419 INFO:teuthology.orchestra.run.smithi115.stdout:3014 clone 28 order 19 su 16384 sc 3 2024-09-17T22:42:13.425 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:13.425 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:13.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:13.425 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:13.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.425 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145246 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3850240~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.425 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:13.425 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.091967477000001, 2024-09-17T22:42:13.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052644991000000002, 2024-09-17T22:42:13.425 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.425 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.426 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.426 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.426 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.426 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.426 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145246 2024-09-17T22:42:13.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.426 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:13.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:13.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329899+0000", 2024-09-17T22:42:13.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-09-17T22:42:13.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329901+0000", 2024-09-17T22:42:13.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1619999999999998e-06 2024-09-17T22:42:13.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329907+0000", 2024-09-17T22:42:13.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9059999999999999e-06 2024-09-17T22:42:13.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329911+0000", 2024-09-17T22:42:13.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7419999999999997e-06 2024-09-17T22:42:13.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382383+0000", 2024-09-17T22:42:13.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052471583000000002 2024-09-17T22:42:13.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382457+0000", 2024-09-17T22:42:13.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4309999999999998e-05 2024-09-17T22:42:13.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382543+0000", 2024-09-17T22:42:13.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5918e-05 2024-09-17T22:42:13.430 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.430 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.430 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.430 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145247 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3946496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.430 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:13.430 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.091948827, 2024-09-17T22:42:13.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052805863000000001, 2024-09-17T22:42:13.430 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.430 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.430 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.430 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.431 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.431 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.431 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145247 2024-09-17T22:42:13.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.431 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:13.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:13.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:13.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3200000000000001e-07 2024-09-17T22:42:13.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329919+0000", 2024-09-17T22:42:13.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.888e-06 2024-09-17T22:42:13.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329925+0000", 2024-09-17T22:42:13.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5369999999999997e-06 2024-09-17T22:42:13.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329929+0000", 2024-09-17T22:42:13.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.639e-06 2024-09-17T22:42:13.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382570+0000", 2024-09-17T22:42:13.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052641213999999999 2024-09-17T22:42:13.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382636+0000", 2024-09-17T22:42:13.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6008999999999998e-05 2024-09-17T22:42:13.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382723+0000", 2024-09-17T22:42:13.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6243999999999996e-05 2024-09-17T22:42:13.434 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.434 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.434 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.435 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145248 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3856384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.435 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:13.435 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.091930778999998, 2024-09-17T22:42:13.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053002858, 2024-09-17T22:42:13.435 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.435 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.435 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.435 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.435 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.435 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.435 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145248 2024-09-17T22:42:13.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.435 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:13.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:13.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:13.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.13e-07 2024-09-17T22:42:13.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329937+0000", 2024-09-17T22:42:13.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9300000000000002e-06 2024-09-17T22:42:13.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329943+0000", 2024-09-17T22:42:13.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-09-17T22:42:13.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329948+0000", 2024-09-17T22:42:13.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8579999999999999e-06 2024-09-17T22:42:13.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382763+0000", 2024-09-17T22:42:13.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052814537000000002 2024-09-17T22:42:13.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382840+0000", 2024-09-17T22:42:13.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6879999999999996e-05 2024-09-17T22:42:13.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382938+0000", 2024-09-17T22:42:13.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8159999999999995e-05 2024-09-17T22:42:13.439 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.439 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.439 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.439 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145249 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3952640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.439 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:13.439 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.091911878000001, 2024-09-17T22:42:13.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053188602000000001, 2024-09-17T22:42:13.439 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.439 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.440 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.440 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.440 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.440 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.440 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145249 2024-09-17T22:42:13.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.440 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:13.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:13.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:13.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3700000000000001e-07 2024-09-17T22:42:13.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329956+0000", 2024-09-17T22:42:13.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1430000000000001e-06 2024-09-17T22:42:13.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329962+0000", 2024-09-17T22:42:13.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8270000000000003e-06 2024-09-17T22:42:13.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329967+0000", 2024-09-17T22:42:13.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7249999999999997e-06 2024-09-17T22:42:13.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382969+0000", 2024-09-17T22:42:13.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053001952999999997 2024-09-17T22:42:13.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383040+0000", 2024-09-17T22:42:13.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1272999999999993e-05 2024-09-17T22:42:13.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383142+0000", 2024-09-17T22:42:13.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010234399999999999 2024-09-17T22:42:13.443 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.443 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.444 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.444 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145250 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3862528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.444 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:13.444 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.091893078999998, 2024-09-17T22:42:13.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053359537999999998, 2024-09-17T22:42:13.444 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.444 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.444 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.444 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.444 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.444 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.445 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145250 2024-09-17T22:42:13.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.445 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:13.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:13.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:13.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:42:13.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329975+0000", 2024-09-17T22:42:13.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2199999999999999e-06 2024-09-17T22:42:13.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329981+0000", 2024-09-17T22:42:13.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1020000000000002e-06 2024-09-17T22:42:13.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329991+0000", 2024-09-17T22:42:13.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6800000000000005e-06 2024-09-17T22:42:13.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383171+0000", 2024-09-17T22:42:13.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053180142 2024-09-17T22:42:13.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383242+0000", 2024-09-17T22:42:13.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1310000000000007e-05 2024-09-17T22:42:13.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383332+0000", 2024-09-17T22:42:13.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9703999999999994e-05 2024-09-17T22:42:13.448 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.448 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.448 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.448 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145251 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3958784~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.449 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:13.449 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.091867379, 2024-09-17T22:42:13.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053438537000000001, 2024-09-17T22:42:13.449 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.449 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.449 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.449 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.449 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.449 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.449 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145251 2024-09-17T22:42:13.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.449 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:13.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:13.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329999+0000", 2024-09-17T22:42:13.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-07 2024-09-17T22:42:13.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330001+0000", 2024-09-17T22:42:13.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:42:13.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330006+0000", 2024-09-17T22:42:13.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000004e-06 2024-09-17T22:42:13.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330011+0000", 2024-09-17T22:42:13.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4039999999999997e-06 2024-09-17T22:42:13.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383350+0000", 2024-09-17T22:42:13.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053339011999999998 2024-09-17T22:42:13.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383387+0000", 2024-09-17T22:42:13.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7085999999999997e-05 2024-09-17T22:42:13.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383437+0000", 2024-09-17T22:42:13.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9988000000000003e-05 2024-09-17T22:42:13.453 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.453 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.453 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.453 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145252 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3868672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.453 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:13.453 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.091848932000001, 2024-09-17T22:42:13.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053526724999999997, 2024-09-17T22:42:13.453 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.453 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.453 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.454 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.454 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.454 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.454 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145252 2024-09-17T22:42:13.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.454 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:13.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:13.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331054+0000", 2024-09-17T22:42:13.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010371670000000001 2024-09-17T22:42:13.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331059+0000", 2024-09-17T22:42:13.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1680000000000003e-06 2024-09-17T22:42:13.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331075+0000", 2024-09-17T22:42:13.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6028e-05 2024-09-17T22:42:13.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331084+0000", 2024-09-17T22:42:13.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3980000000000005e-06 2024-09-17T22:42:13.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383464+0000", 2024-09-17T22:42:13.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052379851999999998 2024-09-17T22:42:13.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383499+0000", 2024-09-17T22:42:13.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4610000000000002e-05 2024-09-17T22:42:13.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383543+0000", 2024-09-17T22:42:13.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4502000000000002e-05 2024-09-17T22:42:13.457 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.457 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.457 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.458 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145255 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3971072~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.458 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:13.458 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.090718840000001, 2024-09-17T22:42:13.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052677527000000002, 2024-09-17T22:42:13.458 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.458 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.458 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.458 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.458 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.458 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.458 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145255 2024-09-17T22:42:13.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.458 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:13.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:13.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:13.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8800000000000002e-07 2024-09-17T22:42:13.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331150+0000", 2024-09-17T22:42:13.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1160000000000002e-06 2024-09-17T22:42:13.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331156+0000", 2024-09-17T22:42:13.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7469999999999999e-06 2024-09-17T22:42:13.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331162+0000", 2024-09-17T22:42:13.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-17T22:42:13.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383750+0000", 2024-09-17T22:42:13.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052588441999999999 2024-09-17T22:42:13.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383782+0000", 2024-09-17T22:42:13.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1610999999999999e-05 2024-09-17T22:42:13.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383824+0000", 2024-09-17T22:42:13.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2512000000000002e-05 2024-09-17T22:42:13.462 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.462 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.462 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.462 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145256 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3880960~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.462 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:13.462 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.090697419000001, 2024-09-17T22:42:13.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052744775000000001, 2024-09-17T22:42:13.463 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.463 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.463 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.463 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.463 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.463 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.463 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145256 2024-09-17T22:42:13.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.463 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:13.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:13.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331169+0000", 2024-09-17T22:42:13.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:42:13.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331175+0000", 2024-09-17T22:42:13.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5660000000000003e-06 2024-09-17T22:42:13.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331182+0000", 2024-09-17T22:42:13.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0169999999999999e-06 2024-09-17T22:42:13.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331188+0000", 2024-09-17T22:42:13.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2750000000000003e-06 2024-09-17T22:42:13.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383839+0000", 2024-09-17T22:42:13.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052651728000000002 2024-09-17T22:42:13.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383872+0000", 2024-09-17T22:42:13.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2347e-05 2024-09-17T22:42:13.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383913+0000", 2024-09-17T22:42:13.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1362000000000001e-05 2024-09-17T22:42:13.466 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.466 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.466 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.467 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145257 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3977216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.467 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:13.467 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.090671370999999, 2024-09-17T22:42:13.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052809598999999999, 2024-09-17T22:42:13.467 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.467 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.467 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.467 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.467 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.467 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.467 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145257 2024-09-17T22:42:13.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.468 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:13.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:13.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333051+0000", 2024-09-17T22:42:13.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018567189999999999 2024-09-17T22:42:13.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333056+0000", 2024-09-17T22:42:13.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-17T22:42:13.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333067+0000", 2024-09-17T22:42:13.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1704e-05 2024-09-17T22:42:13.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333074+0000", 2024-09-17T22:42:13.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1280000000000002e-06 2024-09-17T22:42:13.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383927+0000", 2024-09-17T22:42:13.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050853094000000001 2024-09-17T22:42:13.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383960+0000", 2024-09-17T22:42:13.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2891000000000001e-05 2024-09-17T22:42:13.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.384004+0000", 2024-09-17T22:42:13.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3565e-05 2024-09-17T22:42:13.471 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.471 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.471 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.471 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146168 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2467840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.471 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:13.472 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.027182750000001, 2024-09-17T22:42:13.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052652501999999997, 2024-09-17T22:42:13.472 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.472 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.472 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.472 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.472 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.472 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.472 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146168 2024-09-17T22:42:13.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.472 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:13.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:13.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394989+0000", 2024-09-17T22:42:13.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030618300000000002 2024-09-17T22:42:13.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394994+0000", 2024-09-17T22:42:13.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.566e-06 2024-09-17T22:42:13.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395003+0000", 2024-09-17T22:42:13.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3619999999999995e-06 2024-09-17T22:42:13.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395009+0000", 2024-09-17T22:42:13.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3589999999999996e-06 2024-09-17T22:42:13.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447180+0000", 2024-09-17T22:42:13.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052170643000000003 2024-09-17T22:42:13.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447251+0000", 2024-09-17T22:42:13.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1382000000000002e-05 2024-09-17T22:42:13.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447335+0000", 2024-09-17T22:42:13.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4006999999999997e-05 2024-09-17T22:42:13.476 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.476 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.476 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.476 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146172 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2480128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.476 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:13.476 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.026827865000001, 2024-09-17T22:42:13.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052714629999999998, 2024-09-17T22:42:13.476 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.476 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.476 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.476 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.477 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.477 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.477 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146172 2024-09-17T22:42:13.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.477 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:13.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:13.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395486+0000", 2024-09-17T22:42:13.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044787000000000001 2024-09-17T22:42:13.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395491+0000", 2024-09-17T22:42:13.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8550000000000001e-06 2024-09-17T22:42:13.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395503+0000", 2024-09-17T22:42:13.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2271e-05 2024-09-17T22:42:13.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395510+0000", 2024-09-17T22:42:13.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6320000000000008e-06 2024-09-17T22:42:13.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447565+0000", 2024-09-17T22:42:13.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052055016000000003 2024-09-17T22:42:13.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447644+0000", 2024-09-17T22:42:13.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8306000000000005e-05 2024-09-17T22:42:13.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447752+0000", 2024-09-17T22:42:13.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010868 2024-09-17T22:42:13.480 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.480 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.480 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.481 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146181 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2504704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.481 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:13.481 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.025960475000002, 2024-09-17T22:42:13.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052850394000000002, 2024-09-17T22:42:13.481 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.481 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.481 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.481 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.481 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.481 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.481 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146181 2024-09-17T22:42:13.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.482 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:13.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:13.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396210+0000", 2024-09-17T22:42:13.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000305214 2024-09-17T22:42:13.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396214+0000", 2024-09-17T22:42:13.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9389999999999999e-06 2024-09-17T22:42:13.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396228+0000", 2024-09-17T22:42:13.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3365e-05 2024-09-17T22:42:13.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396235+0000", 2024-09-17T22:42:13.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2559999999999996e-06 2024-09-17T22:42:13.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448568+0000", 2024-09-17T22:42:13.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052333084000000002 2024-09-17T22:42:13.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448646+0000", 2024-09-17T22:42:13.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8287000000000004e-05 2024-09-17T22:42:13.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448756+0000", 2024-09-17T22:42:13.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109249 2024-09-17T22:42:13.485 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.485 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.485 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.485 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146183 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2510848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.485 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:13.486 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.025619928000001, 2024-09-17T22:42:13.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052725453999999998, 2024-09-17T22:42:13.486 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.486 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.486 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.486 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.486 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.486 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.486 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146183 2024-09-17T22:42:13.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.486 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:13.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:13.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396508+0000", 2024-09-17T22:42:13.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026271299999999998 2024-09-17T22:42:13.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396512+0000", 2024-09-17T22:42:13.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8039999999999999e-06 2024-09-17T22:42:13.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396522+0000", 2024-09-17T22:42:13.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9289999999999993e-06 2024-09-17T22:42:13.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396528+0000", 2024-09-17T22:42:13.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2990000000000002e-06 2024-09-17T22:42:13.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448798+0000", 2024-09-17T22:42:13.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052269985999999997 2024-09-17T22:42:13.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448871+0000", 2024-09-17T22:42:13.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2581000000000002e-05 2024-09-17T22:42:13.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448971+0000", 2024-09-17T22:42:13.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100142 2024-09-17T22:42:13.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.490 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.490 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146391 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.490 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:13.490 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.017892436, 2024-09-17T22:42:13.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052666719000000001, 2024-09-17T22:42:13.490 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.490 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.490 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.490 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.491 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146391 2024-09-17T22:42:13.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.491 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:13.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:13.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404098+0000", 2024-09-17T22:42:13.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000124509 2024-09-17T22:42:13.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404107+0000", 2024-09-17T22:42:13.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4220000000000006e-06 2024-09-17T22:42:13.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404117+0000", 2024-09-17T22:42:13.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011e-05 2024-09-17T22:42:13.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404123+0000", 2024-09-17T22:42:13.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1530000000000002e-06 2024-09-17T22:42:13.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456481+0000", 2024-09-17T22:42:13.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052357923000000001 2024-09-17T22:42:13.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456551+0000", 2024-09-17T22:42:13.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9869000000000005e-05 2024-09-17T22:42:13.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456640+0000", 2024-09-17T22:42:13.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8733000000000006e-05 2024-09-17T22:42:13.494 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.494 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.494 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.494 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146393 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2748416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.495 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:13.495 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.017732373000001, 2024-09-17T22:42:13.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052739239, 2024-09-17T22:42:13.495 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.495 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.495 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.495 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.495 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.495 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.495 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146393 2024-09-17T22:42:13.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.495 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:13.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:13.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404443+0000", 2024-09-17T22:42:13.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030977199999999999 2024-09-17T22:42:13.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404447+0000", 2024-09-17T22:42:13.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6370000000000001e-06 2024-09-17T22:42:13.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404469+0000", 2024-09-17T22:42:13.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2739999999999999e-05 2024-09-17T22:42:13.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404477+0000", 2024-09-17T22:42:13.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6559999999999992e-06 2024-09-17T22:42:13.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456734+0000", 2024-09-17T22:42:13.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052256889000000001 2024-09-17T22:42:13.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456796+0000", 2024-09-17T22:42:13.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2485999999999998e-05 2024-09-17T22:42:13.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456873+0000", 2024-09-17T22:42:13.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6058999999999998e-05 2024-09-17T22:42:13.499 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.499 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.499 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.499 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146397 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2760704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.499 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:13.499 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.017356609, 2024-09-17T22:42:13.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052788928999999998, 2024-09-17T22:42:13.499 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.499 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.500 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.500 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.500 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.500 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.500 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146397 2024-09-17T22:42:13.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.500 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:13.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:13.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404763+0000", 2024-09-17T22:42:13.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025351399999999999 2024-09-17T22:42:13.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404767+0000", 2024-09-17T22:42:13.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:42:13.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404777+0000", 2024-09-17T22:42:13.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9310000000000003e-06 2024-09-17T22:42:13.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404783+0000", 2024-09-17T22:42:13.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.336e-06 2024-09-17T22:42:13.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457126+0000", 2024-09-17T22:42:13.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052342975999999999 2024-09-17T22:42:13.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457200+0000", 2024-09-17T22:42:13.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4146000000000005e-05 2024-09-17T22:42:13.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457298+0000", 2024-09-17T22:42:13.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7807999999999996e-05 2024-09-17T22:42:13.503 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.503 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.504 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.504 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146399 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2766848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.504 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:13.504 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.017066467999999, 2024-09-17T22:42:13.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052829779, 2024-09-17T22:42:13.504 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.504 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.504 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.504 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.504 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.504 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.505 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146399 2024-09-17T22:42:13.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.505 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:13.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:13.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404800+0000", 2024-09-17T22:42:13.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8500000000000001e-07 2024-09-17T22:42:13.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404802+0000", 2024-09-17T22:42:13.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5349999999999999e-06 2024-09-17T22:42:13.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404811+0000", 2024-09-17T22:42:13.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9390000000000003e-06 2024-09-17T22:42:13.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404817+0000", 2024-09-17T22:42:13.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.524e-06 2024-09-17T22:42:13.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457461+0000", 2024-09-17T22:42:13.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052643784999999998 2024-09-17T22:42:13.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457539+0000", 2024-09-17T22:42:13.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8589000000000003e-05 2024-09-17T22:42:13.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457629+0000", 2024-09-17T22:42:13.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9821999999999995e-05 2024-09-17T22:42:13.508 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.508 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.508 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.509 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146401 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2772992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.509 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:13.509 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.017041220999999, 2024-09-17T22:42:13.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052910077, 2024-09-17T22:42:13.509 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.509 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.509 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.509 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.509 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.509 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.509 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146401 2024-09-17T22:42:13.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.510 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:13.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:13.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405106+0000", 2024-09-17T22:42:13.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000281286 2024-09-17T22:42:13.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405110+0000", 2024-09-17T22:42:13.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1459999999999998e-06 2024-09-17T22:42:13.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405120+0000", 2024-09-17T22:42:13.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0546e-05 2024-09-17T22:42:13.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405127+0000", 2024-09-17T22:42:13.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4300000000000003e-06 2024-09-17T22:42:13.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457647+0000", 2024-09-17T22:42:13.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052520229000000002 2024-09-17T22:42:13.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457684+0000", 2024-09-17T22:42:13.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6609999999999997e-05 2024-09-17T22:42:13.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457735+0000", 2024-09-17T22:42:13.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.083e-05 2024-09-17T22:42:13.513 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.513 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.513 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.513 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146402 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2779136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.513 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:13.513 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.016727808999999, 2024-09-17T22:42:13.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052689812000000003, 2024-09-17T22:42:13.514 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.514 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.514 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.514 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.514 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.514 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.514 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146402 2024-09-17T22:42:13.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.514 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:13.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:13.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405365+0000", 2024-09-17T22:42:13.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022744599999999999 2024-09-17T22:42:13.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405370+0000", 2024-09-17T22:42:13.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2769999999999999e-06 2024-09-17T22:42:13.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405380+0000", 2024-09-17T22:42:13.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0349e-05 2024-09-17T22:42:13.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405386+0000", 2024-09-17T22:42:13.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5169999999999996e-06 2024-09-17T22:42:13.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457750+0000", 2024-09-17T22:42:13.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052363448999999999 2024-09-17T22:42:13.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457784+0000", 2024-09-17T22:42:13.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4363999999999998e-05 2024-09-17T22:42:13.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457828+0000", 2024-09-17T22:42:13.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999999e-05 2024-09-17T22:42:13.517 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.518 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.518 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.518 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.518 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.518 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:13.519 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:42:13.520 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:13.517+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:13.620 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:13.618+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:13.663 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:13.663 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:13.663 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:13.663 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:13.672 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:42:13.722 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:13.720+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:13.774 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:13.774 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:13.774 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:13.774 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:13.774 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:13.782 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:42:13.827 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:13.824+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:13.887 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:13.887 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:13.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:13.887 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:13.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.887 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4229.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-17T22:42:13.888 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:13.888 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.489394653000005, 2024-09-17T22:42:13.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25708877200000002, 2024-09-17T22:42:13.888 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.888 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.888 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-17T22:42:13.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4229", 2024-09-17T22:42:13.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/3080427038", 2024-09-17T22:42:13.888 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2 2024-09-17T22:42:13.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.888 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:13.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:13.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394611+0000", 2024-09-17T22:42:13.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3149999999999999e-06 2024-09-17T22:42:13.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394619+0000", 2024-09-17T22:42:13.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240000000000003e-06 2024-09-17T22:42:13.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394632+0000", 2024-09-17T22:42:13.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2921e-05 2024-09-17T22:42:13.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394639+0000", 2024-09-17T22:42:13.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-09-17T22:42:13.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394703+0000", 2024-09-17T22:42:13.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4407999999999997e-05 2024-09-17T22:42:13.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:42:13.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394711+0000", 2024-09-17T22:42:13.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5939999999999996e-06 2024-09-17T22:42:13.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651545+0000", 2024-09-17T22:42:13.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25683409200000001 2024-09-17T22:42:13.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651698+0000", 2024-09-17T22:42:13.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000153678 2024-09-17T22:42:13.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.893 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.893 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.893 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38736 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:13.893 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:13.893 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.106798648000002, 2024-09-17T22:42:13.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051395418999999998, 2024-09-17T22:42:13.893 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.893 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.893 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.894 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.894 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38736 2024-09-17T22:42:13.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.894 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:13.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:13.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:13.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6899999999999998e-07 2024-09-17T22:42:13.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777212+0000", 2024-09-17T22:42:13.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1240000000000002e-06 2024-09-17T22:42:13.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777219+0000", 2024-09-17T22:42:13.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5409999999999997e-06 2024-09-17T22:42:13.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777224+0000", 2024-09-17T22:42:13.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:42:13.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.826131+0000", 2024-09-17T22:42:13.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048907369999999999 2024-09-17T22:42:13.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828457+0000", 2024-09-17T22:42:13.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023256209999999999 2024-09-17T22:42:13.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828601+0000", 2024-09-17T22:42:13.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014438799999999999 2024-09-17T22:42:13.897 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.897 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.897 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.897 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38752 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 673792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:13.898 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:13.898 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.106774577000003, 2024-09-17T22:42:13.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051582301999999997, 2024-09-17T22:42:13.898 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.898 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.898 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.898 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38752 2024-09-17T22:42:13.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.898 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:13.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:13.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:13.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3000000000000001e-07 2024-09-17T22:42:13.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777232+0000", 2024-09-17T22:42:13.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9709999999999998e-06 2024-09-17T22:42:13.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777238+0000", 2024-09-17T22:42:13.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7209999999999999e-06 2024-09-17T22:42:13.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777243+0000", 2024-09-17T22:42:13.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5210000000000004e-06 2024-09-17T22:42:13.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828633+0000", 2024-09-17T22:42:13.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051390915000000002 2024-09-17T22:42:13.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828711+0000", 2024-09-17T22:42:13.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7474999999999999e-05 2024-09-17T22:42:13.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828812+0000", 2024-09-17T22:42:13.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010116900000000001 2024-09-17T22:42:13.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.903 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.903 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38753 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:13.903 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:13.903 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.106755304000004, 2024-09-17T22:42:13.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051770444999999998, 2024-09-17T22:42:13.903 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.904 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.904 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.904 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38753 2024-09-17T22:42:13.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.904 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:13.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:13.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:13.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:42:13.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777251+0000", 2024-09-17T22:42:13.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8059999999999999e-06 2024-09-17T22:42:13.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777257+0000", 2024-09-17T22:42:13.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2480000000000001e-06 2024-09-17T22:42:13.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777262+0000", 2024-09-17T22:42:13.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000004e-06 2024-09-17T22:42:13.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828842+0000", 2024-09-17T22:42:13.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051580203999999998 2024-09-17T22:42:13.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828917+0000", 2024-09-17T22:42:13.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4980000000000004e-05 2024-09-17T22:42:13.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829019+0000", 2024-09-17T22:42:13.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102248 2024-09-17T22:42:13.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.908 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.909 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38766 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:13.909 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:13.909 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.106736318999999, 2024-09-17T22:42:13.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051958839999999999, 2024-09-17T22:42:13.909 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.909 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.910 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.910 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.910 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.910 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.910 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38766 2024-09-17T22:42:13.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.910 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:13.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:13.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777269+0000", 2024-09-17T22:42:13.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:42:13.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777271+0000", 2024-09-17T22:42:13.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-17T22:42:13.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777276+0000", 2024-09-17T22:42:13.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:42:13.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777281+0000", 2024-09-17T22:42:13.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:42:13.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829055+0000", 2024-09-17T22:42:13.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051773570999999997 2024-09-17T22:42:13.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829128+0000", 2024-09-17T22:42:13.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3131000000000002e-05 2024-09-17T22:42:13.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829227+0000", 2024-09-17T22:42:13.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8800000000000003e-05 2024-09-17T22:42:13.914 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.914 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.914 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.914 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38767 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:13.914 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:13.915 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.106716335999998, 2024-09-17T22:42:13.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137018, 2024-09-17T22:42:13.915 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.915 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.915 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.915 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.915 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.915 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.915 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38767 2024-09-17T22:42:13.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.915 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:13.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:13.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:13.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:42:13.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777290+0000", 2024-09-17T22:42:13.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.88e-06 2024-09-17T22:42:13.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777296+0000", 2024-09-17T22:42:13.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-09-17T22:42:13.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777300+0000", 2024-09-17T22:42:13.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4510000000000002e-06 2024-09-17T22:42:13.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829257+0000", 2024-09-17T22:42:13.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051956975000000002 2024-09-17T22:42:13.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829328+0000", 2024-09-17T22:42:13.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1026999999999996e-05 2024-09-17T22:42:13.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829425+0000", 2024-09-17T22:42:13.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.679e-05 2024-09-17T22:42:13.919 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.919 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.919 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.919 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38784 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 675840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:13.920 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:13.920 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.106697881999999, 2024-09-17T22:42:13.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052344522999999997, 2024-09-17T22:42:13.920 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.920 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.920 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.920 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38784 2024-09-17T22:42:13.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.920 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:13.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:13.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777307+0000", 2024-09-17T22:42:13.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:42:13.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777309+0000", 2024-09-17T22:42:13.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.063e-06 2024-09-17T22:42:13.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777314+0000", 2024-09-17T22:42:13.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-17T22:42:13.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777319+0000", 2024-09-17T22:42:13.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999996e-06 2024-09-17T22:42:13.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829456+0000", 2024-09-17T22:42:13.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137151 2024-09-17T22:42:13.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829527+0000", 2024-09-17T22:42:13.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1588000000000006e-05 2024-09-17T22:42:13.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829651+0000", 2024-09-17T22:42:13.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012365 2024-09-17T22:42:13.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.924 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.925 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38785 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 749568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:13.925 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:13.925 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.106679767000003, 2024-09-17T22:42:13.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052546572999999999, 2024-09-17T22:42:13.925 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.925 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.925 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.926 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38785 2024-09-17T22:42:13.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.926 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:13.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:13.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:13.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:42:13.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777327+0000", 2024-09-17T22:42:13.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:42:13.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777333+0000", 2024-09-17T22:42:13.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7309999999999998e-06 2024-09-17T22:42:13.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777337+0000", 2024-09-17T22:42:13.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:42:13.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829686+0000", 2024-09-17T22:42:13.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052349159999999999 2024-09-17T22:42:13.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829766+0000", 2024-09-17T22:42:13.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9326999999999995e-05 2024-09-17T22:42:13.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829871+0000", 2024-09-17T22:42:13.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000105466 2024-09-17T22:42:13.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.932 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.932 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38798 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:13.932 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:13.933 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.106046159000002, 2024-09-17T22:42:13.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052117564999999998, 2024-09-17T22:42:13.933 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.933 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.933 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.934 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38798 2024-09-17T22:42:13.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.934 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:13.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:13.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777959+0000", 2024-09-17T22:42:13.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9599999999999999e-07 2024-09-17T22:42:13.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777963+0000", 2024-09-17T22:42:13.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2660000000000003e-06 2024-09-17T22:42:13.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777971+0000", 2024-09-17T22:42:13.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3489999999999992e-06 2024-09-17T22:42:13.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777978+0000", 2024-09-17T22:42:13.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:42:13.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829906+0000", 2024-09-17T22:42:13.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051927635999999999 2024-09-17T22:42:13.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829986+0000", 2024-09-17T22:42:13.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0444000000000004e-05 2024-09-17T22:42:13.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830076+0000", 2024-09-17T22:42:13.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9473000000000002e-05 2024-09-17T22:42:13.939 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.939 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.939 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.939 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38799 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 882688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:13.939 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:13.939 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.106014719999997, 2024-09-17T22:42:13.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052359606000000003, 2024-09-17T22:42:13.939 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.940 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.940 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.940 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38799 2024-09-17T22:42:13.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.940 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:13.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:13.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:13.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8299999999999997e-07 2024-09-17T22:42:13.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777992+0000", 2024-09-17T22:42:13.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.176e-06 2024-09-17T22:42:13.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777998+0000", 2024-09-17T22:42:13.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.173e-06 2024-09-17T22:42:13.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778003+0000", 2024-09-17T22:42:13.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6349999999999997e-06 2024-09-17T22:42:13.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830112+0000", 2024-09-17T22:42:13.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052109366999999997 2024-09-17T22:42:13.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830192+0000", 2024-09-17T22:42:13.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9814000000000005e-05 2024-09-17T22:42:13.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830349+0000", 2024-09-17T22:42:13.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015695800000000001 2024-09-17T22:42:13.944 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.944 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.944 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.944 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38814 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:13.944 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:13.944 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.105731316000004, 2024-09-17T22:42:13.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052274339000000003, 2024-09-17T22:42:13.945 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.945 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.945 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.945 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38814 2024-09-17T22:42:13.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.945 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:13.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:13.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:13.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:42:13.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778276+0000", 2024-09-17T22:42:13.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2029999999999999e-06 2024-09-17T22:42:13.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778282+0000", 2024-09-17T22:42:13.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0059999999999998e-06 2024-09-17T22:42:13.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778287+0000", 2024-09-17T22:42:13.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0039999999999999e-06 2024-09-17T22:42:13.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830382+0000", 2024-09-17T22:42:13.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052095379999999997 2024-09-17T22:42:13.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830455+0000", 2024-09-17T22:42:13.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3127000000000006e-05 2024-09-17T22:42:13.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830547+0000", 2024-09-17T22:42:13.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2207999999999995e-05 2024-09-17T22:42:13.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.949 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.949 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38828 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 163840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:13.949 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:13.949 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.105548908000003, 2024-09-17T22:42:13.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052466039999999999, 2024-09-17T22:42:13.949 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.949 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.950 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.950 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38828 2024-09-17T22:42:13.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.950 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:13.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:13.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778456+0000", 2024-09-17T22:42:13.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7599999999999996e-07 2024-09-17T22:42:13.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778460+0000", 2024-09-17T22:42:13.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8229999999999997e-06 2024-09-17T22:42:13.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778470+0000", 2024-09-17T22:42:13.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8199999999999992e-06 2024-09-17T22:42:13.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778476+0000", 2024-09-17T22:42:13.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2589999999999997e-06 2024-09-17T22:42:13.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830578+0000", 2024-09-17T22:42:13.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052102273999999997 2024-09-17T22:42:13.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830667+0000", 2024-09-17T22:42:13.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8701000000000003e-05 2024-09-17T22:42:13.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830921+0000", 2024-09-17T22:42:13.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025438700000000002 2024-09-17T22:42:13.954 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.954 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.954 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.954 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38829 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:13.954 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:13.954 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.105517511000002, 2024-09-17T22:42:13.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052648882000000001, 2024-09-17T22:42:13.954 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.954 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.955 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.955 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.955 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.955 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.955 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38829 2024-09-17T22:42:13.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.955 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:13.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:13.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:13.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7e-07 2024-09-17T22:42:13.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778490+0000", 2024-09-17T22:42:13.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:42:13.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778495+0000", 2024-09-17T22:42:13.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9329999999999998e-06 2024-09-17T22:42:13.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778500+0000", 2024-09-17T22:42:13.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9509999999999997e-06 2024-09-17T22:42:13.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830958+0000", 2024-09-17T22:42:13.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05245785 2024-09-17T22:42:13.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831051+0000", 2024-09-17T22:42:13.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2989e-05 2024-09-17T22:42:13.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831136+0000", 2024-09-17T22:42:13.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4448000000000002e-05 2024-09-17T22:42:13.959 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.959 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.959 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.959 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38891 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:13.959 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:13.959 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.102205453000003, 2024-09-17T22:42:13.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050847934999999997, 2024-09-17T22:42:13.959 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.959 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.960 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.960 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.960 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.960 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.960 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38891 2024-09-17T22:42:13.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.960 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:13.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:13.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:13.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-09-17T22:42:13.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781802+0000", 2024-09-17T22:42:13.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:42:13.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781810+0000", 2024-09-17T22:42:13.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8019999999999992e-06 2024-09-17T22:42:13.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781816+0000", 2024-09-17T22:42:13.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.285e-06 2024-09-17T22:42:13.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832485+0000", 2024-09-17T22:42:13.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050668462999999997 2024-09-17T22:42:13.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832565+0000", 2024-09-17T22:42:13.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9692999999999997e-05 2024-09-17T22:42:13.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832647+0000", 2024-09-17T22:42:13.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2224000000000004e-05 2024-09-17T22:42:13.964 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.964 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.964 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.964 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38892 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:13.964 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:13.965 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.102177204, 2024-09-17T22:42:13.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050946120999999997, 2024-09-17T22:42:13.965 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.965 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.965 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.965 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.965 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.965 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.965 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38892 2024-09-17T22:42:13.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.965 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:13.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:13.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781828+0000", 2024-09-17T22:42:13.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5400000000000002e-07 2024-09-17T22:42:13.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781830+0000", 2024-09-17T22:42:13.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1959999999999998e-06 2024-09-17T22:42:13.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781837+0000", 2024-09-17T22:42:13.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8310000000000003e-06 2024-09-17T22:42:13.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781843+0000", 2024-09-17T22:42:13.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1e-06 2024-09-17T22:42:13.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832674+0000", 2024-09-17T22:42:13.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050831182000000003 2024-09-17T22:42:13.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832725+0000", 2024-09-17T22:42:13.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1588000000000001e-05 2024-09-17T22:42:13.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832773+0000", 2024-09-17T22:42:13.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7769999999999998e-05 2024-09-17T22:42:13.969 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.969 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.969 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.969 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38909 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 36864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:13.969 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:13.969 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.102154433999999, 2024-09-17T22:42:13.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051046518999999999, 2024-09-17T22:42:13.969 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.969 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.970 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.970 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.970 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.970 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.970 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38909 2024-09-17T22:42:13.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.970 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:13.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:13.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:13.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:42:13.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781852+0000", 2024-09-17T22:42:13.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0590000000000001e-06 2024-09-17T22:42:13.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781859+0000", 2024-09-17T22:42:13.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3380000000000001e-06 2024-09-17T22:42:13.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781864+0000", 2024-09-17T22:42:13.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:42:13.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832804+0000", 2024-09-17T22:42:13.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050939614000000001 2024-09-17T22:42:13.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832845+0000", 2024-09-17T22:42:13.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1014999999999999e-05 2024-09-17T22:42:13.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832896+0000", 2024-09-17T22:42:13.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1836999999999999e-05 2024-09-17T22:42:13.974 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.974 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.974 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.974 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38910 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:13.974 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:13.974 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.102134124000003, 2024-09-17T22:42:13.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051216818999999997, 2024-09-17T22:42:13.974 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.974 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.974 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.975 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38910 2024-09-17T22:42:13.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.975 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:13.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:13.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781871+0000", 2024-09-17T22:42:13.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:42:13.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781873+0000", 2024-09-17T22:42:13.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:42:13.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781879+0000", 2024-09-17T22:42:13.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0700000000000003e-06 2024-09-17T22:42:13.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781884+0000", 2024-09-17T22:42:13.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-09-17T22:42:13.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832932+0000", 2024-09-17T22:42:13.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051048219999999998 2024-09-17T22:42:13.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833001+0000", 2024-09-17T22:42:13.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8504999999999995e-05 2024-09-17T22:42:13.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833087+0000", 2024-09-17T22:42:13.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6464000000000004e-05 2024-09-17T22:42:13.978 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.978 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.978 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.979 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38923 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:13.979 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:13.979 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.102113803999998, 2024-09-17T22:42:13.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051379573999999997, 2024-09-17T22:42:13.979 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.979 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.979 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.979 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38923 2024-09-17T22:42:13.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.980 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:13.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:13.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:13.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:42:13.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781893+0000", 2024-09-17T22:42:13.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-09-17T22:42:13.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781900+0000", 2024-09-17T22:42:13.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3860000000000004e-06 2024-09-17T22:42:13.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781904+0000", 2024-09-17T22:42:13.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9300000000000002e-06 2024-09-17T22:42:13.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833115+0000", 2024-09-17T22:42:13.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051210845999999997 2024-09-17T22:42:13.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833188+0000", 2024-09-17T22:42:13.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2276999999999999e-05 2024-09-17T22:42:13.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833270+0000", 2024-09-17T22:42:13.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2501000000000002e-05 2024-09-17T22:42:13.983 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.983 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.984 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38941 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:13.984 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:13.984 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 44.101321269000003, 2024-09-17T22:42:13.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050969459000000002, 2024-09-17T22:42:13.984 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.984 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.984 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.984 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.984 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.984 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.984 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38941 2024-09-17T22:42:13.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.985 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:13.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:13.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782684+0000", 2024-09-17T22:42:13.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:42:13.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782686+0000", 2024-09-17T22:42:13.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.396e-06 2024-09-17T22:42:13.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782693+0000", 2024-09-17T22:42:13.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1999999999999997e-06 2024-09-17T22:42:13.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782699+0000", 2024-09-17T22:42:13.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7910000000000001e-06 2024-09-17T22:42:13.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833471+0000", 2024-09-17T22:42:13.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050772333000000003 2024-09-17T22:42:13.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833536+0000", 2024-09-17T22:42:13.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4380000000000004e-05 2024-09-17T22:42:13.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833653+0000", 2024-09-17T22:42:13.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011691799999999999 2024-09-17T22:42:13.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.988 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.988 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:151189 3.14 3:2b696b13:::rbd_data.115f14f2d1d5.0000000000000009:e [read 3127296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:13.989 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:13.989 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.171100524, 2024-09-17T22:42:13.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.054254700000000003, 2024-09-17T22:42:13.989 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:13.989 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:13.989 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:13.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:13.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:13.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:13.989 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 151189 2024-09-17T22:42:13.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.989 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:13.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:13.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:13.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:13.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:13.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:13.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:13.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713051+0000", 2024-09-17T22:42:13.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014722100000000001 2024-09-17T22:42:13.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:13.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713055+0000", 2024-09-17T22:42:13.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.16e-06 2024-09-17T22:42:13.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:13.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713065+0000", 2024-09-17T22:42:13.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8390000000000002e-06 2024-09-17T22:42:13.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:13.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713073+0000", 2024-09-17T22:42:13.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0390000000000005e-06 2024-09-17T22:42:13.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:13.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762644+0000", 2024-09-17T22:42:13.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049571160000000003 2024-09-17T22:42:13.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:13.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762739+0000", 2024-09-17T22:42:13.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4565e-05 2024-09-17T22:42:13.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:13.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:13.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:13.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.767159+0000", 2024-09-17T22:42:13.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044197159999999997 2024-09-17T22:42:13.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.993 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:13.993 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:13.993 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:13.994 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:42:13.995 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:42:13.996 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:13.992+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:14.096 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:14.094+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:14.110 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:14.110 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:14.110 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:14.110 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:14.120 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:42:14.138 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:14.154 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:14.154 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:14.170 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:14.171 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:14.186 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:14.187 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:14.203 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:14.203 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:14.204 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:14.201+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:14.221 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:14.222 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:14.238 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:14.238 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:14.255 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:14.255 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:14.258 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:14.259 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:14.270 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:14.270 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:14.279 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:42:14.293 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone27 from 0xacb3a95 (overlap 0x2b7909e) to 0x1a156c5 2024-09-17T22:42:14.297 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:14.295+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:14.398 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:14.395+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:14.414 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-17T22:42:14.421 INFO:teuthology.orchestra.run.smithi115.stdout:3017 punch from 0x88da822 to 0x88dc865, (0x2043 bytes) 2024-09-17T22:42:14.425 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:14.425 INFO:teuthology.orchestra.run.smithi115.stdout:3019 read 0x6bf20ef thru 0x6c01330 (0xf242 bytes) 2024-09-17T22:42:14.426 INFO:teuthology.orchestra.run.smithi115.stdout:3020 punch from 0x7b00f17 to 0x7b1033a, (0xf423 bytes) 2024-09-17T22:42:14.428 INFO:teuthology.orchestra.run.smithi115.stdout:3021 trunc from 0xacb3a95 to 0xc79217d 2024-09-17T22:42:14.432 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:14.433 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:14.433 INFO:teuthology.orchestra.run.smithi115.stdout:3024 read 0x607de60 thru 0x608daa9 (0xfc4a bytes) 2024-09-17T22:42:14.434 INFO:teuthology.orchestra.run.smithi115.stdout:3025 punch from 0x1025e9 to 0x10fab9, (0xd4d0 bytes) 2024-09-17T22:42:14.442 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:42:14.446 INFO:teuthology.orchestra.run.smithi115.stdout:3027 trunc from 0xc79217d to 0x22f090d 2024-09-17T22:42:14.477 INFO:teuthology.orchestra.run.smithi115.stdout:3028 write 0x69d73c5 thru 0x69e2783 (0xb3bf bytes) 2024-09-17T22:42:14.493 INFO:teuthology.orchestra.run.smithi115.stdout:3030 write 0x9c129e5 thru 0x9c17506 (0x4b22 bytes) 2024-09-17T22:42:14.499 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:14.496+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:14.500 INFO:teuthology.orchestra.run.smithi115.stdout:3032 read 0x6228d4a thru 0x6236e44 (0xe0fb bytes) 2024-09-17T22:42:14.505 INFO:teuthology.orchestra.run.smithi115.stdout:3033 trunc from 0x9c17507 to 0x59273b4 2024-09-17T22:42:14.516 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:14.524 INFO:teuthology.orchestra.run.smithi115.stdout:3037 trunc from 0x59273b4 to 0xcce33fc 2024-09-17T22:42:14.530 INFO:teuthology.orchestra.run.smithi115.stdout:3038 writesame 0xa9c0b00 thru 0xa9c5328 data_size 0x31(0x4829 bytes) 2024-09-17T22:42:14.548 INFO:teuthology.orchestra.run.smithi115.stdout:3039 trunc from 0xcce33fc to 0x15dd547 2024-09-17T22:42:14.570 INFO:teuthology.orchestra.run.smithi115.stdout:3041 trunc from 0x15dd547 to 0x40cf995 2024-09-17T22:42:14.576 INFO:teuthology.orchestra.run.smithi115.stdout:3042 read 0x38d4f4d thru 0x38d7e83 (0x2f37 bytes) 2024-09-17T22:42:14.576 INFO:teuthology.orchestra.run.smithi115.stdout:3043 punch from 0x2132c7b to 0x213b7b4, (0x8b39 bytes) 2024-09-17T22:42:14.593 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:14.593 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:14.599 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:14.597+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:14.602 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:42:14.700 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:14.698+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:14.704 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:14.705 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:14.713 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:42:14.785 INFO:teuthology.orchestra.run.smithi115.stdout:3044 writesame 0x4b719ec thru 0x4b7a910 data_size 0x15(0x8f25 bytes) 2024-09-17T22:42:14.785 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:14.801 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:14.798+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:14.802 INFO:teuthology.orchestra.run.smithi115.stdout:3048 write 0xdfb97e0 thru 0xdfba122 (0x943 bytes) 2024-09-17T22:42:14.810 INFO:teuthology.orchestra.run.smithi115.stdout:3049 trunc from 0xdfba123 to 0x8f982bf 2024-09-17T22:42:14.821 INFO:teuthology.orchestra.run.smithi115.stdout:3050 punch from 0x29f6bdb to 0x2a02fd9, (0xc3fe bytes) 2024-09-17T22:42:14.821 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:14.821 INFO:teuthology.orchestra.run.smithi115.stdout:3052 read 0x4ac932a thru 0x4ad8631 (0xf308 bytes) 2024-09-17T22:42:14.821 INFO:teuthology.orchestra.run.smithi115.stdout:3053 read 0x59b8f13 thru 0x59be2ce (0x53bc bytes) 2024-09-17T22:42:14.821 INFO:teuthology.orchestra.run.smithi115.stdout:3055 read 0x2b2f9 thru 0x38a7c (0xd784 bytes) 2024-09-17T22:42:14.833 INFO:teuthology.orchestra.run.smithi115.stdout:3056 writesame 0xd764799 thru 0xd764a0c data_size 0x2(0x274 bytes) 2024-09-17T22:42:14.833 INFO:teuthology.orchestra.run.smithi115.stdout:3057 punch from 0x1e876f3 to 0x1e8f706, (0x8013 bytes) 2024-09-17T22:42:14.833 INFO:teuthology.orchestra.run.smithi115.stdout:3058 read 0xcced53 thru 0xcdcebc (0xe16a bytes) 2024-09-17T22:42:14.834 INFO:teuthology.orchestra.run.smithi115.stdout:3060 punch from 0x52b5fb9 to 0x52b789e, (0x18e5 bytes) 2024-09-17T22:42:14.834 INFO:teuthology.orchestra.run.smithi115.stdout:3064 punch from 0x9fafbb7 to 0x9fb2397, (0x27e0 bytes) 2024-09-17T22:42:14.834 INFO:teuthology.orchestra.run.smithi115.stdout:3065 write 0xc573a11 thru 0xc581f31 (0xe521 bytes) 2024-09-17T22:42:14.839 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:14.839 INFO:teuthology.orchestra.run.smithi115.stdout:3067 punch from 0xafeaa31 to 0xaff7aa2, (0xd071 bytes) 2024-09-17T22:42:14.839 INFO:teuthology.orchestra.run.smithi115.stdout:3070 punch from 0x43562ca to 0x435d0ba, (0x6df0 bytes) 2024-09-17T22:42:14.839 INFO:teuthology.orchestra.run.smithi115.stdout:3071 punch from 0x84f8120 to 0x84fae14, (0x2cf4 bytes) 2024-09-17T22:42:14.839 INFO:teuthology.orchestra.run.smithi115.stdout:3072 read 0xa61892b thru 0xa621dc0 (0x9496 bytes) 2024-09-17T22:42:14.840 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:14.840 INFO:teuthology.orchestra.run.smithi115.stdout:3076 read 0xc565ede thru 0xc56ddb0 (0x7ed3 bytes) 2024-09-17T22:42:14.840 INFO:teuthology.orchestra.run.smithi115.stdout:3077 read 0x3196da thru 0x322fea (0x9911 bytes) 2024-09-17T22:42:14.840 INFO:teuthology.orchestra.run.smithi115.stdout:3078 write 0x5c6b767 thru 0x5c745fc (0x8e96 bytes) 2024-09-17T22:42:14.845 INFO:teuthology.orchestra.run.smithi115.stdout:3079 write 0x3dc8621 thru 0x3dd3070 (0xaa50 bytes) 2024-09-17T22:42:14.850 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:14.850 INFO:teuthology.orchestra.run.smithi115.stdout:3081 punch from 0x556d4b7 to 0x557b9f2, (0xe53b bytes) 2024-09-17T22:42:14.850 INFO:teuthology.orchestra.run.smithi115.stdout:3082 writesame 0x9c5330b thru 0x9c5b4bf data_size 0x5(0x81b5 bytes) 2024-09-17T22:42:14.856 INFO:teuthology.orchestra.run.smithi115.stdout:3084 writesame 0x731280e thru 0x73194cf data_size 0x1(0x6cc2 bytes) 2024-09-17T22:42:14.866 INFO:teuthology.orchestra.run.smithi115.stdout:3085 write 0xd4673f5 thru 0xd469783 (0x238f bytes) 2024-09-17T22:42:14.869 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:14.869 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:14.870 INFO:teuthology.orchestra.run.smithi115.stdout:3086 read 0x5acee10 thru 0x5ad7200 (0x83f1 bytes) 2024-09-17T22:42:14.870 INFO:teuthology.orchestra.run.smithi115.stdout:3088 writesame 0x6d44517 thru 0x6d4c5c1 data_size 0x1(0x80ab bytes) 2024-09-17T22:42:14.880 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:42:14.880 INFO:teuthology.orchestra.run.smithi115.stdout:3089 punch from 0xbffa07f to 0xc007d66, (0xdce7 bytes) 2024-09-17T22:42:14.881 INFO:teuthology.orchestra.run.smithi115.stdout:3090 write 0x1ba8870 thru 0x1bb444b (0xbbdc bytes) 2024-09-17T22:42:14.886 INFO:teuthology.orchestra.run.smithi115.stdout:3091 write 0x60e9989 thru 0x60f2561 (0x8bd9 bytes) 2024-09-17T22:42:14.896 INFO:teuthology.orchestra.run.smithi115.stdout:3092 write 0x14032d7 thru 0x1407d13 (0x4a3d bytes) 2024-09-17T22:42:14.902 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:14.900+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:14.904 INFO:teuthology.orchestra.run.smithi115.stdout:3093 read 0x9d2b7b3 thru 0x9d3ad93 (0xf5e1 bytes) 2024-09-17T22:42:14.907 INFO:teuthology.orchestra.run.smithi115.stdout:3094 trunc from 0xd764a0d to 0xc0fc693 2024-09-17T22:42:14.918 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:14.918 INFO:teuthology.orchestra.run.smithi115.stdout:3096 write 0x540b1 thru 0x574f8 (0x3448 bytes) 2024-09-17T22:42:14.925 INFO:teuthology.orchestra.run.smithi115.stdout:3098 write 0x1b6f812 thru 0x1b7f7a5 (0xff94 bytes) 2024-09-17T22:42:14.927 INFO:teuthology.orchestra.run.smithi115.stdout:3101 writesame 0x78e7713 thru 0x78e9128 data_size 0xe(0x1a16 bytes) 2024-09-17T22:42:14.930 INFO:teuthology.orchestra.run.smithi115.stdout:3103 read 0xf3f06d thru 0xf49f48 (0xaedc bytes) 2024-09-17T22:42:14.931 INFO:teuthology.orchestra.run.smithi115.stdout:3105 read 0x57842b thru 0x57bf24 (0x3afa bytes) 2024-09-17T22:42:14.933 INFO:teuthology.orchestra.run.smithi115.stdout:3106 trunc from 0xc0fc693 to 0xb26add5 2024-09-17T22:42:14.939 INFO:teuthology.orchestra.run.smithi115.stdout:3108 read 0x1108934 thru 0x110926c (0x939 bytes) 2024-09-17T22:42:14.948 INFO:teuthology.orchestra.run.smithi115.stdout:3109 write 0xd2b8af6 thru 0xd2c4f0a (0xc415 bytes) 2024-09-17T22:42:14.963 INFO:teuthology.orchestra.run.smithi115.stdout:3110 write 0xd9bc246 thru 0xd9c79e2 (0xb79d bytes) 2024-09-17T22:42:14.971 INFO:teuthology.orchestra.run.smithi115.stdout:3112 write 0x2f96c35 thru 0x2fa1e60 (0xb22c bytes) 2024-09-17T22:42:14.980 INFO:teuthology.orchestra.run.smithi115.stdout:3114 read 0xb3c761a thru 0xb3cd2ba (0x5ca1 bytes) 2024-09-17T22:42:14.980 INFO:teuthology.orchestra.run.smithi115.stdout:3115 write 0xd0c353a thru 0xd0c6437 (0x2efe bytes) 2024-09-17T22:42:14.985 INFO:teuthology.orchestra.run.smithi115.stdout:3116 punch from 0x7733453 to 0x773b26d, (0x7e1a bytes) 2024-09-17T22:42:14.985 INFO:teuthology.orchestra.run.smithi115.stdout:3119 write 0xbbbf3b2 thru 0xbbbfa59 (0x6a8 bytes) 2024-09-17T22:42:14.988 INFO:teuthology.orchestra.run.smithi115.stdout:3120 punch from 0xbc2e51d to 0xbc30deb, (0x28ce bytes) 2024-09-17T22:42:14.988 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:14.988 INFO:teuthology.orchestra.run.smithi115.stdout:3122 punch from 0x121177d to 0x1220c7f, (0xf502 bytes) 2024-09-17T22:42:14.988 INFO:teuthology.orchestra.run.smithi115.stdout:3123 writesame 0x9a4386e thru 0x9a535c9 data_size 0xbc(0xfd5c bytes) 2024-09-17T22:42:14.992 INFO:teuthology.orchestra.run.smithi115.stdout:3124 write 0x5e8ea4a thru 0x5e99dcd (0xb384 bytes) 2024-09-17T22:42:14.996 INFO:teuthology.orchestra.run.smithi115.stdout:3125 writesame 0x9ceeca4 thru 0x9cf7a32 data_size 0x7(0x8d8f bytes) 2024-09-17T22:42:15.000 INFO:teuthology.orchestra.run.smithi115.stdout:3126 writesame 0x778311 thru 0x77bc87 data_size 0x1(0x3977 bytes) 2024-09-17T22:42:15.003 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:15.000+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:15.009 INFO:teuthology.orchestra.run.smithi115.stdout:3127 read 0x3f23b40 thru 0x3f2d0b0 (0x9571 bytes) 2024-09-17T22:42:15.009 INFO:teuthology.orchestra.run.smithi115.stdout:3128 write 0x43bbe62 thru 0x43bc29d (0x43c bytes) 2024-09-17T22:42:15.012 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:15.015 INFO:teuthology.orchestra.run.smithi115.stdout:3130 trunc from 0xd9c79e3 to 0xb31c023 2024-09-17T22:42:15.025 INFO:teuthology.orchestra.run.smithi115.stdout:3131 write 0x8888b70 thru 0x889782b (0xecbc bytes) 2024-09-17T22:42:15.038 INFO:teuthology.orchestra.run.smithi115.stdout:3133 writesame 0xcc01409 thru 0xcc04301 data_size 0x19(0x2ef9 bytes) 2024-09-17T22:42:15.039 INFO:teuthology.orchestra.run.smithi115.stdout:3134 read 0x6754396 thru 0x67577a9 (0x3414 bytes) 2024-09-17T22:42:15.039 INFO:teuthology.orchestra.run.smithi115.stdout:3135 writesame 0x7668a46 thru 0x7668f25 data_size 0x10(0x4e0 bytes) 2024-09-17T22:42:15.039 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:15.040 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:15.041 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:15.049 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:42:15.056 INFO:teuthology.orchestra.run.smithi115.stdout:3137 trunc from 0xcc04302 to 0x7b2804 2024-09-17T22:42:15.096 INFO:teuthology.orchestra.run.smithi115.stdout:3138 write 0x78451d6 thru 0x7848799 (0x35c4 bytes) 2024-09-17T22:42:15.103 INFO:teuthology.orchestra.run.smithi115.stdout:3140 read 0x2d1a181 thru 0x2d1beef (0x1d6f bytes) 2024-09-17T22:42:15.104 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:15.101+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:15.115 INFO:teuthology.orchestra.run.smithi115.stdout:3143 write 0xc243f6d thru 0xc244a23 (0xab7 bytes) 2024-09-17T22:42:15.122 INFO:teuthology.orchestra.run.smithi115.stdout:3144 read 0x78db668 thru 0x78e9213 (0xdbac bytes) 2024-09-17T22:42:15.123 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:15.123 INFO:teuthology.orchestra.run.smithi115.stdout:3146 punch from 0x345d043 to 0x346271a, (0x56d7 bytes) 2024-09-17T22:42:15.123 INFO:teuthology.orchestra.run.smithi115.stdout:3147 punch from 0x40cf010 to 0x40d0e0e, (0x1dfe bytes) 2024-09-17T22:42:15.123 INFO:teuthology.orchestra.run.smithi115.stdout:3148 read 0xbaf0345 thru 0xbafd615 (0xd2d1 bytes) 2024-09-17T22:42:15.123 INFO:teuthology.orchestra.run.smithi115.stdout:3149 write 0xa4c0482 thru 0xa4c541e (0x4f9d bytes) 2024-09-17T22:42:15.128 INFO:teuthology.orchestra.run.smithi115.stdout:3150 punch from 0x3ac3ffd to 0x3ac4729, (0x72c bytes) 2024-09-17T22:42:15.128 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:15.128 INFO:teuthology.orchestra.run.smithi115.stdout:3152 write 0x7d08b31 thru 0x7d0953a (0xa0a bytes) 2024-09-17T22:42:15.131 INFO:teuthology.orchestra.run.smithi115.stdout:3154 read 0xb6fbb3e thru 0xb708af6 (0xcfb9 bytes) 2024-09-17T22:42:15.140 INFO:teuthology.orchestra.run.smithi115.stdout:3155 trunc from 0xc244a24 to 0x420a152 2024-09-17T22:42:15.170 INFO:teuthology.orchestra.run.smithi115.stdout:3156 writesame 0xc70a9e0 thru 0xc716e44 data_size 0x1(0xc465 bytes) 2024-09-17T22:42:15.170 INFO:teuthology.orchestra.run.smithi115.stdout:3158 read 0x3bc9d1d thru 0x3bcee73 (0x5157 bytes) 2024-09-17T22:42:15.170 INFO:teuthology.orchestra.run.smithi115.stdout:3159 writesame 0x63538c3 thru 0x635ccc5 data_size 0x1(0x9403 bytes) 2024-09-17T22:42:15.189 INFO:teuthology.orchestra.run.smithi115.stdout:3160 read 0x128cea9 thru 0x1297663 (0xa7bb bytes) 2024-09-17T22:42:15.194 INFO:teuthology.orchestra.run.smithi115.stdout:3162 trunc from 0xc716e45 to 0x8846f49 2024-09-17T22:42:15.201 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:15.201 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:15.203 INFO:teuthology.orchestra.run.smithi115.stdout:3163 trunc from 0x8846f49 to 0x652b277 2024-09-17T22:42:15.205 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:15.202+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:15.211 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:42:15.223 INFO:teuthology.orchestra.run.smithi115.stdout:3164 write 0xdfff78d thru 0xe0012e2 (0x1b56 bytes) 2024-09-17T22:42:15.226 INFO:teuthology.orchestra.run.smithi115.stdout:3165 read 0x7fe69cd thru 0x7ff672f (0xfd63 bytes) 2024-09-17T22:42:15.226 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:42:15.240 INFO:teuthology.orchestra.run.smithi115.stdout:3166 trunc from 0xe0012e3 to 0x280acb5 2024-09-17T22:42:15.249 INFO:teuthology.orchestra.run.smithi115.stdout:3167 write 0x248bfe thru 0x24df24 (0x5327 bytes) 2024-09-17T22:42:15.261 INFO:teuthology.orchestra.run.smithi115.stdout:3170 punch from 0x521f4a to 0x5294af, (0x7565 bytes) 2024-09-17T22:42:15.261 INFO:teuthology.orchestra.run.smithi115.stdout:3171 punch from 0x21ce504 to 0x21de43a, (0xff36 bytes) 2024-09-17T22:42:15.269 INFO:teuthology.orchestra.run.smithi115.stdout:3172 write 0x525d891 thru 0x52607e5 (0x2f55 bytes) 2024-09-17T22:42:15.274 INFO:teuthology.orchestra.run.smithi115.stdout:3173 trunc from 0x52607e6 to 0x5e1dd9a 2024-09-17T22:42:15.280 INFO:teuthology.orchestra.run.smithi115.stdout:3174 read 0x2d833a3 thru 0x2d8ad65 (0x79c3 bytes) 2024-09-17T22:42:15.280 INFO:teuthology.orchestra.run.smithi115.stdout:3176 punch from 0x4d6019 to 0x4e1846, (0xb82d bytes) 2024-09-17T22:42:15.280 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:15.280 INFO:teuthology.orchestra.run.smithi115.stdout:3179 write 0x38a291e thru 0x38b1000 (0xe6e3 bytes) 2024-09-17T22:42:15.290 INFO:teuthology.orchestra.run.smithi115.stdout:3180 trunc from 0x5e1dd9a to 0x1cb896c 2024-09-17T22:42:15.305 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:15.303+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:15.312 INFO:teuthology.orchestra.run.smithi115.stdout:3181 writesame 0x6f40512 thru 0x6f483dc data_size 0x1(0x7ecb bytes) 2024-09-17T22:42:15.312 INFO:teuthology.orchestra.run.smithi115.stdout:3183 read 0x19b72c3 thru 0x19c0a05 (0x9743 bytes) 2024-09-17T22:42:15.312 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:15.312 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:15.314 INFO:teuthology.orchestra.run.smithi115.stdout:3188 trunc from 0x6f483dd to 0x5634030 2024-09-17T22:42:15.325 INFO:teuthology.orchestra.run.smithi115.stdout:3190 write 0x63156c1 thru 0x632466e (0xefae bytes) 2024-09-17T22:42:15.330 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:15.330 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:15.339 INFO:teuthology.orchestra.run.smithi115.stdout:3194 write 0x98e5678 thru 0x98f0fd4 (0xb95d bytes) 2024-09-17T22:42:15.349 INFO:teuthology.orchestra.run.smithi115.stdout:3195 read 0x7c53c83 thru 0x7c5d65f (0x99dd bytes) 2024-09-17T22:42:15.355 INFO:teuthology.orchestra.run.smithi115.stdout:3196 trunc from 0x98f0fd5 to 0x50ada11 2024-09-17T22:42:15.370 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:15.370 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:15.376 INFO:teuthology.orchestra.run.smithi115.stdout:3197 write 0x6c564cc thru 0x6c5d73b (0x7270 bytes) 2024-09-17T22:42:15.380 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:42:15.397 INFO:teuthology.orchestra.run.smithi115.stdout:3198 write 0xc3d978e thru 0xc3e0de5 (0x7658 bytes) 2024-09-17T22:42:15.405 INFO:teuthology.orchestra.run.smithi115.stdout:3199 read 0x2188463 thru 0x218e775 (0x6313 bytes) 2024-09-17T22:42:15.406 INFO:teuthology.orchestra.run.smithi115.stdout:3200 read 0x83ef282 thru 0x83fc3bb (0xd13a bytes) 2024-09-17T22:42:15.406 INFO:teuthology.orchestra.run.smithi115.stdout:3201 punch from 0xee366e to 0xeeb075, (0x7a07 bytes) 2024-09-17T22:42:15.406 INFO:teuthology.orchestra.run.smithi115.stdout:3202 writesame 0x28f34ba thru 0x2901d37 data_size 0x1(0xe87e bytes) 2024-09-17T22:42:15.420 INFO:teuthology.orchestra.run.smithi115.stdout:3205 writesame 0xc319b52 thru 0xc31bebc data_size 0x1(0x236b bytes) 2024-09-17T22:42:15.424 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:15.422+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:15.425 INFO:teuthology.orchestra.run.smithi115.stdout:3206 read 0x1fdbc55 thru 0x1fe4605 (0x89b1 bytes) 2024-09-17T22:42:15.425 INFO:teuthology.orchestra.run.smithi115.stdout:3207 read 0x1403b3b thru 0x140850c (0x49d2 bytes) 2024-09-17T22:42:15.425 INFO:teuthology.orchestra.run.smithi115.stdout:3209 write 0x437cd93 thru 0x438335e (0x65cc bytes) 2024-09-17T22:42:15.430 INFO:teuthology.orchestra.run.smithi115.stdout:3212 read 0x1193947 thru 0x119e05f (0xa719 bytes) 2024-09-17T22:42:15.430 INFO:teuthology.orchestra.run.smithi115.stdout:3214 read 0x32b9484 thru 0x32ba783 (0x1300 bytes) 2024-09-17T22:42:15.438 INFO:teuthology.orchestra.run.smithi115.stdout:3216 write 0xe40a288 thru 0xe411118 (0x6e91 bytes) 2024-09-17T22:42:15.446 INFO:teuthology.orchestra.run.smithi115.stdout:3217 write 0x91eeccb thru 0x91f0c07 (0x1f3d bytes) 2024-09-17T22:42:15.451 INFO:teuthology.orchestra.run.smithi115.stdout:3218 write 0x328be2 thru 0x32b100 (0x251f bytes) 2024-09-17T22:42:15.459 INFO:teuthology.orchestra.run.smithi115.stdout:3219 read 0xe2add20 thru 0xe2b496e (0x6c4f bytes) 2024-09-17T22:42:15.459 INFO:teuthology.orchestra.run.smithi115.stdout:3220 read 0xda3bcce thru 0xda40fcc (0x52ff bytes) 2024-09-17T22:42:15.461 INFO:teuthology.orchestra.run.smithi115.stdout:3221 trunc from 0xe411119 to 0xc41a87e 2024-09-17T22:42:15.481 INFO:teuthology.orchestra.run.smithi115.stdout:3222 writesame 0xdd15955 thru 0xdd247f0 data_size 0x2(0xee9c bytes) 2024-09-17T22:42:15.481 INFO:teuthology.orchestra.run.smithi115.stdout:3223 write 0x3683db6 thru 0x368ca93 (0x8cde bytes) 2024-09-17T22:42:15.488 INFO:teuthology.orchestra.run.smithi115.stdout:3224 punch from 0x2d77fad to 0x2d84c77, (0xccca bytes) 2024-09-17T22:42:15.488 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:15.488 INFO:teuthology.orchestra.run.smithi115.stdout:3227 read 0x7e00fef thru 0x7e0b624 (0xa636 bytes) 2024-09-17T22:42:15.488 INFO:teuthology.orchestra.run.smithi115.stdout:3230 read 0xd152144 thru 0xd15fd9a (0xdc57 bytes) 2024-09-17T22:42:15.488 INFO:teuthology.orchestra.run.smithi115.stdout:3231 punch from 0x9995277 to 0x99a10ee, (0xbe77 bytes) 2024-09-17T22:42:15.496 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:15.496 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:15.499 INFO:teuthology.orchestra.run.smithi115.stdout:3232 trunc from 0xdd247f1 to 0x3fa9e12 2024-09-17T22:42:15.505 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:42:15.510 INFO:teuthology.orchestra.run.smithi115.stdout:3233 punch from 0x243161d to 0x243ceae, (0xb891 bytes) 2024-09-17T22:42:15.515 INFO:teuthology.orchestra.run.smithi115.stdout:3235 write 0x44dc045 thru 0x44e7f0d (0xbec9 bytes) 2024-09-17T22:42:15.520 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:15.520 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:15.520 INFO:teuthology.orchestra.run.smithi115.stdout:3240 write 0x2e00c71 thru 0x2e09713 (0x8aa3 bytes) 2024-09-17T22:42:15.524 INFO:teuthology.orchestra.run.smithi115.stdout:3241 read 0x1609571 thru 0x160d00d (0x3a9d bytes) 2024-09-17T22:42:15.524 INFO:teuthology.orchestra.run.smithi115.stdout:3242 write 0x42c7ce thru 0x430d1f (0x4552 bytes) 2024-09-17T22:42:15.531 INFO:teuthology.orchestra.run.smithi115.stdout:3244 write 0x2729c8e thru 0x2734776 (0xaae9 bytes) 2024-09-17T22:42:15.535 INFO:teuthology.orchestra.run.smithi115.stdout:3246 read 0x2de4853 thru 0x2df21f8 (0xd9a6 bytes) 2024-09-17T22:42:15.535 INFO:teuthology.orchestra.run.smithi115.stdout:3247 read 0x1bea4bd thru 0x1bf07a3 (0x62e7 bytes) 2024-09-17T22:42:15.546 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:15.544+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:15.547 INFO:teuthology.orchestra.run.smithi115.stdout:3248 trunc from 0x44e7f0e to 0xd68d751 2024-09-17T22:42:15.552 INFO:teuthology.orchestra.run.smithi115.stdout:3250 read 0x9f029be thru 0x9f0af7b (0x85be bytes) 2024-09-17T22:42:15.552 INFO:teuthology.orchestra.run.smithi115.stdout:3252 write 0x30e87cf thru 0x30f7435 (0xec67 bytes) 2024-09-17T22:42:15.557 INFO:teuthology.orchestra.run.smithi115.stdout:3253 read 0xda9372 thru 0xdb1ff8 (0x8c87 bytes) 2024-09-17T22:42:15.557 INFO:teuthology.orchestra.run.smithi115.stdout:3254 read 0x3a3c5fd thru 0x3a44709 (0x810d bytes) 2024-09-17T22:42:15.557 INFO:teuthology.orchestra.run.smithi115.stdout:3255 writesame 0xc7b6097 thru 0xc7c0441 data_size 0x1(0xa3ab bytes) 2024-09-17T22:42:15.569 INFO:teuthology.orchestra.run.smithi115.stdout:3256 read 0x86998e5 thru 0x869c8ca (0x2fe6 bytes) 2024-09-17T22:42:15.569 INFO:teuthology.orchestra.run.smithi115.stdout:3257 read 0x26424bc thru 0x2646968 (0x44ad bytes) 2024-09-17T22:42:15.569 INFO:teuthology.orchestra.run.smithi115.stdout:3260 read 0x6daf6e1 thru 0x6db3aeb (0x440b bytes) 2024-09-17T22:42:15.570 INFO:teuthology.orchestra.run.smithi115.stdout:3261 punch from 0xc574e4 to 0xc642f0, (0xce0c bytes) 2024-09-17T22:42:15.570 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:15.576 INFO:teuthology.orchestra.run.smithi115.stdout:3264 write 0xe64e91d thru 0xe65c23d (0xd921 bytes) 2024-09-17T22:42:15.581 INFO:teuthology.orchestra.run.smithi115.stdout:3265 write 0xa34d248 thru 0xa3564a6 (0x925f bytes) 2024-09-17T22:42:15.586 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:15.586 INFO:teuthology.orchestra.run.smithi115.stdout:3268 punch from 0x2dd6c9d to 0x2de3dd4, (0xd137 bytes) 2024-09-17T22:42:15.586 INFO:teuthology.orchestra.run.smithi115.stdout:3270 punch from 0xc7759c1 to 0xc78074b, (0xad8a bytes) 2024-09-17T22:42:15.586 INFO:teuthology.orchestra.run.smithi115.stdout:3271 write 0x4be265d thru 0x4bef265 (0xcc09 bytes) 2024-09-17T22:42:15.590 INFO:teuthology.orchestra.run.smithi115.stdout:3273 clone 29 order 22 su 2097152 sc 7 2024-09-17T22:42:15.609 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:15.606+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:15.655 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:15.655 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:15.663 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:42:15.710 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:15.707+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:15.805 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:15.805 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:15.810 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:15.808+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:15.814 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:42:15.911 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:15.909+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:15.917 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:15.917 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:15.926 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:42:16.012 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:16.009+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:16.077 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:16.077 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:16.078 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:42:16.138 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:16.135+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:16.193 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:16.194 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:16.203 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:42:16.220 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is unset 2024-09-17T22:42:16.221 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:16.218+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:16.260 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:42:16.315 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:16.313+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:16.318 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone28 from 0xe65c23e (overlap 0x7b2804) to 0x18a19a 2024-09-17T22:42:16.345 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:16.345 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:16.356 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:42:16.402 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:16.416 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:16.413+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:16.421 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:16.421 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:16.438 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:16.439 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:16.455 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:16.455 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:16.458 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-17T22:42:16.473 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:16.474 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:16.490 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:16.490 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:16.495 INFO:teuthology.orchestra.run.smithi115.stdout:3274 read 0xbf601a4 thru 0xbf62257 (0x20b4 bytes) 2024-09-17T22:42:16.496 INFO:teuthology.orchestra.run.smithi115.stdout:3276 write 0x4d711b3 thru 0x4d76ce1 (0x5b2f bytes) 2024-09-17T22:42:16.504 INFO:teuthology.orchestra.run.smithi115.stdout:3278 read 0x33869c7 thru 0x338abb5 (0x41ef bytes) 2024-09-17T22:42:16.504 INFO:teuthology.orchestra.run.smithi115.stdout:3279 write 0xc32aaa4 thru 0xc337995 (0xcef2 bytes) 2024-09-17T22:42:16.509 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:16.509 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:16.510 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:16.511 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:16.511 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:16.511 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:16.511 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:16.514 INFO:teuthology.orchestra.run.smithi115.stdout:3280 trunc from 0xe65c23e to 0x8381379 2024-09-17T22:42:16.516 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:16.514+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:16.523 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:42:16.528 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:16.529 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:16.531 INFO:teuthology.orchestra.run.smithi115.stdout:3281 write 0x244dfad thru 0x24529fb (0x4a4f bytes) 2024-09-17T22:42:16.532 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:16.532 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:16.534 INFO:teuthology.orchestra.run.smithi115.stdout:3282 trunc from 0x8381379 to 0x77f1321 2024-09-17T22:42:16.549 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:16.549 INFO:teuthology.orchestra.run.smithi115.stdout:3284 read 0x5798da3 thru 0x57a72ed (0xe54b bytes) 2024-09-17T22:42:16.549 INFO:teuthology.orchestra.run.smithi115.stdout:3286 read 0x211e691 thru 0x212e507 (0xfe77 bytes) 2024-09-17T22:42:16.550 INFO:teuthology.orchestra.run.smithi115.stdout:3287 read 0x3dfc51d thru 0x3e04169 (0x7c4d bytes) 2024-09-17T22:42:16.550 INFO:teuthology.orchestra.run.smithi115.stdout:3288 read 0x5e5bc64 thru 0x5e63e75 (0x8212 bytes) 2024-09-17T22:42:16.550 INFO:teuthology.orchestra.run.smithi115.stdout:3290 write 0xb2f089 thru 0xb3d951 (0xe8c9 bytes) 2024-09-17T22:42:16.553 INFO:teuthology.orchestra.run.smithi115.stdout:3291 read 0x213b975 thru 0x2143e89 (0x8515 bytes) 2024-09-17T22:42:16.553 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:16.561 INFO:teuthology.orchestra.run.smithi115.stdout:3293 trunc from 0x77f1321 to 0xc79b34 2024-09-17T22:42:16.615 INFO:teuthology.orchestra.run.smithi115.stdout:3295 write 0xac784a2 thru 0xac8083b (0x839a bytes) 2024-09-17T22:42:16.617 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:16.614+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:16.627 INFO:teuthology.orchestra.run.smithi115.stdout:3296 write 0xd3dce62 thru 0xd3e3943 (0x6ae2 bytes) 2024-09-17T22:42:16.630 INFO:teuthology.orchestra.run.smithi115.stdout:3297 read 0x31850f0 thru 0x31930ed (0xdffe bytes) 2024-09-17T22:42:16.632 INFO:teuthology.orchestra.run.smithi115.stdout:3298 trunc from 0xd3e3944 to 0xb85ad35 2024-09-17T22:42:16.642 INFO:teuthology.orchestra.run.smithi115.stdout:3299 punch from 0x5bb815c to 0x5bbacfc, (0x2ba0 bytes) 2024-09-17T22:42:16.643 INFO:teuthology.orchestra.run.smithi115.stdout:3300 trunc from 0xb85ad35 to 0xc6f93ff 2024-09-17T22:42:16.650 INFO:teuthology.orchestra.run.smithi115.stdout:3301 trunc from 0xc6f93ff to 0xe84e810 2024-09-17T22:42:16.653 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:16.653 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:16.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:16.653 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:16.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.653 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145246 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3850240~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.653 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:16.653 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.319543867, 2024-09-17T22:42:16.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052644991000000002, 2024-09-17T22:42:16.654 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.654 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.654 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.654 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145246 2024-09-17T22:42:16.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.654 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:16.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:16.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329899+0000", 2024-09-17T22:42:16.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-09-17T22:42:16.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329901+0000", 2024-09-17T22:42:16.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1619999999999998e-06 2024-09-17T22:42:16.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329907+0000", 2024-09-17T22:42:16.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9059999999999999e-06 2024-09-17T22:42:16.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329911+0000", 2024-09-17T22:42:16.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7419999999999997e-06 2024-09-17T22:42:16.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382383+0000", 2024-09-17T22:42:16.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052471583000000002 2024-09-17T22:42:16.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382457+0000", 2024-09-17T22:42:16.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4309999999999998e-05 2024-09-17T22:42:16.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382543+0000", 2024-09-17T22:42:16.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5918e-05 2024-09-17T22:42:16.658 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.659 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.659 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.659 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145247 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3946496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.659 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:16.659 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.319525216999999, 2024-09-17T22:42:16.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052805863000000001, 2024-09-17T22:42:16.659 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.659 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.659 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.660 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.660 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.660 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.660 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145247 2024-09-17T22:42:16.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.660 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:16.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:16.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:16.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3200000000000001e-07 2024-09-17T22:42:16.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329919+0000", 2024-09-17T22:42:16.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.888e-06 2024-09-17T22:42:16.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329925+0000", 2024-09-17T22:42:16.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5369999999999997e-06 2024-09-17T22:42:16.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329929+0000", 2024-09-17T22:42:16.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.639e-06 2024-09-17T22:42:16.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382570+0000", 2024-09-17T22:42:16.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052641213999999999 2024-09-17T22:42:16.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382636+0000", 2024-09-17T22:42:16.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6008999999999998e-05 2024-09-17T22:42:16.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382723+0000", 2024-09-17T22:42:16.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6243999999999996e-05 2024-09-17T22:42:16.664 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.664 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.664 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.665 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145248 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3856384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.665 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:16.665 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.319507169000001, 2024-09-17T22:42:16.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053002858, 2024-09-17T22:42:16.665 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.665 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.665 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.665 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.665 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.665 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.665 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145248 2024-09-17T22:42:16.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.666 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:16.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:16.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:16.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.13e-07 2024-09-17T22:42:16.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329937+0000", 2024-09-17T22:42:16.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9300000000000002e-06 2024-09-17T22:42:16.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329943+0000", 2024-09-17T22:42:16.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-09-17T22:42:16.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329948+0000", 2024-09-17T22:42:16.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8579999999999999e-06 2024-09-17T22:42:16.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382763+0000", 2024-09-17T22:42:16.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052814537000000002 2024-09-17T22:42:16.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382840+0000", 2024-09-17T22:42:16.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6879999999999996e-05 2024-09-17T22:42:16.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382938+0000", 2024-09-17T22:42:16.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8159999999999995e-05 2024-09-17T22:42:16.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.669 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.669 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145249 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3952640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.319488268000001, 2024-09-17T22:42:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053188602000000001, 2024-09-17T22:42:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145249 2024-09-17T22:42:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3700000000000001e-07 2024-09-17T22:42:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329956+0000", 2024-09-17T22:42:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1430000000000001e-06 2024-09-17T22:42:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329962+0000", 2024-09-17T22:42:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8270000000000003e-06 2024-09-17T22:42:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329967+0000", 2024-09-17T22:42:16.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7249999999999997e-06 2024-09-17T22:42:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382969+0000", 2024-09-17T22:42:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053001952999999997 2024-09-17T22:42:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383040+0000", 2024-09-17T22:42:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1272999999999993e-05 2024-09-17T22:42:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383142+0000", 2024-09-17T22:42:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010234399999999999 2024-09-17T22:42:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145250 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3862528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.319469469000001, 2024-09-17T22:42:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053359537999999998, 2024-09-17T22:42:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.674 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145250 2024-09-17T22:42:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:42:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329975+0000", 2024-09-17T22:42:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2199999999999999e-06 2024-09-17T22:42:16.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329981+0000", 2024-09-17T22:42:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1020000000000002e-06 2024-09-17T22:42:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329991+0000", 2024-09-17T22:42:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6800000000000005e-06 2024-09-17T22:42:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383171+0000", 2024-09-17T22:42:16.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053180142 2024-09-17T22:42:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383242+0000", 2024-09-17T22:42:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1310000000000007e-05 2024-09-17T22:42:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383332+0000", 2024-09-17T22:42:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9703999999999994e-05 2024-09-17T22:42:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145251 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3958784~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.319443768999999, 2024-09-17T22:42:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053438537000000001, 2024-09-17T22:42:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145251 2024-09-17T22:42:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329999+0000", 2024-09-17T22:42:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-07 2024-09-17T22:42:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330001+0000", 2024-09-17T22:42:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:42:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330006+0000", 2024-09-17T22:42:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000004e-06 2024-09-17T22:42:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330011+0000", 2024-09-17T22:42:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4039999999999997e-06 2024-09-17T22:42:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383350+0000", 2024-09-17T22:42:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053339011999999998 2024-09-17T22:42:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383387+0000", 2024-09-17T22:42:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7085999999999997e-05 2024-09-17T22:42:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383437+0000", 2024-09-17T22:42:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9988000000000003e-05 2024-09-17T22:42:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145252 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3868672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.684 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.319425322000001, 2024-09-17T22:42:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053526724999999997, 2024-09-17T22:42:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145252 2024-09-17T22:42:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.685 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331054+0000", 2024-09-17T22:42:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010371670000000001 2024-09-17T22:42:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331059+0000", 2024-09-17T22:42:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1680000000000003e-06 2024-09-17T22:42:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331075+0000", 2024-09-17T22:42:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6028e-05 2024-09-17T22:42:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331084+0000", 2024-09-17T22:42:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3980000000000005e-06 2024-09-17T22:42:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383464+0000", 2024-09-17T22:42:16.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052379851999999998 2024-09-17T22:42:16.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383499+0000", 2024-09-17T22:42:16.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4610000000000002e-05 2024-09-17T22:42:16.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383543+0000", 2024-09-17T22:42:16.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4502000000000002e-05 2024-09-17T22:42:16.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.690 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.690 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145255 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3971072~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.690 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:16.690 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.31829523, 2024-09-17T22:42:16.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052677527000000002, 2024-09-17T22:42:16.690 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.691 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.691 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.691 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.691 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.691 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.691 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145255 2024-09-17T22:42:16.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.691 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:16.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:16.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:16.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8800000000000002e-07 2024-09-17T22:42:16.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331150+0000", 2024-09-17T22:42:16.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1160000000000002e-06 2024-09-17T22:42:16.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331156+0000", 2024-09-17T22:42:16.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7469999999999999e-06 2024-09-17T22:42:16.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331162+0000", 2024-09-17T22:42:16.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-17T22:42:16.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383750+0000", 2024-09-17T22:42:16.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052588441999999999 2024-09-17T22:42:16.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383782+0000", 2024-09-17T22:42:16.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1610999999999999e-05 2024-09-17T22:42:16.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383824+0000", 2024-09-17T22:42:16.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2512000000000002e-05 2024-09-17T22:42:16.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.695 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.696 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145256 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3880960~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.696 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:16.696 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.318273809000001, 2024-09-17T22:42:16.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052744775000000001, 2024-09-17T22:42:16.696 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.696 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.696 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145256 2024-09-17T22:42:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331169+0000", 2024-09-17T22:42:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:42:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331175+0000", 2024-09-17T22:42:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5660000000000003e-06 2024-09-17T22:42:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331182+0000", 2024-09-17T22:42:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0169999999999999e-06 2024-09-17T22:42:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331188+0000", 2024-09-17T22:42:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2750000000000003e-06 2024-09-17T22:42:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383839+0000", 2024-09-17T22:42:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052651728000000002 2024-09-17T22:42:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383872+0000", 2024-09-17T22:42:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2347e-05 2024-09-17T22:42:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383913+0000", 2024-09-17T22:42:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1362000000000001e-05 2024-09-17T22:42:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145257 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3977216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.318247760999999, 2024-09-17T22:42:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052809598999999999, 2024-09-17T22:42:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145257 2024-09-17T22:42:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333051+0000", 2024-09-17T22:42:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018567189999999999 2024-09-17T22:42:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333056+0000", 2024-09-17T22:42:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-17T22:42:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333067+0000", 2024-09-17T22:42:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1704e-05 2024-09-17T22:42:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333074+0000", 2024-09-17T22:42:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1280000000000002e-06 2024-09-17T22:42:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383927+0000", 2024-09-17T22:42:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050853094000000001 2024-09-17T22:42:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383960+0000", 2024-09-17T22:42:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2891000000000001e-05 2024-09-17T22:42:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.384004+0000", 2024-09-17T22:42:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3565e-05 2024-09-17T22:42:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146168 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2467840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.254759140000001, 2024-09-17T22:42:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052652501999999997, 2024-09-17T22:42:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146168 2024-09-17T22:42:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394989+0000", 2024-09-17T22:42:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030618300000000002 2024-09-17T22:42:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394994+0000", 2024-09-17T22:42:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.566e-06 2024-09-17T22:42:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395003+0000", 2024-09-17T22:42:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3619999999999995e-06 2024-09-17T22:42:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395009+0000", 2024-09-17T22:42:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3589999999999996e-06 2024-09-17T22:42:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447180+0000", 2024-09-17T22:42:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052170643000000003 2024-09-17T22:42:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447251+0000", 2024-09-17T22:42:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1382000000000002e-05 2024-09-17T22:42:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447335+0000", 2024-09-17T22:42:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4006999999999997e-05 2024-09-17T22:42:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146172 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2480128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.254404255000001, 2024-09-17T22:42:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052714629999999998, 2024-09-17T22:42:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146172 2024-09-17T22:42:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395486+0000", 2024-09-17T22:42:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044787000000000001 2024-09-17T22:42:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395491+0000", 2024-09-17T22:42:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8550000000000001e-06 2024-09-17T22:42:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395503+0000", 2024-09-17T22:42:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2271e-05 2024-09-17T22:42:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395510+0000", 2024-09-17T22:42:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6320000000000008e-06 2024-09-17T22:42:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447565+0000", 2024-09-17T22:42:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052055016000000003 2024-09-17T22:42:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447644+0000", 2024-09-17T22:42:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8306000000000005e-05 2024-09-17T22:42:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447752+0000", 2024-09-17T22:42:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010868 2024-09-17T22:42:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146181 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2504704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.253536865000001, 2024-09-17T22:42:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052850394000000002, 2024-09-17T22:42:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146181 2024-09-17T22:42:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396210+0000", 2024-09-17T22:42:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000305214 2024-09-17T22:42:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396214+0000", 2024-09-17T22:42:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9389999999999999e-06 2024-09-17T22:42:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396228+0000", 2024-09-17T22:42:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3365e-05 2024-09-17T22:42:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396235+0000", 2024-09-17T22:42:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2559999999999996e-06 2024-09-17T22:42:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448568+0000", 2024-09-17T22:42:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052333084000000002 2024-09-17T22:42:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448646+0000", 2024-09-17T22:42:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8287000000000004e-05 2024-09-17T22:42:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448756+0000", 2024-09-17T22:42:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109249 2024-09-17T22:42:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146183 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2510848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.253196318000001, 2024-09-17T22:42:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052725453999999998, 2024-09-17T22:42:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146183 2024-09-17T22:42:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396508+0000", 2024-09-17T22:42:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026271299999999998 2024-09-17T22:42:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396512+0000", 2024-09-17T22:42:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8039999999999999e-06 2024-09-17T22:42:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396522+0000", 2024-09-17T22:42:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9289999999999993e-06 2024-09-17T22:42:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396528+0000", 2024-09-17T22:42:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2990000000000002e-06 2024-09-17T22:42:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448798+0000", 2024-09-17T22:42:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052269985999999997 2024-09-17T22:42:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448871+0000", 2024-09-17T22:42:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2581000000000002e-05 2024-09-17T22:42:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448971+0000", 2024-09-17T22:42:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100142 2024-09-17T22:42:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146391 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.245468826, 2024-09-17T22:42:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052666719000000001, 2024-09-17T22:42:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146391 2024-09-17T22:42:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404098+0000", 2024-09-17T22:42:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000124509 2024-09-17T22:42:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404107+0000", 2024-09-17T22:42:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4220000000000006e-06 2024-09-17T22:42:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404117+0000", 2024-09-17T22:42:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011e-05 2024-09-17T22:42:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404123+0000", 2024-09-17T22:42:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1530000000000002e-06 2024-09-17T22:42:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456481+0000", 2024-09-17T22:42:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052357923000000001 2024-09-17T22:42:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456551+0000", 2024-09-17T22:42:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9869000000000005e-05 2024-09-17T22:42:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456640+0000", 2024-09-17T22:42:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8733000000000006e-05 2024-09-17T22:42:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146393 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2748416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.245308763000001, 2024-09-17T22:42:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052739239, 2024-09-17T22:42:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146393 2024-09-17T22:42:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404443+0000", 2024-09-17T22:42:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030977199999999999 2024-09-17T22:42:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404447+0000", 2024-09-17T22:42:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6370000000000001e-06 2024-09-17T22:42:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404469+0000", 2024-09-17T22:42:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2739999999999999e-05 2024-09-17T22:42:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404477+0000", 2024-09-17T22:42:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6559999999999992e-06 2024-09-17T22:42:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456734+0000", 2024-09-17T22:42:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052256889000000001 2024-09-17T22:42:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456796+0000", 2024-09-17T22:42:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2485999999999998e-05 2024-09-17T22:42:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456873+0000", 2024-09-17T22:42:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6058999999999998e-05 2024-09-17T22:42:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146397 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2760704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.244932999, 2024-09-17T22:42:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052788928999999998, 2024-09-17T22:42:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146397 2024-09-17T22:42:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404763+0000", 2024-09-17T22:42:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025351399999999999 2024-09-17T22:42:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404767+0000", 2024-09-17T22:42:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:42:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404777+0000", 2024-09-17T22:42:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9310000000000003e-06 2024-09-17T22:42:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404783+0000", 2024-09-17T22:42:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.336e-06 2024-09-17T22:42:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457126+0000", 2024-09-17T22:42:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052342975999999999 2024-09-17T22:42:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457200+0000", 2024-09-17T22:42:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4146000000000005e-05 2024-09-17T22:42:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457298+0000", 2024-09-17T22:42:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7807999999999996e-05 2024-09-17T22:42:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146399 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2766848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.244642857999999, 2024-09-17T22:42:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052829779, 2024-09-17T22:42:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146399 2024-09-17T22:42:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404800+0000", 2024-09-17T22:42:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8500000000000001e-07 2024-09-17T22:42:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404802+0000", 2024-09-17T22:42:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5349999999999999e-06 2024-09-17T22:42:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404811+0000", 2024-09-17T22:42:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9390000000000003e-06 2024-09-17T22:42:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404817+0000", 2024-09-17T22:42:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.524e-06 2024-09-17T22:42:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457461+0000", 2024-09-17T22:42:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052643784999999998 2024-09-17T22:42:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457539+0000", 2024-09-17T22:42:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8589000000000003e-05 2024-09-17T22:42:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457629+0000", 2024-09-17T22:42:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9821999999999995e-05 2024-09-17T22:42:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146401 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2772992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.244617610999999, 2024-09-17T22:42:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052910077, 2024-09-17T22:42:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146401 2024-09-17T22:42:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405106+0000", 2024-09-17T22:42:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000281286 2024-09-17T22:42:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405110+0000", 2024-09-17T22:42:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1459999999999998e-06 2024-09-17T22:42:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405120+0000", 2024-09-17T22:42:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0546e-05 2024-09-17T22:42:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405127+0000", 2024-09-17T22:42:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4300000000000003e-06 2024-09-17T22:42:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457647+0000", 2024-09-17T22:42:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052520229000000002 2024-09-17T22:42:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457684+0000", 2024-09-17T22:42:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6609999999999997e-05 2024-09-17T22:42:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457735+0000", 2024-09-17T22:42:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.083e-05 2024-09-17T22:42:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146402 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2779136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.244304198999998, 2024-09-17T22:42:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052689812000000003, 2024-09-17T22:42:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146402 2024-09-17T22:42:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405365+0000", 2024-09-17T22:42:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022744599999999999 2024-09-17T22:42:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405370+0000", 2024-09-17T22:42:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2769999999999999e-06 2024-09-17T22:42:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405380+0000", 2024-09-17T22:42:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0349e-05 2024-09-17T22:42:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405386+0000", 2024-09-17T22:42:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5169999999999996e-06 2024-09-17T22:42:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457750+0000", 2024-09-17T22:42:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052363448999999999 2024-09-17T22:42:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457784+0000", 2024-09-17T22:42:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4363999999999998e-05 2024-09-17T22:42:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457828+0000", 2024-09-17T22:42:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999999e-05 2024-09-17T22:42:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:16.758 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:16.758 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:16.759 INFO:teuthology.orchestra.run.smithi115.stdout:3302 write 0x6c869de thru 0x6c877d1 (0xdf4 bytes) 2024-09-17T22:42:16.759 INFO:teuthology.orchestra.run.smithi115.stdout:3303 write 0xb0d178 thru 0xb186ef (0xb578 bytes) 2024-09-17T22:42:16.759 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:16.759 INFO:teuthology.orchestra.run.smithi115.stdout:3305 read 0xb434124 thru 0xb43c49e (0x837b bytes) 2024-09-17T22:42:16.759 INFO:teuthology.orchestra.run.smithi115.stdout:3306 trunc from 0xe84e810 to 0xd2403aa 2024-09-17T22:42:16.760 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:42:16.761 INFO:teuthology.orchestra.run.smithi115.stdout:3307 trunc from 0xd2403aa to 0x80bd49d 2024-09-17T22:42:16.761 INFO:teuthology.orchestra.run.smithi115.stdout:3308 trunc from 0x80bd49d to 0x1847932 2024-09-17T22:42:16.762 INFO:teuthology.orchestra.run.smithi115.stdout:3309 punch from 0xd37c62 to 0xd3eee6, (0x7284 bytes) 2024-09-17T22:42:16.762 INFO:teuthology.orchestra.run.smithi115.stdout:3312 punch from 0x10da6e5 to 0x10e9324, (0xec3f bytes) 2024-09-17T22:42:16.762 INFO:teuthology.orchestra.run.smithi115.stdout:3313 write 0xbceb834 thru 0xbcedbf1 (0x23be bytes) 2024-09-17T22:42:16.762 INFO:teuthology.orchestra.run.smithi115.stdout:3315 writesame 0x39e99f thru 0x3a58ea data_size 0x22(0x6f4c bytes) 2024-09-17T22:42:16.762 INFO:teuthology.orchestra.run.smithi115.stdout:3316 punch from 0x930e2a1 to 0x9310d41, (0x2aa0 bytes) 2024-09-17T22:42:16.762 INFO:teuthology.orchestra.run.smithi115.stdout:3317 punch from 0xaeb1d05 to 0xaec0a69, (0xed64 bytes) 2024-09-17T22:42:16.762 INFO:teuthology.orchestra.run.smithi115.stdout:3318 punch from 0x39f24de to 0x39f936f, (0x6e91 bytes) 2024-09-17T22:42:16.762 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:16.762 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:16.762 INFO:teuthology.orchestra.run.smithi115.stdout:3321 punch from 0x77d724e to 0x77dfb80, (0x8932 bytes) 2024-09-17T22:42:16.762 INFO:teuthology.orchestra.run.smithi115.stdout:3322 punch from 0x33426e3 to 0x3347c7e, (0x559b bytes) 2024-09-17T22:42:16.762 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:16.762 INFO:teuthology.orchestra.run.smithi115.stdout:3325 punch from 0x685e055 to 0x686b355, (0xd300 bytes) 2024-09-17T22:42:16.763 INFO:teuthology.orchestra.run.smithi115.stdout:3327 write 0x8207a65 thru 0x8213041 (0xb5dd bytes) 2024-09-17T22:42:16.763 INFO:teuthology.orchestra.run.smithi115.stdout:3328 read 0x71ce5f9 thru 0x71d68f5 (0x82fd bytes) 2024-09-17T22:42:16.763 INFO:teuthology.orchestra.run.smithi115.stdout:3329 trunc from 0xbcedbf2 to 0xe043e5c 2024-09-17T22:42:16.763 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:16.763 INFO:teuthology.orchestra.run.smithi115.stdout:3331 writesame 0xd476882 thru 0xd4844cc data_size 0x1(0xdc4b bytes) 2024-09-17T22:42:16.763 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:16.756+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:16.772 INFO:teuthology.orchestra.run.smithi115.stdout:3332 trunc from 0xe043e5c to 0x115b1fe 2024-09-17T22:42:16.783 INFO:teuthology.orchestra.run.smithi115.stdout:3333 write 0x1b2d39 thru 0x1bc7c9 (0x9a91 bytes) 2024-09-17T22:42:16.785 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:16.785 INFO:teuthology.orchestra.run.smithi115.stdout:3336 trunc from 0x115b1fe to 0xb18a3c 2024-09-17T22:42:16.811 INFO:teuthology.orchestra.run.smithi115.stdout:3337 writesame 0x9ef5559 thru 0x9efd69b data_size 0x1(0x8143 bytes) 2024-09-17T22:42:16.811 INFO:teuthology.orchestra.run.smithi115.stdout:3338 read 0x7a93542 thru 0x7a9dcc1 (0xa780 bytes) 2024-09-17T22:42:16.811 INFO:teuthology.orchestra.run.smithi115.stdout:3339 writesame 0x2fafb72 thru 0x2fb447a data_size 0x1(0x4909 bytes) 2024-09-17T22:42:16.821 INFO:teuthology.orchestra.run.smithi115.stdout:3340 clone 30 order 22 su 16384 sc 8 2024-09-17T22:42:16.859 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:16.858+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:16.880 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:16.880 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:16.880 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:16.880 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:16.888 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:42:16.972 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:16.970+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:17.032 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:17.032 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:17.032 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:17.033 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:17.033 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:17.041 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:42:17.082 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:17.079+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:17.162 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:17.160+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:17.185 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:17.185 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4229.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-17T22:42:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 67.787814338000004, 2024-09-17T22:42:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25708877200000002, 2024-09-17T22:42:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.186 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4229", 2024-09-17T22:42:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/3080427038", 2024-09-17T22:42:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2 2024-09-17T22:42:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:17.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394611+0000", 2024-09-17T22:42:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3149999999999999e-06 2024-09-17T22:42:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394619+0000", 2024-09-17T22:42:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240000000000003e-06 2024-09-17T22:42:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394632+0000", 2024-09-17T22:42:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2921e-05 2024-09-17T22:42:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394639+0000", 2024-09-17T22:42:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-09-17T22:42:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394703+0000", 2024-09-17T22:42:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4407999999999997e-05 2024-09-17T22:42:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:42:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394711+0000", 2024-09-17T22:42:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5939999999999996e-06 2024-09-17T22:42:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651545+0000", 2024-09-17T22:42:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25683409200000001 2024-09-17T22:42:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651698+0000", 2024-09-17T22:42:17.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000153678 2024-09-17T22:42:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38736 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.405218333000001, 2024-09-17T22:42:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051395418999999998, 2024-09-17T22:42:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:17.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38736 2024-09-17T22:42:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6899999999999998e-07 2024-09-17T22:42:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777212+0000", 2024-09-17T22:42:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1240000000000002e-06 2024-09-17T22:42:17.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777219+0000", 2024-09-17T22:42:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5409999999999997e-06 2024-09-17T22:42:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777224+0000", 2024-09-17T22:42:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:42:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.826131+0000", 2024-09-17T22:42:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048907369999999999 2024-09-17T22:42:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828457+0000", 2024-09-17T22:42:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023256209999999999 2024-09-17T22:42:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828601+0000", 2024-09-17T22:42:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014438799999999999 2024-09-17T22:42:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38752 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 673792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.405194262000002, 2024-09-17T22:42:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051582301999999997, 2024-09-17T22:42:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.196 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38752 2024-09-17T22:42:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3000000000000001e-07 2024-09-17T22:42:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777232+0000", 2024-09-17T22:42:17.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9709999999999998e-06 2024-09-17T22:42:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777238+0000", 2024-09-17T22:42:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7209999999999999e-06 2024-09-17T22:42:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777243+0000", 2024-09-17T22:42:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5210000000000004e-06 2024-09-17T22:42:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828633+0000", 2024-09-17T22:42:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051390915000000002 2024-09-17T22:42:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828711+0000", 2024-09-17T22:42:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7474999999999999e-05 2024-09-17T22:42:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828812+0000", 2024-09-17T22:42:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010116900000000001 2024-09-17T22:42:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38753 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.405174989000002, 2024-09-17T22:42:17.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051770444999999998, 2024-09-17T22:42:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38753 2024-09-17T22:42:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:42:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777251+0000", 2024-09-17T22:42:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8059999999999999e-06 2024-09-17T22:42:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777257+0000", 2024-09-17T22:42:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2480000000000001e-06 2024-09-17T22:42:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777262+0000", 2024-09-17T22:42:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000004e-06 2024-09-17T22:42:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828842+0000", 2024-09-17T22:42:17.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051580203999999998 2024-09-17T22:42:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828917+0000", 2024-09-17T22:42:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4980000000000004e-05 2024-09-17T22:42:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829019+0000", 2024-09-17T22:42:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102248 2024-09-17T22:42:17.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38766 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.405156003999998, 2024-09-17T22:42:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051958839999999999, 2024-09-17T22:42:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.207 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38766 2024-09-17T22:42:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:17.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777269+0000", 2024-09-17T22:42:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:42:17.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777271+0000", 2024-09-17T22:42:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-17T22:42:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777276+0000", 2024-09-17T22:42:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:42:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777281+0000", 2024-09-17T22:42:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:42:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829055+0000", 2024-09-17T22:42:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051773570999999997 2024-09-17T22:42:17.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829128+0000", 2024-09-17T22:42:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3131000000000002e-05 2024-09-17T22:42:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829227+0000", 2024-09-17T22:42:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8800000000000003e-05 2024-09-17T22:42:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38767 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.405136020999997, 2024-09-17T22:42:17.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137018, 2024-09-17T22:42:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38767 2024-09-17T22:42:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:42:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777290+0000", 2024-09-17T22:42:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.88e-06 2024-09-17T22:42:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777296+0000", 2024-09-17T22:42:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-09-17T22:42:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777300+0000", 2024-09-17T22:42:17.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4510000000000002e-06 2024-09-17T22:42:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829257+0000", 2024-09-17T22:42:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051956975000000002 2024-09-17T22:42:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829328+0000", 2024-09-17T22:42:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1026999999999996e-05 2024-09-17T22:42:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829425+0000", 2024-09-17T22:42:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.679e-05 2024-09-17T22:42:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38784 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 675840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.405117566999998, 2024-09-17T22:42:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052344522999999997, 2024-09-17T22:42:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.218 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38784 2024-09-17T22:42:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777307+0000", 2024-09-17T22:42:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:42:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777309+0000", 2024-09-17T22:42:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.063e-06 2024-09-17T22:42:17.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777314+0000", 2024-09-17T22:42:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-17T22:42:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777319+0000", 2024-09-17T22:42:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999996e-06 2024-09-17T22:42:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829456+0000", 2024-09-17T22:42:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137151 2024-09-17T22:42:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829527+0000", 2024-09-17T22:42:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1588000000000006e-05 2024-09-17T22:42:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829651+0000", 2024-09-17T22:42:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012365 2024-09-17T22:42:17.222 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38785 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 749568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.405099452000002, 2024-09-17T22:42:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052546572999999999, 2024-09-17T22:42:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.223 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38785 2024-09-17T22:42:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:42:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777327+0000", 2024-09-17T22:42:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:42:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777333+0000", 2024-09-17T22:42:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7309999999999998e-06 2024-09-17T22:42:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777337+0000", 2024-09-17T22:42:17.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:42:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829686+0000", 2024-09-17T22:42:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052349159999999999 2024-09-17T22:42:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829766+0000", 2024-09-17T22:42:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9326999999999995e-05 2024-09-17T22:42:17.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829871+0000", 2024-09-17T22:42:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000105466 2024-09-17T22:42:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38798 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:17.228 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.404465844000001, 2024-09-17T22:42:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052117564999999998, 2024-09-17T22:42:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38798 2024-09-17T22:42:17.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:17.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777959+0000", 2024-09-17T22:42:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9599999999999999e-07 2024-09-17T22:42:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777963+0000", 2024-09-17T22:42:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2660000000000003e-06 2024-09-17T22:42:17.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777971+0000", 2024-09-17T22:42:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3489999999999992e-06 2024-09-17T22:42:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777978+0000", 2024-09-17T22:42:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:42:17.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829906+0000", 2024-09-17T22:42:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051927635999999999 2024-09-17T22:42:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829986+0000", 2024-09-17T22:42:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0444000000000004e-05 2024-09-17T22:42:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830076+0000", 2024-09-17T22:42:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9473000000000002e-05 2024-09-17T22:42:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38799 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 882688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:17.234 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.404434405000003, 2024-09-17T22:42:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052359606000000003, 2024-09-17T22:42:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38799 2024-09-17T22:42:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8299999999999997e-07 2024-09-17T22:42:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777992+0000", 2024-09-17T22:42:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.176e-06 2024-09-17T22:42:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777998+0000", 2024-09-17T22:42:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.173e-06 2024-09-17T22:42:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778003+0000", 2024-09-17T22:42:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6349999999999997e-06 2024-09-17T22:42:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830112+0000", 2024-09-17T22:42:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052109366999999997 2024-09-17T22:42:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830192+0000", 2024-09-17T22:42:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9814000000000005e-05 2024-09-17T22:42:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830349+0000", 2024-09-17T22:42:17.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015695800000000001 2024-09-17T22:42:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38814 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.404151001000002, 2024-09-17T22:42:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052274339000000003, 2024-09-17T22:42:17.240 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38814 2024-09-17T22:42:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:17.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:42:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778276+0000", 2024-09-17T22:42:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2029999999999999e-06 2024-09-17T22:42:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778282+0000", 2024-09-17T22:42:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0059999999999998e-06 2024-09-17T22:42:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778287+0000", 2024-09-17T22:42:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0039999999999999e-06 2024-09-17T22:42:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830382+0000", 2024-09-17T22:42:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052095379999999997 2024-09-17T22:42:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830455+0000", 2024-09-17T22:42:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3127000000000006e-05 2024-09-17T22:42:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830547+0000", 2024-09-17T22:42:17.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2207999999999995e-05 2024-09-17T22:42:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38828 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 163840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.403968593000002, 2024-09-17T22:42:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052466039999999999, 2024-09-17T22:42:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38828 2024-09-17T22:42:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:17.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778456+0000", 2024-09-17T22:42:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7599999999999996e-07 2024-09-17T22:42:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778460+0000", 2024-09-17T22:42:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8229999999999997e-06 2024-09-17T22:42:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778470+0000", 2024-09-17T22:42:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8199999999999992e-06 2024-09-17T22:42:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778476+0000", 2024-09-17T22:42:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2589999999999997e-06 2024-09-17T22:42:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830578+0000", 2024-09-17T22:42:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052102273999999997 2024-09-17T22:42:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830667+0000", 2024-09-17T22:42:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8701000000000003e-05 2024-09-17T22:42:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830921+0000", 2024-09-17T22:42:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025438700000000002 2024-09-17T22:42:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.249 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38829 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.403937196000001, 2024-09-17T22:42:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052648882000000001, 2024-09-17T22:42:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38829 2024-09-17T22:42:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7e-07 2024-09-17T22:42:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778490+0000", 2024-09-17T22:42:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:42:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778495+0000", 2024-09-17T22:42:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9329999999999998e-06 2024-09-17T22:42:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778500+0000", 2024-09-17T22:42:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9509999999999997e-06 2024-09-17T22:42:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830958+0000", 2024-09-17T22:42:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05245785 2024-09-17T22:42:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831051+0000", 2024-09-17T22:42:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2989e-05 2024-09-17T22:42:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831136+0000", 2024-09-17T22:42:17.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4448000000000002e-05 2024-09-17T22:42:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38891 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.400625138000002, 2024-09-17T22:42:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050847934999999997, 2024-09-17T22:42:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.255 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38891 2024-09-17T22:42:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:17.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-09-17T22:42:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781802+0000", 2024-09-17T22:42:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:42:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781810+0000", 2024-09-17T22:42:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8019999999999992e-06 2024-09-17T22:42:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781816+0000", 2024-09-17T22:42:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.285e-06 2024-09-17T22:42:17.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832485+0000", 2024-09-17T22:42:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050668462999999997 2024-09-17T22:42:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832565+0000", 2024-09-17T22:42:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9692999999999997e-05 2024-09-17T22:42:17.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832647+0000", 2024-09-17T22:42:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2224000000000004e-05 2024-09-17T22:42:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38892 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.400596888999999, 2024-09-17T22:42:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050946120999999997, 2024-09-17T22:42:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38892 2024-09-17T22:42:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781828+0000", 2024-09-17T22:42:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5400000000000002e-07 2024-09-17T22:42:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781830+0000", 2024-09-17T22:42:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1959999999999998e-06 2024-09-17T22:42:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781837+0000", 2024-09-17T22:42:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8310000000000003e-06 2024-09-17T22:42:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781843+0000", 2024-09-17T22:42:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1e-06 2024-09-17T22:42:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832674+0000", 2024-09-17T22:42:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050831182000000003 2024-09-17T22:42:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832725+0000", 2024-09-17T22:42:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1588000000000001e-05 2024-09-17T22:42:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832773+0000", 2024-09-17T22:42:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7769999999999998e-05 2024-09-17T22:42:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38909 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 36864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.400574118999998, 2024-09-17T22:42:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051046518999999999, 2024-09-17T22:42:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38909 2024-09-17T22:42:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:42:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781852+0000", 2024-09-17T22:42:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0590000000000001e-06 2024-09-17T22:42:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781859+0000", 2024-09-17T22:42:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3380000000000001e-06 2024-09-17T22:42:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781864+0000", 2024-09-17T22:42:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:42:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832804+0000", 2024-09-17T22:42:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050939614000000001 2024-09-17T22:42:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832845+0000", 2024-09-17T22:42:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1014999999999999e-05 2024-09-17T22:42:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832896+0000", 2024-09-17T22:42:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1836999999999999e-05 2024-09-17T22:42:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38910 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.400553809000002, 2024-09-17T22:42:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051216818999999997, 2024-09-17T22:42:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38910 2024-09-17T22:42:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781871+0000", 2024-09-17T22:42:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:42:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781873+0000", 2024-09-17T22:42:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:42:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781879+0000", 2024-09-17T22:42:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0700000000000003e-06 2024-09-17T22:42:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781884+0000", 2024-09-17T22:42:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-09-17T22:42:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832932+0000", 2024-09-17T22:42:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051048219999999998 2024-09-17T22:42:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833001+0000", 2024-09-17T22:42:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8504999999999995e-05 2024-09-17T22:42:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833087+0000", 2024-09-17T22:42:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6464000000000004e-05 2024-09-17T22:42:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38923 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.400533488999997, 2024-09-17T22:42:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051379573999999997, 2024-09-17T22:42:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38923 2024-09-17T22:42:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:42:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781893+0000", 2024-09-17T22:42:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-09-17T22:42:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781900+0000", 2024-09-17T22:42:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3860000000000004e-06 2024-09-17T22:42:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781904+0000", 2024-09-17T22:42:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9300000000000002e-06 2024-09-17T22:42:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833115+0000", 2024-09-17T22:42:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051210845999999997 2024-09-17T22:42:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833188+0000", 2024-09-17T22:42:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2276999999999999e-05 2024-09-17T22:42:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833270+0000", 2024-09-17T22:42:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2501000000000002e-05 2024-09-17T22:42:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38941 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.399740954000002, 2024-09-17T22:42:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050969459000000002, 2024-09-17T22:42:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38941 2024-09-17T22:42:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782684+0000", 2024-09-17T22:42:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:42:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782686+0000", 2024-09-17T22:42:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.396e-06 2024-09-17T22:42:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782693+0000", 2024-09-17T22:42:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1999999999999997e-06 2024-09-17T22:42:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782699+0000", 2024-09-17T22:42:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7910000000000001e-06 2024-09-17T22:42:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833471+0000", 2024-09-17T22:42:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050772333000000003 2024-09-17T22:42:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833536+0000", 2024-09-17T22:42:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4380000000000004e-05 2024-09-17T22:42:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833653+0000", 2024-09-17T22:42:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011691799999999999 2024-09-17T22:42:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:151189 3.14 3:2b696b13:::rbd_data.115f14f2d1d5.0000000000000009:e [read 3127296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.469520208999999, 2024-09-17T22:42:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.054254700000000003, 2024-09-17T22:42:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 151189 2024-09-17T22:42:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713051+0000", 2024-09-17T22:42:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014722100000000001 2024-09-17T22:42:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713055+0000", 2024-09-17T22:42:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.16e-06 2024-09-17T22:42:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713065+0000", 2024-09-17T22:42:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8390000000000002e-06 2024-09-17T22:42:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713073+0000", 2024-09-17T22:42:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0390000000000005e-06 2024-09-17T22:42:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762644+0000", 2024-09-17T22:42:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049571160000000003 2024-09-17T22:42:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762739+0000", 2024-09-17T22:42:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4565e-05 2024-09-17T22:42:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.767159+0000", 2024-09-17T22:42:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044197159999999997 2024-09-17T22:42:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:17.292 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:17.292 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:42:17.292 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:42:17.292 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T22:42:17.293 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 1.000000 2024-09-17T22:42:17.293 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-17T22:42:17.295 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:42:17.295 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:17.291+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:17.327 INFO:teuthology.orchestra.run.smithi115.stdout:flattening image image_client.0-clone29 2024-09-17T22:42:17.396 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:17.393+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:17.400 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:17.400 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:17.400 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:17.400 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:17.409 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:42:17.481 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-17T22:42:17.497 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:17.494+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:17.501 INFO:teuthology.orchestra.run.smithi115.stdout:3341 write 0xab0278a thru 0xab087bf (0x6036 bytes) 2024-09-17T22:42:17.509 INFO:teuthology.orchestra.run.smithi115.stdout:3342 writesame 0xa6742ec thru 0xa680e3e data_size 0x1(0xcb53 bytes) 2024-09-17T22:42:17.518 INFO:teuthology.orchestra.run.smithi115.stdout:3343 read 0x827b906 thru 0x828712c (0xb827 bytes) 2024-09-17T22:42:17.519 INFO:teuthology.orchestra.run.smithi115.stdout:3344 write 0x6f904d6 thru 0x6f91e9e (0x19c9 bytes) 2024-09-17T22:42:17.522 INFO:teuthology.orchestra.run.smithi115.stdout:3347 read 0x5fb82ef thru 0x5fbd396 (0x50a8 bytes) 2024-09-17T22:42:17.523 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:17.523 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:17.523 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:17.523 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:17.523 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:17.528 INFO:teuthology.orchestra.run.smithi115.stdout:3348 write 0xb24e146 thru 0xb25ce43 (0xecfe bytes) 2024-09-17T22:42:17.531 INFO:teuthology.orchestra.run.smithi115.stdout:truncating to largest ever: 0xed58973 2024-09-17T22:42:17.533 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:42:17.536 INFO:teuthology.orchestra.run.smithi115.stdout:3349 trunc from 0xb25ce44 to 0xed58973 2024-09-17T22:42:17.541 INFO:teuthology.orchestra.run.smithi115.stdout:3350 write 0xa1b3892 thru 0xa1bbf29 (0x8698 bytes) 2024-09-17T22:42:17.543 INFO:teuthology.orchestra.run.smithi115.stdout:3352 write 0x8a29c49 thru 0x8a2e525 (0x48dd bytes) 2024-09-17T22:42:17.547 INFO:teuthology.orchestra.run.smithi115.stdout:3353 read 0x7e01c3a thru 0x7e0bc46 (0xa00d bytes) 2024-09-17T22:42:17.547 INFO:teuthology.orchestra.run.smithi115.stdout:3354 writesame 0xecbf0bf thru 0xecc0821 data_size 0x1(0x1763 bytes) 2024-09-17T22:42:17.551 INFO:teuthology.orchestra.run.smithi115.stdout:3357 writesame 0x7805c2f thru 0x780a5bc data_size 0xe(0x498e bytes) 2024-09-17T22:42:17.555 INFO:teuthology.orchestra.run.smithi115.stdout:3358 punch from 0xc785701 to 0xc7910cb, (0xb9ca bytes) 2024-09-17T22:42:17.555 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:17.555 INFO:teuthology.orchestra.run.smithi115.stdout:3360 punch from 0xbc56079 to 0xbc5dfff, (0x7f86 bytes) 2024-09-17T22:42:17.556 INFO:teuthology.orchestra.run.smithi115.stdout:3362 write 0x2b89490 thru 0x2b96c7a (0xd7eb bytes) 2024-09-17T22:42:17.575 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:17.576 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:17.577 INFO:teuthology.orchestra.run.smithi115.stdout:3367 trunc from 0xed58973 to 0xd99c14b 2024-09-17T22:42:17.587 INFO:teuthology.orchestra.run.smithi115.stdout:3370 writesame 0x198b814 thru 0x19915fe data_size 0x1(0x5deb bytes) 2024-09-17T22:42:17.618 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:17.616+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:17.623 INFO:teuthology.orchestra.run.smithi115.stdout:3371 read 0x34e9b4e thru 0x34f8240 (0xe6f3 bytes) 2024-09-17T22:42:17.629 INFO:teuthology.orchestra.run.smithi115.stdout:3372 trunc from 0xd99c14b to 0x93f1984 2024-09-17T22:42:17.650 INFO:teuthology.orchestra.run.smithi115.stdout:3373 read 0x46cb14 thru 0x46f84f (0x2d3c bytes) 2024-09-17T22:42:17.651 INFO:teuthology.orchestra.run.smithi115.stdout:3374 read 0x10a6bb0 thru 0x10adfe9 (0x743a bytes) 2024-09-17T22:42:17.652 INFO:teuthology.orchestra.run.smithi115.stdout:3375 punch from 0x8a91d0d to 0x8a98842, (0x6b35 bytes) 2024-09-17T22:42:17.652 INFO:teuthology.orchestra.run.smithi115.stdout:3376 writesame 0x831ea26 thru 0x832b6c0 data_size 0x1(0xcc9b bytes) 2024-09-17T22:42:17.664 INFO:teuthology.orchestra.run.smithi115.stdout:3377 trunc from 0x93f1984 to 0xbb276b5 2024-09-17T22:42:17.670 INFO:teuthology.orchestra.run.smithi115.stdout:3379 read 0x159566b thru 0x15a3c71 (0xe607 bytes) 2024-09-17T22:42:17.671 INFO:teuthology.orchestra.run.smithi115.stdout:3380 read 0x5b21836 thru 0x5b316a8 (0xfe73 bytes) 2024-09-17T22:42:17.671 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:17.671 INFO:teuthology.orchestra.run.smithi115.stdout:3384 read 0x8a1032c thru 0x8a10769 (0x43e bytes) 2024-09-17T22:42:17.672 INFO:teuthology.orchestra.run.smithi115.stdout:3386 write 0xaca7721 thru 0xacb3d2f (0xc60f bytes) 2024-09-17T22:42:17.682 INFO:teuthology.orchestra.run.smithi115.stdout:3387 read 0x6ab3b1e thru 0x6abcae5 (0x8fc8 bytes) 2024-09-17T22:42:17.683 INFO:teuthology.orchestra.run.smithi115.stdout:3388 write 0x8d63de7 thru 0x8d739cc (0xfbe6 bytes) 2024-09-17T22:42:17.685 INFO:teuthology.orchestra.run.smithi115.stdout:3389 write 0x61d469a thru 0x61e2896 (0xe1fd bytes) 2024-09-17T22:42:17.695 INFO:teuthology.orchestra.run.smithi115.stdout:3391 punch from 0x2e7f4aa to 0x2e80278, (0xdce bytes) 2024-09-17T22:42:17.695 INFO:teuthology.orchestra.run.smithi115.stdout:3392 punch from 0x9692f2d to 0x96a0078, (0xd14b bytes) 2024-09-17T22:42:17.695 INFO:teuthology.orchestra.run.smithi115.stdout:3393 read 0xa0cd91e thru 0xa0cf137 (0x181a bytes) 2024-09-17T22:42:17.696 INFO:teuthology.orchestra.run.smithi115.stdout:3395 punch from 0x2bbc8a4 to 0x2bc5cc0, (0x941c bytes) 2024-09-17T22:42:17.696 INFO:teuthology.orchestra.run.smithi115.stdout:3396 read 0x428cfed thru 0x428dfa5 (0xfb9 bytes) 2024-09-17T22:42:17.699 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:17.696+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:17.703 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:17.703 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:17.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:17.703 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:17.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.703 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158673 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.703 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:17.703 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458427830000002, 2024-09-17T22:42:17.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039400614, 2024-09-17T22:42:17.703 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.704 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.704 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.704 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.704 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.704 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.704 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158673 2024-09-17T22:42:17.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.704 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:17.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:17.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:17.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.04e-07 2024-09-17T22:42:17.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241563+0000", 2024-09-17T22:42:17.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5799999999999998e-07 2024-09-17T22:42:17.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241566+0000", 2024-09-17T22:42:17.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8279999999999999e-06 2024-09-17T22:42:17.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241568+0000", 2024-09-17T22:42:17.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2709999999999999e-06 2024-09-17T22:42:17.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280954+0000", 2024-09-17T22:42:17.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039386051999999998 2024-09-17T22:42:17.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.707 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280962+0000", 2024-09-17T22:42:17.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3010000000000007e-06 2024-09-17T22:42:17.707 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.707 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.708 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.708 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158675 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.708 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:17.708 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458418539, 2024-09-17T22:42:17.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039408033000000002, 2024-09-17T22:42:17.708 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.708 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.708 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.708 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.708 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158675 2024-09-17T22:42:17.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.709 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:17.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:17.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:17.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.36e-07 2024-09-17T22:42:17.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241572+0000", 2024-09-17T22:42:17.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6599999999999994e-07 2024-09-17T22:42:17.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241575+0000", 2024-09-17T22:42:17.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8389999999999998e-06 2024-09-17T22:42:17.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241577+0000", 2024-09-17T22:42:17.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4119999999999999e-06 2024-09-17T22:42:17.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280975+0000", 2024-09-17T22:42:17.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039397205999999997 2024-09-17T22:42:17.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280979+0000", 2024-09-17T22:42:17.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.374e-06 2024-09-17T22:42:17.712 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.712 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.712 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.712 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158677 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.712 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:17.712 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458396096000001, 2024-09-17T22:42:17.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039409307999999997, 2024-09-17T22:42:17.712 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.712 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.712 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.712 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.713 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.713 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.714 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158677 2024-09-17T22:42:17.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.714 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:17.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:17.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241594+0000", 2024-09-17T22:42:17.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:42:17.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241595+0000", 2024-09-17T22:42:17.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2729999999999999e-06 2024-09-17T22:42:17.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241602+0000", 2024-09-17T22:42:17.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:42:17.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241606+0000", 2024-09-17T22:42:17.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3019999999999999e-06 2024-09-17T22:42:17.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280994+0000", 2024-09-17T22:42:17.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039388484000000001 2024-09-17T22:42:17.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281003+0000", 2024-09-17T22:42:17.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5459999999999992e-06 2024-09-17T22:42:17.717 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.717 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.717 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.717 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158679 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.718 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:17.718 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458380418000001, 2024-09-17T22:42:17.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039413403999999999, 2024-09-17T22:42:17.718 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.718 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.718 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.718 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.718 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.718 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.718 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158679 2024-09-17T22:42:17.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.718 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:17.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:17.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:17.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4999999999999999e-07 2024-09-17T22:42:17.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241613+0000", 2024-09-17T22:42:17.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.207e-06 2024-09-17T22:42:17.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241616+0000", 2024-09-17T22:42:17.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.275e-06 2024-09-17T22:42:17.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241618+0000", 2024-09-17T22:42:17.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:42:17.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281017+0000", 2024-09-17T22:42:17.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039398385000000001 2024-09-17T22:42:17.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281022+0000", 2024-09-17T22:42:17.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.896e-06 2024-09-17T22:42:17.722 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.722 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.722 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.722 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158681 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.722 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:17.722 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458368239999999, 2024-09-17T22:42:17.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039425292000000001, 2024-09-17T22:42:17.722 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.722 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.722 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.723 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.723 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.723 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.723 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158681 2024-09-17T22:42:17.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.723 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:17.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:17.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:17.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:42:17.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241622+0000", 2024-09-17T22:42:17.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:42:17.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241625+0000", 2024-09-17T22:42:17.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9629999999999998e-06 2024-09-17T22:42:17.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241628+0000", 2024-09-17T22:42:17.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4339999999999999e-06 2024-09-17T22:42:17.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281039+0000", 2024-09-17T22:42:17.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039410672000000001 2024-09-17T22:42:17.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281047+0000", 2024-09-17T22:42:17.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9969999999999998e-06 2024-09-17T22:42:17.726 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.726 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.726 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.727 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158682 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.727 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:17.727 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458358956000001, 2024-09-17T22:42:17.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039436455000000002, 2024-09-17T22:42:17.727 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.727 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.727 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.727 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.727 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.727 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.727 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158682 2024-09-17T22:42:17.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.727 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:17.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:17.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:17.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:42:17.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241632+0000", 2024-09-17T22:42:17.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:42:17.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241635+0000", 2024-09-17T22:42:17.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8940000000000002e-06 2024-09-17T22:42:17.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241637+0000", 2024-09-17T22:42:17.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3659999999999999e-06 2024-09-17T22:42:17.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281061+0000", 2024-09-17T22:42:17.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039423664999999997 2024-09-17T22:42:17.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281067+0000", 2024-09-17T22:42:17.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.285e-06 2024-09-17T22:42:17.731 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.731 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.731 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.731 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158684 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.731 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:17.731 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458349609999999, 2024-09-17T22:42:17.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462980000000002, 2024-09-17T22:42:17.731 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.731 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.731 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.731 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.731 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.732 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.732 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158684 2024-09-17T22:42:17.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.732 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:17.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:17.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:17.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:17.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241641+0000", 2024-09-17T22:42:17.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:42:17.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241644+0000", 2024-09-17T22:42:17.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8870000000000001e-06 2024-09-17T22:42:17.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241646+0000", 2024-09-17T22:42:17.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3530000000000002e-06 2024-09-17T22:42:17.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281093+0000", 2024-09-17T22:42:17.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039446947000000003 2024-09-17T22:42:17.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281103+0000", 2024-09-17T22:42:17.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6299999999999993e-06 2024-09-17T22:42:17.735 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.735 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.735 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.735 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158686 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.735 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:17.735 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458340440000001, 2024-09-17T22:42:17.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039474214000000001, 2024-09-17T22:42:17.735 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.735 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.735 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.736 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.736 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.736 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.736 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158686 2024-09-17T22:42:17.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.736 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:17.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:17.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:17.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:42:17.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241650+0000", 2024-09-17T22:42:17.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2999999999999999e-07 2024-09-17T22:42:17.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241653+0000", 2024-09-17T22:42:17.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9689999999999999e-06 2024-09-17T22:42:17.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241655+0000", 2024-09-17T22:42:17.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3460000000000001e-06 2024-09-17T22:42:17.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281118+0000", 2024-09-17T22:42:17.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462876000000001 2024-09-17T22:42:17.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281123+0000", 2024-09-17T22:42:17.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8840000000000002e-06 2024-09-17T22:42:17.739 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.739 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.739 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.739 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158688 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.739 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:17.739 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458331209000001, 2024-09-17T22:42:17.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039490458999999999, 2024-09-17T22:42:17.740 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.740 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.740 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.740 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.740 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.740 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.740 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158688 2024-09-17T22:42:17.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.740 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:17.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:17.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:17.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:42:17.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241659+0000", 2024-09-17T22:42:17.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3699999999999999e-07 2024-09-17T22:42:17.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241662+0000", 2024-09-17T22:42:17.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8499999999999998e-06 2024-09-17T22:42:17.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241665+0000", 2024-09-17T22:42:17.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.356e-06 2024-09-17T22:42:17.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281141+0000", 2024-09-17T22:42:17.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039476031000000002 2024-09-17T22:42:17.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281149+0000", 2024-09-17T22:42:17.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0709999999999995e-06 2024-09-17T22:42:17.743 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.743 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.743 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.744 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158690 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.744 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:17.744 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458322150000001, 2024-09-17T22:42:17.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039502378999999997, 2024-09-17T22:42:17.744 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.744 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.744 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.744 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.744 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.744 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.744 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158690 2024-09-17T22:42:17.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.744 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:17.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:17.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241668+0000", 2024-09-17T22:42:17.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:17.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241669+0000", 2024-09-17T22:42:17.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1009999999999999e-06 2024-09-17T22:42:17.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241672+0000", 2024-09-17T22:42:17.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9059999999999998e-06 2024-09-17T22:42:17.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241674+0000", 2024-09-17T22:42:17.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4439999999999998e-06 2024-09-17T22:42:17.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281162+0000", 2024-09-17T22:42:17.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039488189 2024-09-17T22:42:17.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281170+0000", 2024-09-17T22:42:17.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5310000000000004e-06 2024-09-17T22:42:17.747 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.748 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.748 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.748 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158692 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.748 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:17.748 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458312716999998, 2024-09-17T22:42:17.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039531064999999997, 2024-09-17T22:42:17.748 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.748 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.748 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.748 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.748 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.749 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.749 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158692 2024-09-17T22:42:17.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.749 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:17.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:17.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:17.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:17.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241678+0000", 2024-09-17T22:42:17.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0800000000000003e-07 2024-09-17T22:42:17.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241681+0000", 2024-09-17T22:42:17.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8729999999999999e-06 2024-09-17T22:42:17.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241683+0000", 2024-09-17T22:42:17.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.447e-06 2024-09-17T22:42:17.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281199+0000", 2024-09-17T22:42:17.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039515388999999998 2024-09-17T22:42:17.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281208+0000", 2024-09-17T22:42:17.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2399999999999996e-06 2024-09-17T22:42:17.752 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.752 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.752 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.752 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158694 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.752 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:17.752 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458303546, 2024-09-17T22:42:17.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039543986000000003, 2024-09-17T22:42:17.753 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.753 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.753 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.753 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.753 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.753 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.753 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158694 2024-09-17T22:42:17.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.753 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:17.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:17.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:17.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:42:17.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241687+0000", 2024-09-17T22:42:17.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:42:17.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241691+0000", 2024-09-17T22:42:17.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.811e-06 2024-09-17T22:42:17.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241693+0000", 2024-09-17T22:42:17.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.385e-06 2024-09-17T22:42:17.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281223+0000", 2024-09-17T22:42:17.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039529332 2024-09-17T22:42:17.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281230+0000", 2024-09-17T22:42:17.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2139999999999999e-06 2024-09-17T22:42:17.756 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.756 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.756 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.757 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158696 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.757 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:17.757 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458293158, 2024-09-17T22:42:17.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039570458000000003, 2024-09-17T22:42:17.757 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.757 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.757 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.757 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.757 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.757 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.757 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158696 2024-09-17T22:42:17.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.757 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:17.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:17.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:42:17.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:42:17.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:42:17.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.33e-07 2024-09-17T22:42:17.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241700+0000", 2024-09-17T22:42:17.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8949999999999998e-06 2024-09-17T22:42:17.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241703+0000", 2024-09-17T22:42:17.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3159999999999999e-06 2024-09-17T22:42:17.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281257+0000", 2024-09-17T22:42:17.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039553862000000002 2024-09-17T22:42:17.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281267+0000", 2024-09-17T22:42:17.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.024e-05 2024-09-17T22:42:17.760 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.760 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.761 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.761 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158698 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.761 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:17.761 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458283080000001, 2024-09-17T22:42:17.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039576493999999997, 2024-09-17T22:42:17.761 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.761 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.761 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.761 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.761 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.762 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.762 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158698 2024-09-17T22:42:17.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.762 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:17.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.763 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:17.763 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.763 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.763 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241707+0000", 2024-09-17T22:42:17.763 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:42:17.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.763 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.763 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241708+0000", 2024-09-17T22:42:17.763 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5000000000000001e-07 2024-09-17T22:42:17.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241711+0000", 2024-09-17T22:42:17.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9670000000000001e-06 2024-09-17T22:42:17.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241713+0000", 2024-09-17T22:42:17.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2689999999999998e-06 2024-09-17T22:42:17.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281278+0000", 2024-09-17T22:42:17.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039565206999999998 2024-09-17T22:42:17.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281283+0000", 2024-09-17T22:42:17.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:42:17.765 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.765 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.765 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.765 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158700 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.765 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:17.765 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458273868999999, 2024-09-17T22:42:17.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039604292999999999, 2024-09-17T22:42:17.766 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.766 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.766 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.766 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.766 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.766 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.766 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158700 2024-09-17T22:42:17.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.766 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:17.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:17.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:17.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:17.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.768 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241717+0000", 2024-09-17T22:42:17.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.78e-07 2024-09-17T22:42:17.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.768 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.768 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241720+0000", 2024-09-17T22:42:17.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8119999999999999e-06 2024-09-17T22:42:17.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.768 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.768 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241722+0000", 2024-09-17T22:42:17.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-17T22:42:17.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281311+0000", 2024-09-17T22:42:17.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039588894999999999 2024-09-17T22:42:17.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281320+0000", 2024-09-17T22:42:17.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9989999999999997e-06 2024-09-17T22:42:17.769 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.769 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.769 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.770 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158702 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.770 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:17.770 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458262547, 2024-09-17T22:42:17.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039616142, 2024-09-17T22:42:17.770 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.770 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.770 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.770 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.770 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.770 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.770 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158702 2024-09-17T22:42:17.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.771 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:17.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:17.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:17.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4400000000000001e-07 2024-09-17T22:42:17.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241728+0000", 2024-09-17T22:42:17.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0839999999999999e-06 2024-09-17T22:42:17.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241731+0000", 2024-09-17T22:42:17.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0690000000000002e-06 2024-09-17T22:42:17.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241734+0000", 2024-09-17T22:42:17.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4159999999999998e-06 2024-09-17T22:42:17.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281334+0000", 2024-09-17T22:42:17.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039600110000000001 2024-09-17T22:42:17.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281343+0000", 2024-09-17T22:42:17.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2189999999999993e-06 2024-09-17T22:42:17.774 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.774 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.774 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.774 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158704 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.774 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:17.774 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458252683000001, 2024-09-17T22:42:17.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039641485999999997, 2024-09-17T22:42:17.774 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.774 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.774 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.774 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.775 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.775 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.775 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158704 2024-09-17T22:42:17.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.775 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:17.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:17.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:17.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:42:17.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241738+0000", 2024-09-17T22:42:17.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0589999999999999e-06 2024-09-17T22:42:17.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241741+0000", 2024-09-17T22:42:17.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.943e-06 2024-09-17T22:42:17.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241744+0000", 2024-09-17T22:42:17.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4819999999999999e-06 2024-09-17T22:42:17.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281370+0000", 2024-09-17T22:42:17.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039625661 2024-09-17T22:42:17.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281378+0000", 2024-09-17T22:42:17.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1310000000000006e-06 2024-09-17T22:42:17.778 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.778 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.778 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.778 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158705 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.778 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:17.779 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458242132999999, 2024-09-17T22:42:17.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039652637999999997, 2024-09-17T22:42:17.779 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.779 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.779 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.779 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.779 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.779 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.779 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158705 2024-09-17T22:42:17.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.779 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:17.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:17.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:42:17.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-07 2024-09-17T22:42:17.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:42:17.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0599999999999999e-07 2024-09-17T22:42:17.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241751+0000", 2024-09-17T22:42:17.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.926e-06 2024-09-17T22:42:17.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241754+0000", 2024-09-17T22:42:17.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4360000000000001e-06 2024-09-17T22:42:17.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281392+0000", 2024-09-17T22:42:17.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039638566 2024-09-17T22:42:17.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281400+0000", 2024-09-17T22:42:17.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.571e-06 2024-09-17T22:42:17.782 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.783 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.783 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.783 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158707 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.783 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:17.783 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458221032000001, 2024-09-17T22:42:17.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039654809999999999, 2024-09-17T22:42:17.783 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.783 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.783 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.783 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.783 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.784 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.784 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158707 2024-09-17T22:42:17.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.784 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:17.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:17.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241769+0000", 2024-09-17T22:42:17.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4299999999999999e-07 2024-09-17T22:42:17.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241770+0000", 2024-09-17T22:42:17.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3769999999999999e-06 2024-09-17T22:42:17.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241773+0000", 2024-09-17T22:42:17.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2109999999999998e-06 2024-09-17T22:42:17.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241776+0000", 2024-09-17T22:42:17.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5069999999999999e-06 2024-09-17T22:42:17.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281415+0000", 2024-09-17T22:42:17.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039639499000000002 2024-09-17T22:42:17.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.787 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281423+0000", 2024-09-17T22:42:17.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8730000000000007e-06 2024-09-17T22:42:17.787 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.787 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.787 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.787 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158709 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:17.787 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:17.787 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.458210423000001, 2024-09-17T22:42:17.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039661937000000001, 2024-09-17T22:42:17.787 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:17.788 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:17.788 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:17.788 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:17.788 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:17.788 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:17.788 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158709 2024-09-17T22:42:17.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.788 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:17.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:17.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:17.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:17.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:17.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:17.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:17.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242843+0000", 2024-09-17T22:42:17.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010634780000000001 2024-09-17T22:42:17.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:17.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242845+0000", 2024-09-17T22:42:17.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7209999999999999e-06 2024-09-17T22:42:17.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:17.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242851+0000", 2024-09-17T22:42:17.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6949999999999997e-06 2024-09-17T22:42:17.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:17.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242855+0000", 2024-09-17T22:42:17.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7960000000000002e-06 2024-09-17T22:42:17.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:17.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281434+0000", 2024-09-17T22:42:17.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038579519 2024-09-17T22:42:17.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:17.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:17.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:17.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281441+0000", 2024-09-17T22:42:17.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7279999999999998e-06 2024-09-17T22:42:17.791 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.791 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.791 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.791 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:17.791 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:17.791 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:17.793 INFO:teuthology.orchestra.run.smithi115.stdout:3398 trunc from 0xbb276b5 to 0x139bdc8 2024-09-17T22:42:17.793 INFO:teuthology.orchestra.run.smithi115.stdout:3399 write 0x39df75c thru 0x39e14ec (0x1d91 bytes) 2024-09-17T22:42:17.793 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:17.793 INFO:teuthology.orchestra.run.smithi115.stdout:3401 write 0x3a17cfe thru 0x3a1fbcc (0x7ecf bytes) 2024-09-17T22:42:17.793 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:42:17.800 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:17.797+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:17.803 INFO:teuthology.orchestra.run.smithi115.stdout:3403 writesame 0x57ffbde thru 0x5801bd8 data_size 0x1(0x1ffb bytes) 2024-09-17T22:42:17.803 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:17.803 INFO:teuthology.orchestra.run.smithi115.stdout:3405 write 0x1060ba2 thru 0x106562c (0x4a8b bytes) 2024-09-17T22:42:17.810 INFO:teuthology.orchestra.run.smithi115.stdout:3406 trunc from 0x5801bd9 to 0xa85397a 2024-09-17T22:42:17.816 INFO:teuthology.orchestra.run.smithi115.stdout:3407 punch from 0x69d8426 to 0x69dc4dd, (0x40b7 bytes) 2024-09-17T22:42:17.816 INFO:teuthology.orchestra.run.smithi115.stdout:3408 punch from 0x377428e to 0x377c4b2, (0x8224 bytes) 2024-09-17T22:42:17.816 INFO:teuthology.orchestra.run.smithi115.stdout:3410 read 0x502cc6f thru 0x5039a22 (0xcdb4 bytes) 2024-09-17T22:42:17.817 INFO:teuthology.orchestra.run.smithi115.stdout:3411 writesame 0x482f247 thru 0x483b711 data_size 0x7(0xc4cb bytes) 2024-09-17T22:42:17.823 INFO:teuthology.orchestra.run.smithi115.stdout:3412 write 0x45cd843 thru 0x45d03fa (0x2bb8 bytes) 2024-09-17T22:42:17.825 INFO:teuthology.orchestra.run.smithi115.stdout:3414 punch from 0x3ac5358 to 0x3acdc4f, (0x88f7 bytes) 2024-09-17T22:42:17.825 INFO:teuthology.orchestra.run.smithi115.stdout:3417 write 0x9f1931b thru 0x9f26782 (0xd468 bytes) 2024-09-17T22:42:17.830 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:17.830 INFO:teuthology.orchestra.run.smithi115.stdout:3419 write 0x2212449 thru 0x221c7a0 (0xa358 bytes) 2024-09-17T22:42:17.834 INFO:teuthology.orchestra.run.smithi115.stdout:3420 trunc from 0xa85397a to 0x9bfe8a5 2024-09-17T22:42:17.840 INFO:teuthology.orchestra.run.smithi115.stdout:3421 write 0x7e260df thru 0x7e28d34 (0x2c56 bytes) 2024-09-17T22:42:17.851 INFO:teuthology.orchestra.run.smithi115.stdout:3422 write 0xb4a91bd thru 0xb4add2c (0x4b70 bytes) 2024-09-17T22:42:17.854 INFO:teuthology.orchestra.run.smithi115.stdout:3426 read 0x103088d thru 0x1031960 (0x10d4 bytes) 2024-09-17T22:42:17.854 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:17.855 INFO:teuthology.orchestra.run.smithi115.stdout:3429 write 0x2aad098 thru 0x2aae906 (0x186f bytes) 2024-09-17T22:42:17.857 INFO:teuthology.orchestra.run.smithi115.stdout:3430 write 0x45274f thru 0x461892 (0xf144 bytes) 2024-09-17T22:42:17.859 INFO:teuthology.orchestra.run.smithi115.stdout:3431 read 0x7136363 thru 0x71420b2 (0xbd50 bytes) 2024-09-17T22:42:17.859 INFO:teuthology.orchestra.run.smithi115.stdout:3432 punch from 0x744d71c to 0x744ee38, (0x171c bytes) 2024-09-17T22:42:17.860 INFO:teuthology.orchestra.run.smithi115.stdout:3433 read 0x88b6ac1 thru 0x88b8515 (0x1a55 bytes) 2024-09-17T22:42:17.860 INFO:teuthology.orchestra.run.smithi115.stdout:3434 write 0x1f390b2 thru 0x1f39dba (0xd09 bytes) 2024-09-17T22:42:17.860 INFO:teuthology.orchestra.run.smithi115.stdout:3436 read 0x677b5d4 thru 0x677f34f (0x3d7c bytes) 2024-09-17T22:42:17.861 INFO:teuthology.orchestra.run.smithi115.stdout:3437 write 0x9bc1a4b thru 0x9bc2e38 (0x13ee bytes) 2024-09-17T22:42:17.868 INFO:teuthology.orchestra.run.smithi115.stdout:3439 write 0xbe5e6a7 thru 0xbe6b8d8 (0xd232 bytes) 2024-09-17T22:42:17.876 INFO:teuthology.orchestra.run.smithi115.stdout:3440 writesame 0x933e489 thru 0x933f969 data_size 0x1(0x14e1 bytes) 2024-09-17T22:42:17.886 INFO:teuthology.orchestra.run.smithi115.stdout:3441 write 0xdcbb52b thru 0xdcc7824 (0xc2fa bytes) 2024-09-17T22:42:17.896 INFO:teuthology.orchestra.run.smithi115.stdout:3442 write 0x8e43bbc thru 0x8e53179 (0xf5be bytes) 2024-09-17T22:42:17.900 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:17.898+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:17.904 INFO:teuthology.orchestra.run.smithi115.stdout:3443 read 0x8eb6a58 thru 0x8ec3a14 (0xcfbd bytes) 2024-09-17T22:42:17.904 INFO:teuthology.orchestra.run.smithi115.stdout:3444 writesame 0x45c371c thru 0x45d15e5 data_size 0x1(0xdeca bytes) 2024-09-17T22:42:17.921 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:17.922 INFO:teuthology.orchestra.run.smithi115.stdout:3446 read 0xb166146 thru 0xb17531c (0xf1d7 bytes) 2024-09-17T22:42:17.924 INFO:teuthology.orchestra.run.smithi115.stdout:3448 trunc from 0xdcc7825 to 0xbdba21e 2024-09-17T22:42:17.939 INFO:teuthology.orchestra.run.smithi115.stdout:3449 write 0x439b37b thru 0x43a9d7b (0xea01 bytes) 2024-09-17T22:42:17.943 INFO:teuthology.orchestra.run.smithi115.stdout:3450 write 0x1b26ccd thru 0x1b2ebd7 (0x7f0b bytes) 2024-09-17T22:42:17.945 INFO:teuthology.orchestra.run.smithi115.stdout:3452 write 0xbc1ea91 thru 0xbc229a4 (0x3f14 bytes) 2024-09-17T22:42:17.948 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:17.948 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:17.948 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:17.948 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:17.952 INFO:teuthology.orchestra.run.smithi115.stdout:3453 write 0xc6566f0 thru 0xc660e8c (0xa79d bytes) 2024-09-17T22:42:17.957 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:42:17.971 INFO:teuthology.orchestra.run.smithi115.stdout:3454 writesame 0xe74d5b8 thru 0xe75c2ae data_size 0x49(0xecf7 bytes) 2024-09-17T22:42:17.971 INFO:teuthology.orchestra.run.smithi115.stdout:3455 write 0x6564f5d thru 0x656b06e (0x6112 bytes) 2024-09-17T22:42:17.982 INFO:teuthology.orchestra.run.smithi115.stdout:3457 trunc from 0xe75c2af to 0x6dfdb3f 2024-09-17T22:42:17.994 INFO:teuthology.orchestra.run.smithi115.stdout:3458 write 0x23547ce thru 0x23618c8 (0xd0fb bytes) 2024-09-17T22:42:17.996 INFO:teuthology.orchestra.run.smithi115.stdout:3459 read 0x1e93608 thru 0x1e99333 (0x5d2c bytes) 2024-09-17T22:42:17.996 INFO:teuthology.orchestra.run.smithi115.stdout:3460 punch from 0x594dc63 to 0x5952fd2, (0x536f bytes) 2024-09-17T22:42:17.996 INFO:teuthology.orchestra.run.smithi115.stdout:3461 write 0x40277f7 thru 0x4036e68 (0xf672 bytes) 2024-09-17T22:42:18.006 INFO:teuthology.orchestra.run.smithi115.stdout:3464 write 0x82cceaa thru 0x82d2f11 (0x6068 bytes) 2024-09-17T22:42:18.009 INFO:teuthology.orchestra.run.smithi115.stdout:3465 read 0x10363 thru 0x1ee59 (0xeaf7 bytes) 2024-09-17T22:42:18.009 INFO:teuthology.orchestra.run.smithi115.stdout:3467 read 0x2111f60 thru 0x211a13e (0x81df bytes) 2024-09-17T22:42:18.010 INFO:teuthology.orchestra.run.smithi115.stdout:3468 read 0x7bdbc thru 0x877de (0xba23 bytes) 2024-09-17T22:42:18.011 INFO:teuthology.orchestra.run.smithi115.stdout:3469 punch from 0x416aec3 to 0x417245a, (0x7597 bytes) 2024-09-17T22:42:18.011 INFO:teuthology.orchestra.run.smithi115.stdout:3470 read 0x1ad80d9 thru 0x1ae2b4d (0xaa75 bytes) 2024-09-17T22:42:18.012 INFO:teuthology.orchestra.run.smithi115.stdout:3471 read 0x78b46cc thru 0x78b9fd3 (0x5908 bytes) 2024-09-17T22:42:18.012 INFO:teuthology.orchestra.run.smithi115.stdout:3472 read 0x44628c thru 0x446295 (0xa bytes) 2024-09-17T22:42:18.013 INFO:teuthology.orchestra.run.smithi115.stdout:3473 trunc from 0x82d2f12 to 0x773f831 2024-09-17T22:42:18.024 INFO:teuthology.orchestra.run.smithi115.stdout:3474 read 0x3b9d25f thru 0x3bace42 (0xfbe4 bytes) 2024-09-17T22:42:18.039 INFO:teuthology.orchestra.run.smithi115.stdout:3476 write 0xe398fda thru 0xe3a04ab (0x74d2 bytes) 2024-09-17T22:42:18.041 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:18.039+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:18.044 INFO:teuthology.orchestra.run.smithi115.stdout:3477 write 0x751e292 thru 0x7522735 (0x44a4 bytes) 2024-09-17T22:42:18.051 INFO:teuthology.orchestra.run.smithi115.stdout:3478 trunc from 0xe3a04ac to 0xe083f72 2024-09-17T22:42:18.059 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:18.059 INFO:teuthology.orchestra.run.smithi115.stdout:3482 writesame 0x2e3af0a thru 0x2e41391 data_size 0x4(0x6488 bytes) 2024-09-17T22:42:18.063 INFO:teuthology.orchestra.run.smithi115.stdout:3483 punch from 0x509527b to 0x50a0caa, (0xba2f bytes) 2024-09-17T22:42:18.063 INFO:teuthology.orchestra.run.smithi115.stdout:3484 punch from 0xa8b7ab1 to 0xa8ba3a5, (0x28f4 bytes) 2024-09-17T22:42:18.063 INFO:teuthology.orchestra.run.smithi115.stdout:3487 writesame 0xdc7d372 thru 0xdc8166b data_size 0x1(0x42fa bytes) 2024-09-17T22:42:18.074 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:18.074 INFO:teuthology.orchestra.run.smithi115.stdout:3491 write 0x9efc45f thru 0x9f0b376 (0xef18 bytes) 2024-09-17T22:42:18.081 INFO:teuthology.orchestra.run.smithi115.stdout:3492 write 0xd22be28 thru 0xd22e004 (0x21dd bytes) 2024-09-17T22:42:18.085 INFO:teuthology.orchestra.run.smithi115.stdout:3493 read 0x166dd50 thru 0x1672cc2 (0x4f73 bytes) 2024-09-17T22:42:18.085 INFO:teuthology.orchestra.run.smithi115.stdout:3494 writesame 0x20a5c0f thru 0x20b09fd data_size 0x1(0xadef bytes) 2024-09-17T22:42:18.094 INFO:teuthology.orchestra.run.smithi115.stdout:3495 write 0x4e72835 thru 0x4e73fe8 (0x17b4 bytes) 2024-09-17T22:42:18.097 INFO:teuthology.orchestra.run.smithi115.stdout:3496 trunc from 0xe083f72 to 0xd599291 2024-09-17T22:42:18.101 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:18.098+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:18.110 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:18.111 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:18.111 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:18.111 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:18.111 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:18.117 INFO:teuthology.orchestra.run.smithi115.stdout:3498 write 0xe11a117 thru 0xe128f40 (0xee2a bytes) 2024-09-17T22:42:18.121 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:42:18.127 INFO:teuthology.orchestra.run.smithi115.stdout:3499 read 0x1d5f536 thru 0x1d6a15f (0xac2a bytes) 2024-09-17T22:42:18.127 INFO:teuthology.orchestra.run.smithi115.stdout:3500 punch from 0x6c24bb2 to 0x6c2d902, (0x8d50 bytes) 2024-09-17T22:42:18.128 INFO:teuthology.orchestra.run.smithi115.stdout:3502 punch from 0x9916f6c to 0x99195c7, (0x265b bytes) 2024-09-17T22:42:18.128 INFO:teuthology.orchestra.run.smithi115.stdout:3504 read 0xacaf4e6 thru 0xacb8163 (0x8c7e bytes) 2024-09-17T22:42:18.128 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:18.128 INFO:teuthology.orchestra.run.smithi115.stdout:3506 read 0x4e0cff8 thru 0x4e11504 (0x450d bytes) 2024-09-17T22:42:18.128 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:18.128 INFO:teuthology.orchestra.run.smithi115.stdout:3508 writesame 0xbe81650 thru 0xbe904d5 data_size 0x3(0xee86 bytes) 2024-09-17T22:42:18.141 INFO:teuthology.orchestra.run.smithi115.stdout:3509 write 0xc1e84b5 thru 0xc1f7520 (0xf06c bytes) 2024-09-17T22:42:18.145 INFO:teuthology.orchestra.run.smithi115.stdout:3513 write 0x1fd8e02 thru 0x1fe89b7 (0xfbb6 bytes) 2024-09-17T22:42:18.148 INFO:teuthology.orchestra.run.smithi115.stdout:3514 write 0x9ef5e86 thru 0x9effaaf (0x9c2a bytes) 2024-09-17T22:42:18.151 INFO:teuthology.orchestra.run.smithi115.stdout:3518 read 0x2331732 thru 0x2332110 (0x9df bytes) 2024-09-17T22:42:18.151 INFO:teuthology.orchestra.run.smithi115.stdout:3519 writesame 0xd35d686 thru 0xd35ecfb data_size 0x2e(0x1676 bytes) 2024-09-17T22:42:18.155 INFO:teuthology.orchestra.run.smithi115.stdout:3520 trunc from 0xe128f41 to 0xc22dc50 2024-09-17T22:42:18.168 INFO:teuthology.orchestra.run.smithi115.stdout:3522 punch from 0x791055 to 0x79ab90, (0x9b3b bytes) 2024-09-17T22:42:18.168 INFO:teuthology.orchestra.run.smithi115.stdout:3524 punch from 0x17a58ee to 0x17b2383, (0xca95 bytes) 2024-09-17T22:42:18.168 INFO:teuthology.orchestra.run.smithi115.stdout:3525 read 0x6557705 thru 0x655c3f7 (0x4cf3 bytes) 2024-09-17T22:42:18.182 INFO:teuthology.orchestra.run.smithi115.stdout:3526 trunc from 0xc22dc50 to 0xb6b8de 2024-09-17T22:42:18.202 INFO:teuthology.orchestra.run.smithi115.stdout:3527 read 0x6c0e8b thru 0x6ce149 (0xd2bf bytes) 2024-09-17T22:42:18.202 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:18.199+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:18.214 INFO:teuthology.orchestra.run.smithi115.stdout:3528 write 0x5563943 thru 0x556439e (0xa5c bytes) 2024-09-17T22:42:18.221 INFO:teuthology.orchestra.run.smithi115.stdout:3529 read 0x3678ba3 thru 0x367d03e (0x449c bytes) 2024-09-17T22:42:18.221 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:18.221 INFO:teuthology.orchestra.run.smithi115.stdout:3532 read 0xc96e64 thru 0xc9dd2b (0x6ec8 bytes) 2024-09-17T22:42:18.221 INFO:teuthology.orchestra.run.smithi115.stdout:3533 write 0x1706005 thru 0x1712076 (0xc072 bytes) 2024-09-17T22:42:18.223 INFO:teuthology.orchestra.run.smithi115.stdout:3534 write 0x138fb0b thru 0x1397435 (0x792b bytes) 2024-09-17T22:42:18.237 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:42:18.237 INFO:teuthology.orchestra.run.smithi115.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-17T22:42:18.246 INFO:teuthology.orchestra.run.smithi115.stdout:3537 write 0xec1076d thru 0xec16f98 (0x682c bytes) 2024-09-17T22:42:18.251 INFO:teuthology.orchestra.run.smithi115.stdout:3539 write 0x3b75321 thru 0x3b780b0 (0x2d90 bytes) 2024-09-17T22:42:18.255 INFO:teuthology.orchestra.run.smithi115.stdout:3541 read 0x3077c2 thru 0x315fb6 (0xe7f5 bytes) 2024-09-17T22:42:18.255 INFO:teuthology.orchestra.run.smithi115.stdout:3542 write 0x9394423 thru 0x939ca01 (0x85df bytes) 2024-09-17T22:42:18.260 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:18.260 INFO:teuthology.orchestra.run.smithi115.stdout:3545 read 0x65d0931 thru 0x65d24ac (0x1b7c bytes) 2024-09-17T22:42:18.261 INFO:teuthology.orchestra.run.smithi115.stdout:3546 read 0x3f09063 thru 0x3f13ee8 (0xae86 bytes) 2024-09-17T22:42:18.261 INFO:teuthology.orchestra.run.smithi115.stdout:3547 punch from 0x8ed71e2 to 0x8edde19, (0x6c37 bytes) 2024-09-17T22:42:18.261 INFO:teuthology.orchestra.run.smithi115.stdout:3548 writesame 0x99e95e5 thru 0x99ea93c data_size 0x4(0x1358 bytes) 2024-09-17T22:42:18.277 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:18.277 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:18.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:18.277 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:18.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.277 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157229 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4073472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.277 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:18.278 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.127572260000001, 2024-09-17T22:42:18.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041957959000000003, 2024-09-17T22:42:18.278 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.278 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.278 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.278 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.278 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.278 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.278 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157229 2024-09-17T22:42:18.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.278 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:18.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:18.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146207+0000", 2024-09-17T22:42:18.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0299999999999999e-07 2024-09-17T22:42:18.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146215+0000", 2024-09-17T22:42:18.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3059999999999998e-06 2024-09-17T22:42:18.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146222+0000", 2024-09-17T22:42:18.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1180000000000003e-06 2024-09-17T22:42:18.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146227+0000", 2024-09-17T22:42:18.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1749999999999998e-06 2024-09-17T22:42:18.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.187985+0000", 2024-09-17T22:42:18.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041758391999999998 2024-09-17T22:42:18.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188065+0000", 2024-09-17T22:42:18.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0290000000000005e-05 2024-09-17T22:42:18.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188164+0000", 2024-09-17T22:42:18.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9074999999999996e-05 2024-09-17T22:42:18.282 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.282 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.282 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.282 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157231 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4079616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.282 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:18.282 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.127434636, 2024-09-17T22:42:18.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042034581000000001, 2024-09-17T22:42:18.282 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.282 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.283 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.283 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.283 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.283 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.283 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157231 2024-09-17T22:42:18.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.283 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:18.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:18.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146345+0000", 2024-09-17T22:42:18.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:42:18.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146347+0000", 2024-09-17T22:42:18.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.357e-06 2024-09-17T22:42:18.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146352+0000", 2024-09-17T22:42:18.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T22:42:18.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146361+0000", 2024-09-17T22:42:18.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0219999999999993e-06 2024-09-17T22:42:18.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188199+0000", 2024-09-17T22:42:18.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041838130000000001 2024-09-17T22:42:18.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188285+0000", 2024-09-17T22:42:18.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5160000000000005e-05 2024-09-17T22:42:18.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.286 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.286 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188379+0000", 2024-09-17T22:42:18.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3949000000000005e-05 2024-09-17T22:42:18.286 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.286 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.287 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.287 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157233 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4085760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.287 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:18.287 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.127218487, 2024-09-17T22:42:18.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042008960999999997, 2024-09-17T22:42:18.287 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.287 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.287 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.287 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.288 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157233 2024-09-17T22:42:18.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.288 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:18.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:18.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146561+0000", 2024-09-17T22:42:18.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6300000000000005e-07 2024-09-17T22:42:18.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146564+0000", 2024-09-17T22:42:18.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-17T22:42:18.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146575+0000", 2024-09-17T22:42:18.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0891e-05 2024-09-17T22:42:18.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146579+0000", 2024-09-17T22:42:18.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.284e-06 2024-09-17T22:42:18.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188407+0000", 2024-09-17T22:42:18.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041828390999999999 2024-09-17T22:42:18.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188476+0000", 2024-09-17T22:42:18.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9129999999999997e-05 2024-09-17T22:42:18.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.291 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.291 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188569+0000", 2024-09-17T22:42:18.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2669000000000003e-05 2024-09-17T22:42:18.291 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.291 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.291 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.291 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157235 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4091904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.291 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:18.291 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.12702449, 2024-09-17T22:42:18.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042047517, 2024-09-17T22:42:18.292 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.292 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.292 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.292 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.292 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.292 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.292 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157235 2024-09-17T22:42:18.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.292 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:18.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:18.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146755+0000", 2024-09-17T22:42:18.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0900000000000001e-07 2024-09-17T22:42:18.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146758+0000", 2024-09-17T22:42:18.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.399e-06 2024-09-17T22:42:18.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146768+0000", 2024-09-17T22:42:18.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4730000000000006e-06 2024-09-17T22:42:18.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146774+0000", 2024-09-17T22:42:18.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6939999999999998e-06 2024-09-17T22:42:18.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188616+0000", 2024-09-17T22:42:18.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041841481999999999 2024-09-17T22:42:18.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188700+0000", 2024-09-17T22:42:18.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4227999999999994e-05 2024-09-17T22:42:18.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188802+0000", 2024-09-17T22:42:18.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101532 2024-09-17T22:42:18.295 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.296 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.296 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.296 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157237 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4098048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.296 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:18.296 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.126994219, 2024-09-17T22:42:18.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165452999999999, 2024-09-17T22:42:18.296 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.296 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.296 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.296 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.296 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.296 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.297 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157237 2024-09-17T22:42:18.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.297 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:18.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:18.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146785+0000", 2024-09-17T22:42:18.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6800000000000001e-07 2024-09-17T22:42:18.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146787+0000", 2024-09-17T22:42:18.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.046e-06 2024-09-17T22:42:18.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146794+0000", 2024-09-17T22:42:18.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9249999999999998e-06 2024-09-17T22:42:18.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146799+0000", 2024-09-17T22:42:18.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9289999999999997e-06 2024-09-17T22:42:18.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188844+0000", 2024-09-17T22:42:18.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042045386999999997 2024-09-17T22:42:18.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188888+0000", 2024-09-17T22:42:18.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3760000000000001e-05 2024-09-17T22:42:18.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188950+0000", 2024-09-17T22:42:18.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1938000000000002e-05 2024-09-17T22:42:18.300 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.300 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.300 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.300 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157239 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4104192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.300 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:18.300 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.126965833, 2024-09-17T22:42:18.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042347193999999998, 2024-09-17T22:42:18.301 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.301 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.301 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.301 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.301 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.301 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.301 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157239 2024-09-17T22:42:18.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.301 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:18.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:18.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:18.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:42:18.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146816+0000", 2024-09-17T22:42:18.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4700000000000001e-06 2024-09-17T22:42:18.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146822+0000", 2024-09-17T22:42:18.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.579e-06 2024-09-17T22:42:18.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146826+0000", 2024-09-17T22:42:18.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3780000000000001e-06 2024-09-17T22:42:18.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188991+0000", 2024-09-17T22:42:18.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165553000000001 2024-09-17T22:42:18.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189068+0000", 2024-09-17T22:42:18.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6548e-05 2024-09-17T22:42:18.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189160+0000", 2024-09-17T22:42:18.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2163000000000005e-05 2024-09-17T22:42:18.304 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.304 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.305 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.305 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157242 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4110336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.305 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:18.305 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.126928047, 2024-09-17T22:42:18.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042499783999999999, 2024-09-17T22:42:18.305 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.305 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.305 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.306 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157242 2024-09-17T22:42:18.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.306 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:18.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:18.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:18.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:42:18.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146852+0000", 2024-09-17T22:42:18.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.37e-06 2024-09-17T22:42:18.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146856+0000", 2024-09-17T22:42:18.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1690000000000002e-06 2024-09-17T22:42:18.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146863+0000", 2024-09-17T22:42:18.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9920000000000001e-06 2024-09-17T22:42:18.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189193+0000", 2024-09-17T22:42:18.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042329355999999999 2024-09-17T22:42:18.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189246+0000", 2024-09-17T22:42:18.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3643000000000001e-05 2024-09-17T22:42:18.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189350+0000", 2024-09-17T22:42:18.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010394 2024-09-17T22:42:18.309 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.309 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.309 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.309 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157244 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4116480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.309 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:18.310 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.126908835999998, 2024-09-17T22:42:18.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042675684999999998, 2024-09-17T22:42:18.310 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.310 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.310 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.310 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.310 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.310 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.310 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157244 2024-09-17T22:42:18.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.310 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:18.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:18.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:18.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1e-07 2024-09-17T22:42:18.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146871+0000", 2024-09-17T22:42:18.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.189e-06 2024-09-17T22:42:18.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146877+0000", 2024-09-17T22:42:18.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8100000000000003e-06 2024-09-17T22:42:18.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146880+0000", 2024-09-17T22:42:18.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.83e-06 2024-09-17T22:42:18.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189388+0000", 2024-09-17T22:42:18.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042507637000000001 2024-09-17T22:42:18.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189443+0000", 2024-09-17T22:42:18.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5742999999999998e-05 2024-09-17T22:42:18.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189545+0000", 2024-09-17T22:42:18.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102166 2024-09-17T22:42:18.314 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.314 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.314 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.314 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157246 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4122624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.314 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:18.314 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.126879784, 2024-09-17T22:42:18.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042847083000000001, 2024-09-17T22:42:18.314 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.314 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.314 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.314 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.314 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.315 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.315 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157246 2024-09-17T22:42:18.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.315 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:18.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:18.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:18.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0100000000000001e-07 2024-09-17T22:42:18.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146901+0000", 2024-09-17T22:42:18.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.531e-06 2024-09-17T22:42:18.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146905+0000", 2024-09-17T22:42:18.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0759999999999996e-06 2024-09-17T22:42:18.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146908+0000", 2024-09-17T22:42:18.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.083e-06 2024-09-17T22:42:18.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189585+0000", 2024-09-17T22:42:18.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042677009000000002 2024-09-17T22:42:18.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189656+0000", 2024-09-17T22:42:18.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0712999999999999e-05 2024-09-17T22:42:18.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189746+0000", 2024-09-17T22:42:18.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0370000000000004e-05 2024-09-17T22:42:18.319 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.319 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.319 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.319 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157247 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4128768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.319 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:18.319 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.126807948, 2024-09-17T22:42:18.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043016916000000002, 2024-09-17T22:42:18.319 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.319 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.319 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.319 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.320 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.320 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.320 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157247 2024-09-17T22:42:18.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.320 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:18.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:18.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147071+0000", 2024-09-17T22:42:18.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100769 2024-09-17T22:42:18.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147081+0000", 2024-09-17T22:42:18.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9229999999999997e-06 2024-09-17T22:42:18.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147090+0000", 2024-09-17T22:42:18.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.935e-06 2024-09-17T22:42:18.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147097+0000", 2024-09-17T22:42:18.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T22:42:18.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189787+0000", 2024-09-17T22:42:18.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042690525 2024-09-17T22:42:18.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189884+0000", 2024-09-17T22:42:18.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6858e-05 2024-09-17T22:42:18.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189988+0000", 2024-09-17T22:42:18.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103336 2024-09-17T22:42:18.323 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.323 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.323 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.324 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157250 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4134912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.324 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:18.324 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.126664486999999, 2024-09-17T22:42:18.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043099446, 2024-09-17T22:42:18.324 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.324 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.324 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.324 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.324 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.324 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.324 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157250 2024-09-17T22:42:18.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.324 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:18.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:18.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147235+0000", 2024-09-17T22:42:18.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120975 2024-09-17T22:42:18.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147238+0000", 2024-09-17T22:42:18.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5289999999999998e-06 2024-09-17T22:42:18.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147244+0000", 2024-09-17T22:42:18.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6050000000000003e-06 2024-09-17T22:42:18.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147249+0000", 2024-09-17T22:42:18.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4939999999999997e-06 2024-09-17T22:42:18.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190020+0000", 2024-09-17T22:42:18.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042771300999999998 2024-09-17T22:42:18.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190102+0000", 2024-09-17T22:42:18.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1600000000000005e-05 2024-09-17T22:42:18.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190214+0000", 2024-09-17T22:42:18.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011194200000000001 2024-09-17T22:42:18.328 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.328 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.328 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.328 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157252 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4141056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.328 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:18.328 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.126521818000001, 2024-09-17T22:42:18.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043165508999999998, 2024-09-17T22:42:18.329 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.329 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.329 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.329 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.329 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.329 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.329 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157252 2024-09-17T22:42:18.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.329 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:18.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:18.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147402+0000", 2024-09-17T22:42:18.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014475600000000001 2024-09-17T22:42:18.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147411+0000", 2024-09-17T22:42:18.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4350000000000003e-06 2024-09-17T22:42:18.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147420+0000", 2024-09-17T22:42:18.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4790000000000002e-06 2024-09-17T22:42:18.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147427+0000", 2024-09-17T22:42:18.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5400000000000001e-06 2024-09-17T22:42:18.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190254+0000", 2024-09-17T22:42:18.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042826731 2024-09-17T22:42:18.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190326+0000", 2024-09-17T22:42:18.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1798999999999995e-05 2024-09-17T22:42:18.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190422+0000", 2024-09-17T22:42:18.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6768999999999995e-05 2024-09-17T22:42:18.332 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.332 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.333 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.333 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157253 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4147200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.333 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:18.333 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.126335451999999, 2024-09-17T22:42:18.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043157752000000001, 2024-09-17T22:42:18.333 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.333 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.333 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.333 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.333 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.333 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.334 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157253 2024-09-17T22:42:18.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.334 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:18.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:18.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147590+0000", 2024-09-17T22:42:18.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014709 2024-09-17T22:42:18.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147593+0000", 2024-09-17T22:42:18.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5840000000000002e-06 2024-09-17T22:42:18.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147598+0000", 2024-09-17T22:42:18.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2469999999999999e-06 2024-09-17T22:42:18.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147602+0000", 2024-09-17T22:42:18.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0049999999999998e-06 2024-09-17T22:42:18.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190475+0000", 2024-09-17T22:42:18.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872767999999999 2024-09-17T22:42:18.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190528+0000", 2024-09-17T22:42:18.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2889999999999997e-05 2024-09-17T22:42:18.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190601+0000", 2024-09-17T22:42:18.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3168000000000001e-05 2024-09-17T22:42:18.337 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.337 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.337 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.337 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157255 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4153344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.337 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:18.337 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.126169471000001, 2024-09-17T22:42:18.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043132251000000003, 2024-09-17T22:42:18.338 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.338 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.338 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.338 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.338 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.338 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.338 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157255 2024-09-17T22:42:18.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.338 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:18.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:18.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147820+0000", 2024-09-17T22:42:18.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021123600000000001 2024-09-17T22:42:18.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147824+0000", 2024-09-17T22:42:18.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6780000000000002e-06 2024-09-17T22:42:18.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147833+0000", 2024-09-17T22:42:18.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4250000000000004e-06 2024-09-17T22:42:18.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147842+0000", 2024-09-17T22:42:18.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0679999999999997e-06 2024-09-17T22:42:18.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190632+0000", 2024-09-17T22:42:18.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042790863999999998 2024-09-17T22:42:18.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190674+0000", 2024-09-17T22:42:18.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1804000000000002e-05 2024-09-17T22:42:18.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190741+0000", 2024-09-17T22:42:18.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7175999999999995e-05 2024-09-17T22:42:18.342 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.342 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.342 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.342 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157257 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4159488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.342 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:18.342 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.125928981000001, 2024-09-17T22:42:18.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.04307035, 2024-09-17T22:42:18.342 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.342 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.342 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.342 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.342 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.343 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.343 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157257 2024-09-17T22:42:18.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.343 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:18.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:18.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148844+0000", 2024-09-17T22:42:18.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00099455599999999996 2024-09-17T22:42:18.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148848+0000", 2024-09-17T22:42:18.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0180000000000003e-06 2024-09-17T22:42:18.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148858+0000", 2024-09-17T22:42:18.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.825e-06 2024-09-17T22:42:18.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148865+0000", 2024-09-17T22:42:18.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8290000000000002e-06 2024-09-17T22:42:18.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190789+0000", 2024-09-17T22:42:18.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041924630999999997 2024-09-17T22:42:18.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190860+0000", 2024-09-17T22:42:18.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0557999999999998e-05 2024-09-17T22:42:18.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190920+0000", 2024-09-17T22:42:18.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9932999999999997e-05 2024-09-17T22:42:18.346 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.346 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.346 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.347 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157259 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4165632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.347 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:18.347 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.124902962, 2024-09-17T22:42:18.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042261411999999998, 2024-09-17T22:42:18.347 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.347 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.347 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.347 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.347 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.347 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.347 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157259 2024-09-17T22:42:18.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.347 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:18.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:18.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:18.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:42:18.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148878+0000", 2024-09-17T22:42:18.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:42:18.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148885+0000", 2024-09-17T22:42:18.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7290000000000003e-06 2024-09-17T22:42:18.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148893+0000", 2024-09-17T22:42:18.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8110000000000003e-06 2024-09-17T22:42:18.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190967+0000", 2024-09-17T22:42:18.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042073948999999999 2024-09-17T22:42:18.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191069+0000", 2024-09-17T22:42:18.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102075 2024-09-17T22:42:18.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191137+0000", 2024-09-17T22:42:18.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8281000000000006e-05 2024-09-17T22:42:18.351 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.351 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.351 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.351 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157261 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4171776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.351 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:18.351 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.124853354999999, 2024-09-17T22:42:18.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042437255, 2024-09-17T22:42:18.352 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.352 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.352 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.352 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.352 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.352 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.352 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157261 2024-09-17T22:42:18.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.352 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:18.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:18.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148926+0000", 2024-09-17T22:42:18.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T22:42:18.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148928+0000", 2024-09-17T22:42:18.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4329999999999998e-06 2024-09-17T22:42:18.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148936+0000", 2024-09-17T22:42:18.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6969999999999993e-06 2024-09-17T22:42:18.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148942+0000", 2024-09-17T22:42:18.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9479999999999996e-06 2024-09-17T22:42:18.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191185+0000", 2024-09-17T22:42:18.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042243147000000002 2024-09-17T22:42:18.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191266+0000", 2024-09-17T22:42:18.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1186000000000005e-05 2024-09-17T22:42:18.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191362+0000", 2024-09-17T22:42:18.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6384000000000004e-05 2024-09-17T22:42:18.355 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.356 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.356 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.356 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157263 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4177920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.356 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:18.356 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.124828956000002, 2024-09-17T22:42:18.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042636395000000001, 2024-09-17T22:42:18.356 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.356 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.356 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.356 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.356 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.356 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.357 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157263 2024-09-17T22:42:18.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.357 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:18.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:18.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:18.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.37e-07 2024-09-17T22:42:18.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148952+0000", 2024-09-17T22:42:18.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.88e-06 2024-09-17T22:42:18.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148959+0000", 2024-09-17T22:42:18.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5649999999999998e-06 2024-09-17T22:42:18.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148963+0000", 2024-09-17T22:42:18.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8570000000000003e-06 2024-09-17T22:42:18.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191408+0000", 2024-09-17T22:42:18.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042444707999999998 2024-09-17T22:42:18.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191489+0000", 2024-09-17T22:42:18.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0612999999999995e-05 2024-09-17T22:42:18.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191586+0000", 2024-09-17T22:42:18.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7235e-05 2024-09-17T22:42:18.360 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.360 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.360 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.360 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157265 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4184064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.360 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:18.361 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.124807406999999, 2024-09-17T22:42:18.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042791830000000003, 2024-09-17T22:42:18.361 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.361 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.361 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.361 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.361 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.361 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.361 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157265 2024-09-17T22:42:18.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.361 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:18.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:18.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148972+0000", 2024-09-17T22:42:18.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:42:18.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148973+0000", 2024-09-17T22:42:18.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.894e-06 2024-09-17T22:42:18.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148985+0000", 2024-09-17T22:42:18.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1995e-05 2024-09-17T22:42:18.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148990+0000", 2024-09-17T22:42:18.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8269999999999997e-06 2024-09-17T22:42:18.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191630+0000", 2024-09-17T22:42:18.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042639639 2024-09-17T22:42:18.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191686+0000", 2024-09-17T22:42:18.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6066000000000002e-05 2024-09-17T22:42:18.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191763+0000", 2024-09-17T22:42:18.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7035999999999999e-05 2024-09-17T22:42:18.365 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.365 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.365 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.365 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157267 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4190208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:18.365 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:18.365 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.124781594000002, 2024-09-17T22:42:18.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043028813999999999, 2024-09-17T22:42:18.365 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:18.365 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:18.365 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:18.366 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:18.366 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:18.366 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:18.366 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157267 2024-09-17T22:42:18.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.366 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:18.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:18.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:18.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:18.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:18.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:18.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:18.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:18.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:42:18.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:18.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148999+0000", 2024-09-17T22:42:18.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.432e-06 2024-09-17T22:42:18.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:18.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149002+0000", 2024-09-17T22:42:18.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7850000000000002e-06 2024-09-17T22:42:18.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:18.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149005+0000", 2024-09-17T22:42:18.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:42:18.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:18.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191878+0000", 2024-09-17T22:42:18.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872312000000003 2024-09-17T22:42:18.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:18.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191961+0000", 2024-09-17T22:42:18.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3418000000000007e-05 2024-09-17T22:42:18.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:18.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:18.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:18.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.192026+0000", 2024-09-17T22:42:18.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4689000000000004e-05 2024-09-17T22:42:18.369 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.369 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.369 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.369 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:18.369 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:18.370 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:18.372 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:42:18.372 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:18.368+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:18.373 INFO:teuthology.orchestra.run.smithi115.stdout:3549 trunc from 0xec16f99 to 0x2f28ff8 2024-09-17T22:42:18.374 INFO:teuthology.orchestra.run.smithi115.stdout:3554 read 0x5066f3 thru 0x509b7e (0x348c bytes) 2024-09-17T22:42:18.374 INFO:teuthology.orchestra.run.smithi115.stdout:3555 read 0x4ca295 thru 0x4d8f61 (0xeccd bytes) 2024-09-17T22:42:18.374 INFO:teuthology.orchestra.run.smithi115.stdout:3558 punch from 0x124e9b4 to 0x12509aa, (0x1ff6 bytes) 2024-09-17T22:42:18.374 INFO:teuthology.orchestra.run.smithi115.stdout:3560 write 0xe033271 thru 0xe033fbd (0xd4d bytes) 2024-09-17T22:42:18.374 INFO:teuthology.orchestra.run.smithi115.stdout:3561 punch from 0x7597335 to 0x759dff5, (0x6cc0 bytes) 2024-09-17T22:42:18.374 INFO:teuthology.orchestra.run.smithi115.stdout:3562 trunc from 0xe033fbe to 0x868b564 2024-09-17T22:42:18.374 INFO:teuthology.orchestra.run.smithi115.stdout:3563 writesame 0xc22c22f thru 0xc238065 data_size 0x1(0xbe37 bytes) 2024-09-17T22:42:18.374 INFO:teuthology.orchestra.run.smithi115.stdout:3564 read 0xc178d89 thru 0xc17a69f (0x1917 bytes) 2024-09-17T22:42:18.374 INFO:teuthology.orchestra.run.smithi115.stdout:3565 write 0xcdc8ad thru 0xcdf314 (0x2a68 bytes) 2024-09-17T22:42:18.374 INFO:teuthology.orchestra.run.smithi115.stdout:3567 write 0xb9c152d thru 0xb9c975a (0x822e bytes) 2024-09-17T22:42:18.374 INFO:teuthology.orchestra.run.smithi115.stdout:3570 punch from 0x1685fdf to 0x168e70a, (0x872b bytes) 2024-09-17T22:42:18.374 INFO:teuthology.orchestra.run.smithi115.stdout:3571 trunc from 0xc238066 to 0xc779892 2024-09-17T22:42:18.374 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:18.375 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:18.375 INFO:teuthology.orchestra.run.smithi115.stdout:3574 punch from 0x1907e96 to 0x190c97b, (0x4ae5 bytes) 2024-09-17T22:42:18.375 INFO:teuthology.orchestra.run.smithi115.stdout:3577 read 0x6162d9a thru 0x616de40 (0xb0a7 bytes) 2024-09-17T22:42:18.375 INFO:teuthology.orchestra.run.smithi115.stdout:3578 write 0x896c0f3 thru 0x89735b2 (0x74c0 bytes) 2024-09-17T22:42:18.375 INFO:teuthology.orchestra.run.smithi115.stdout:3579 writesame 0x4d30b3b thru 0x4d3413e data_size 0x2(0x3604 bytes) 2024-09-17T22:42:18.375 INFO:teuthology.orchestra.run.smithi115.stdout:3581 write 0x7753e6b thru 0x7763299 (0xf42f bytes) 2024-09-17T22:42:18.375 INFO:teuthology.orchestra.run.smithi115.stdout:3583 write 0x7a70b5f thru 0x7a7e96c (0xde0e bytes) 2024-09-17T22:42:18.375 INFO:teuthology.orchestra.run.smithi115.stdout:3585 clone 31 order 20 su 131072 sc 11 2024-09-17T22:42:18.471 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:18.469+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:18.517 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:18.517 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:18.517 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:18.517 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:18.525 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:42:18.533 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:42:18.578 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:18.575+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:18.638 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:18.655 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:18.655 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:18.669 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:18.670 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:18.670 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:18.670 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:18.670 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:18.671 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:18.671 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:18.673 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:18.670+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:18.679 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:42:18.687 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:18.687 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:18.704 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:18.704 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:18.720 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:18.720 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:18.736 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:18.736 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:18.752 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:18.752 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:18.755 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:18.756 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:18.797 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:18.794+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:18.822 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:18.822 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:18.831 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:42:18.874 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:18.871+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:18.934 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:18.934 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:18.942 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:42:18.983 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:18.980+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:19.076 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:19.073+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:19.085 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:19.086 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:19.094 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:42:19.178 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:19.175+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:19.238 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:19.238 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:19.247 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:42:19.248 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is set 2024-09-17T22:42:19.278 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:19.275+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:19.307 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone30 from 0xc779892 (overlap 0xb6b8de) to 0x36d10f 2024-09-17T22:42:19.369 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:19.369 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:19.373 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-17T22:42:19.379 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:19.376+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:19.380 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:42:19.479 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:19.477+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:19.493 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:19.493 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:19.502 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:42:19.553 INFO:teuthology.orchestra.run.smithi115.stdout:3586 punch from 0x3636cf3 to 0x363b97a, (0x4c87 bytes) 2024-09-17T22:42:19.557 INFO:teuthology.orchestra.run.smithi115.stdout:3587 read 0x3cecec9 thru 0x3cede66 (0xf9e bytes) 2024-09-17T22:42:19.557 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:19.557 INFO:teuthology.orchestra.run.smithi115.stdout:3589 write 0x266ca17 thru 0x267bae8 (0xf0d2 bytes) 2024-09-17T22:42:19.566 INFO:teuthology.orchestra.run.smithi115.stdout:3590 trunc from 0xc779892 to 0xa52b3cc 2024-09-17T22:42:19.577 INFO:teuthology.orchestra.run.smithi115.stdout:3591 writesame 0x725f700 thru 0x726e0bd data_size 0x3(0xe9be bytes) 2024-09-17T22:42:19.580 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:19.578+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:19.590 INFO:teuthology.orchestra.run.smithi115.stdout:3592 write 0x545c541 thru 0x5465fdb (0x9a9b bytes) 2024-09-17T22:42:19.597 INFO:teuthology.orchestra.run.smithi115.stdout:3593 read 0x73eaccf thru 0x73f788a (0xcbbc bytes) 2024-09-17T22:42:19.597 INFO:teuthology.orchestra.run.smithi115.stdout:3595 read 0x98ec0f7 thru 0x98f08e3 (0x47ed bytes) 2024-09-17T22:42:19.597 INFO:teuthology.orchestra.run.smithi115.stdout:3596 punch from 0x2bbebfd to 0x2bcdc31, (0xf034 bytes) 2024-09-17T22:42:19.609 INFO:teuthology.orchestra.run.smithi115.stdout:3597 trunc from 0xa52b3cc to 0xaf5c90 2024-09-17T22:42:19.645 INFO:teuthology.orchestra.run.smithi115.stdout:3598 writesame 0x60b35b6 thru 0x60bb810 data_size 0x11(0x825b bytes) 2024-09-17T22:42:19.660 INFO:teuthology.orchestra.run.smithi115.stdout:3600 write 0xd43d133 thru 0xd448d86 (0xbc54 bytes) 2024-09-17T22:42:19.663 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:19.664 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:19.665 INFO:teuthology.orchestra.run.smithi115.stdout:3602 read 0xc151e77 thru 0xc155df7 (0x3f81 bytes) 2024-09-17T22:42:19.665 INFO:teuthology.orchestra.run.smithi115.stdout:3603 writesame 0x34979cc thru 0x34a342b data_size 0x70(0xba60 bytes) 2024-09-17T22:42:19.673 INFO:teuthology.orchestra.run.smithi115.stdout:3605 trunc from 0xd448d87 to 0xa32504d 2024-09-17T22:42:19.675 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:42:19.694 INFO:teuthology.orchestra.run.smithi115.stdout:3606 write 0xe7bb1c1 thru 0xe7c1aaa (0x68ea bytes) 2024-09-17T22:42:19.694 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:19.691+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:19.696 INFO:teuthology.orchestra.run.smithi115.stdout:3608 write 0x17e8bea thru 0x17eedcf (0x61e6 bytes) 2024-09-17T22:42:19.704 INFO:teuthology.orchestra.run.smithi115.stdout:3609 trunc from 0xe7c1aab to 0xad9bbc3 2024-09-17T22:42:19.712 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:19.713 INFO:teuthology.orchestra.run.smithi115.stdout:3611 punch from 0x2fe662d to 0x2ff05a5, (0x9f78 bytes) 2024-09-17T22:42:19.713 INFO:teuthology.orchestra.run.smithi115.stdout:3613 read 0x37041ea thru 0x3708aa1 (0x48b8 bytes) 2024-09-17T22:42:19.713 INFO:teuthology.orchestra.run.smithi115.stdout:3614 read 0xc9da76 thru 0xca8d6a (0xb2f5 bytes) 2024-09-17T22:42:19.713 INFO:teuthology.orchestra.run.smithi115.stdout:3615 read 0x9c7a0b7 thru 0x9c88623 (0xe56d bytes) 2024-09-17T22:42:19.713 INFO:teuthology.orchestra.run.smithi115.stdout:3617 write 0x250113a thru 0x2510cff (0xfbc6 bytes) 2024-09-17T22:42:19.715 INFO:teuthology.orchestra.run.smithi115.stdout:3618 read 0xc4e70 thru 0xc6a6a (0x1bfb bytes) 2024-09-17T22:42:19.724 INFO:teuthology.orchestra.run.smithi115.stdout:3619 write 0xce12d6d thru 0xce1ad03 (0x7f97 bytes) 2024-09-17T22:42:19.726 INFO:teuthology.orchestra.run.smithi115.stdout:3621 punch from 0x8c73928 to 0x8c816ef, (0xddc7 bytes) 2024-09-17T22:42:19.732 INFO:teuthology.orchestra.run.smithi115.stdout:3623 trunc from 0xce1ad04 to 0x81ef1f3 2024-09-17T22:42:19.740 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:19.747 INFO:teuthology.orchestra.run.smithi115.stdout:3625 writesame 0x990fb3e thru 0x99199ec data_size 0xb(0x9eaf bytes) 2024-09-17T22:42:19.747 INFO:teuthology.orchestra.run.smithi115.stdout:3626 read 0x339bb0b thru 0x33a8ea8 (0xd39e bytes) 2024-09-17T22:42:19.748 INFO:teuthology.orchestra.run.smithi115.stdout:3627 read 0x858458e thru 0x85927c5 (0xe238 bytes) 2024-09-17T22:42:19.748 INFO:teuthology.orchestra.run.smithi115.stdout:3628 punch from 0x12f36c1 to 0x12ff56a, (0xbea9 bytes) 2024-09-17T22:42:19.748 INFO:teuthology.orchestra.run.smithi115.stdout:3629 write 0x963c6c3 thru 0x9640735 (0x4073 bytes) 2024-09-17T22:42:19.752 INFO:teuthology.orchestra.run.smithi115.stdout:3631 punch from 0x87fbd36 to 0x8806061, (0xa32b bytes) 2024-09-17T22:42:19.760 INFO:teuthology.orchestra.run.smithi115.stdout:3632 trunc from 0x99199ed to 0x3314c83 2024-09-17T22:42:19.782 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:19.779+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:19.791 INFO:teuthology.orchestra.run.smithi115.stdout:3633 write 0xd933ba0 thru 0xd93547a (0x18db bytes) 2024-09-17T22:42:19.793 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:19.807 INFO:teuthology.orchestra.run.smithi115.stdout:3636 trunc from 0xd93547b to 0x1f71e21 2024-09-17T22:42:19.814 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:19.814 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:19.818 INFO:teuthology.orchestra.run.smithi115.stdout:3637 read 0x80cf04 thru 0x815563 (0x8660 bytes) 2024-09-17T22:42:19.819 INFO:teuthology.orchestra.run.smithi115.stdout:3638 punch from 0x606a4e to 0x613b80, (0xd132 bytes) 2024-09-17T22:42:19.824 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:42:19.828 INFO:teuthology.orchestra.run.smithi115.stdout:3640 write 0x462ea5a thru 0x463c276 (0xd81d bytes) 2024-09-17T22:42:19.831 INFO:teuthology.orchestra.run.smithi115.stdout:3642 trunc from 0x463c277 to 0x51fbd44 2024-09-17T22:42:19.837 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:19.839 INFO:teuthology.orchestra.run.smithi115.stdout:3646 trunc from 0x51fbd44 to 0x7821466 2024-09-17T22:42:19.848 INFO:teuthology.orchestra.run.smithi115.stdout:3648 trunc from 0x7821466 to 0x4f64ced 2024-09-17T22:42:19.853 INFO:teuthology.orchestra.run.smithi115.stdout:3651 read 0x3d1f777 thru 0x3d2cdd8 (0xd662 bytes) 2024-09-17T22:42:19.854 INFO:teuthology.orchestra.run.smithi115.stdout:3653 read 0x13c61c5 thru 0x13d1b05 (0xb941 bytes) 2024-09-17T22:42:19.854 INFO:teuthology.orchestra.run.smithi115.stdout:3654 write 0x4e6edcf thru 0x4e7900c (0xa23e bytes) 2024-09-17T22:42:19.870 INFO:teuthology.orchestra.run.smithi115.stdout:3655 writesame 0xbc2de65 thru 0xbc315b3 data_size 0x1(0x374f bytes) 2024-09-17T22:42:19.870 INFO:teuthology.orchestra.run.smithi115.stdout:3656 read 0x2217191 thru 0x2224402 (0xd272 bytes) 2024-09-17T22:42:19.870 INFO:teuthology.orchestra.run.smithi115.stdout:3657 write 0x127cbd4 thru 0x1287345 (0xa772 bytes) 2024-09-17T22:42:19.874 INFO:teuthology.orchestra.run.smithi115.stdout:3658 punch from 0x3e03f41 to 0x3e0ced8, (0x8f97 bytes) 2024-09-17T22:42:19.874 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:19.874 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:19.874 INFO:teuthology.orchestra.run.smithi115.stdout:3661 write 0x97c8969 thru 0x97ce8f7 (0x5f8f bytes) 2024-09-17T22:42:19.877 INFO:teuthology.orchestra.run.smithi115.stdout:3662 punch from 0x3bda97f to 0x3be0b1d, (0x619e bytes) 2024-09-17T22:42:19.877 INFO:teuthology.orchestra.run.smithi115.stdout:3664 read 0x9fa9954 thru 0x9faf2a5 (0x5952 bytes) 2024-09-17T22:42:19.877 INFO:teuthology.orchestra.run.smithi115.stdout:3665 punch from 0x4f68584 to 0x4f6f606, (0x7082 bytes) 2024-09-17T22:42:19.883 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:19.880+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:19.884 INFO:teuthology.orchestra.run.smithi115.stdout:3666 trunc from 0xbc315b4 to 0x5e00240 2024-09-17T22:42:19.908 INFO:teuthology.orchestra.run.smithi115.stdout:3667 write 0xd51e27b thru 0xd52722f (0x8fb5 bytes) 2024-09-17T22:42:19.911 INFO:teuthology.orchestra.run.smithi115.stdout:3668 writesame 0x988c669 thru 0x989bc74 data_size 0xae(0xf60c bytes) 2024-09-17T22:42:19.914 INFO:teuthology.orchestra.run.smithi115.stdout:3670 read 0xc569230 thru 0xc573c0a (0xa9db bytes) 2024-09-17T22:42:19.914 INFO:teuthology.orchestra.run.smithi115.stdout:3671 punch from 0x173c467 to 0x173ce5f, (0x9f8 bytes) 2024-09-17T22:42:19.914 INFO:teuthology.orchestra.run.smithi115.stdout:3672 read 0x6cdff0a thru 0x6cea5d9 (0xa6d0 bytes) 2024-09-17T22:42:19.914 INFO:teuthology.orchestra.run.smithi115.stdout:3673 write 0x92670ad thru 0x926ae61 (0x3db5 bytes) 2024-09-17T22:42:19.916 INFO:teuthology.orchestra.run.smithi115.stdout:3674 read 0xa9eb573 thru 0xa9edd31 (0x27bf bytes) 2024-09-17T22:42:19.917 INFO:teuthology.orchestra.run.smithi115.stdout:3675 read 0x3e3e206 thru 0x3e4475e (0x6559 bytes) 2024-09-17T22:42:19.917 INFO:teuthology.orchestra.run.smithi115.stdout:3677 write 0xbd7aaab thru 0xbd87d4b (0xd2a1 bytes) 2024-09-17T22:42:19.920 INFO:teuthology.orchestra.run.smithi115.stdout:3679 punch from 0x69b1eae to 0x69b54cb, (0x361d bytes) 2024-09-17T22:42:19.934 INFO:teuthology.orchestra.run.smithi115.stdout:3680 trunc from 0xd527230 to 0x1b8495b 2024-09-17T22:42:19.945 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:19.945 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:19.954 INFO:teuthology.orchestra.run.smithi115.stdout:3681 write 0x6e7c03b thru 0x6e7fa12 (0x39d8 bytes) 2024-09-17T22:42:19.957 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:42:19.968 INFO:teuthology.orchestra.run.smithi115.stdout:3682 write 0xb1e4f6e thru 0xb1f07a6 (0xb839 bytes) 2024-09-17T22:42:19.970 INFO:teuthology.orchestra.run.smithi115.stdout:3683 write 0x8cd3d26 thru 0x8cd4bdf (0xeba bytes) 2024-09-17T22:42:19.973 INFO:teuthology.orchestra.run.smithi115.stdout:3684 punch from 0x5192904 to 0x51985cb, (0x5cc7 bytes) 2024-09-17T22:42:19.973 INFO:teuthology.orchestra.run.smithi115.stdout:3685 writesame 0x13d9c4a thru 0x13e6af1 data_size 0x44(0xcea8 bytes) 2024-09-17T22:42:19.983 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:19.980+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:19.983 INFO:teuthology.orchestra.run.smithi115.stdout:3687 write 0xdd68080 thru 0xdd6a358 (0x22d9 bytes) 2024-09-17T22:42:19.999 INFO:teuthology.orchestra.run.smithi115.stdout:3688 trunc from 0xdd6a359 to 0x2710b85 2024-09-17T22:42:20.013 INFO:teuthology.orchestra.run.smithi115.stdout:3689 trunc from 0x2710b85 to 0x69bb6dc 2024-09-17T22:42:20.027 INFO:teuthology.orchestra.run.smithi115.stdout:3690 trunc from 0x69bb6dc to 0xe3de366 2024-09-17T22:42:20.033 INFO:teuthology.orchestra.run.smithi115.stdout:3691 write 0x38e6430 thru 0x38e9a36 (0x3607 bytes) 2024-09-17T22:42:20.036 INFO:teuthology.orchestra.run.smithi115.stdout:3692 punch from 0x96492f7 to 0x9656c48, (0xd951 bytes) 2024-09-17T22:42:20.036 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:20.041 INFO:teuthology.orchestra.run.smithi115.stdout:3694 writesame 0xe6bfaa8 thru 0xe6ca377 data_size 0x4a(0xa8d0 bytes) 2024-09-17T22:42:20.041 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:20.041 INFO:teuthology.orchestra.run.smithi115.stdout:3697 read 0xe4cd6bf thru 0xe4db82c (0xe16e bytes) 2024-09-17T22:42:20.041 INFO:teuthology.orchestra.run.smithi115.stdout:3698 write 0x1967178 thru 0x1973326 (0xc1af bytes) 2024-09-17T22:42:20.044 INFO:teuthology.orchestra.run.smithi115.stdout:3699 writesame 0xacbcae3 thru 0xacccaba data_size 0xc(0xffd8 bytes) 2024-09-17T22:42:20.048 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:20.048 INFO:teuthology.orchestra.run.smithi115.stdout:3703 writesame 0xac98278 thru 0xaca0f23 data_size 0x6(0x8cac bytes) 2024-09-17T22:42:20.052 INFO:teuthology.orchestra.run.smithi115.stdout:3704 write 0x352c20d thru 0x3534781 (0x8575 bytes) 2024-09-17T22:42:20.055 INFO:teuthology.orchestra.run.smithi115.stdout:3705 read 0xbba1d85 thru 0xbbb10fa (0xf376 bytes) 2024-09-17T22:42:20.055 INFO:teuthology.orchestra.run.smithi115.stdout:3706 write 0xdc8f71e thru 0xdc9a048 (0xa92b bytes) 2024-09-17T22:42:20.058 INFO:teuthology.orchestra.run.smithi115.stdout:3707 read 0xa3735d6 thru 0xa381c00 (0xe62b bytes) 2024-09-17T22:42:20.065 INFO:teuthology.orchestra.run.smithi115.stdout:3708 trunc from 0xe6ca378 to 0x88c7289 2024-09-17T22:42:20.075 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:20.076 INFO:teuthology.orchestra.run.smithi115.stdout:3710 read 0x27f6882 thru 0x27fca3e (0x61bd bytes) 2024-09-17T22:42:20.076 INFO:teuthology.orchestra.run.smithi115.stdout:3711 punch from 0x6dc57a to 0x6ebbe0, (0xf666 bytes) 2024-09-17T22:42:20.084 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:20.081+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:20.085 INFO:teuthology.orchestra.run.smithi115.stdout:3712 write 0xd6e8256 thru 0xd6f2de6 (0xab91 bytes) 2024-09-17T22:42:20.088 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:20.088 INFO:teuthology.orchestra.run.smithi115.stdout:3714 read 0xac47f33 thru 0xac4b7c5 (0x3893 bytes) 2024-09-17T22:42:20.088 INFO:teuthology.orchestra.run.smithi115.stdout:3716 write 0x1690ceb thru 0x16923b4 (0x16ca bytes) 2024-09-17T22:42:20.089 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:20.089 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:20.094 INFO:teuthology.orchestra.run.smithi115.stdout:3718 writesame 0xda219e4 thru 0xda27f44 data_size 0x3(0x6561 bytes) 2024-09-17T22:42:20.094 INFO:teuthology.orchestra.run.smithi115.stdout:3723 read 0x290907f thru 0x2917e68 (0xedea bytes) 2024-09-17T22:42:20.095 INFO:teuthology.orchestra.run.smithi115.stdout:3724 read 0x93cbcd2 thru 0x93d693f (0xac6e bytes) 2024-09-17T22:42:20.095 INFO:teuthology.orchestra.run.smithi115.stdout:3725 read 0x8ef5b5 thru 0x8f8c20 (0x966c bytes) 2024-09-17T22:42:20.096 INFO:teuthology.orchestra.run.smithi115.stdout:3726 writesame 0xa44bc1a thru 0xa4568ca data_size 0x1(0xacb1 bytes) 2024-09-17T22:42:20.099 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:42:20.118 INFO:teuthology.orchestra.run.smithi115.stdout:3728 punch from 0x3804cb to 0x38f29e, (0xedd3 bytes) 2024-09-17T22:42:20.118 INFO:teuthology.orchestra.run.smithi115.stdout:3729 write 0x338bc76 thru 0x33950c4 (0x944f bytes) 2024-09-17T22:42:20.121 INFO:teuthology.orchestra.run.smithi115.stdout:3730 write 0x1281870 thru 0x128c249 (0xa9da bytes) 2024-09-17T22:42:20.124 INFO:teuthology.orchestra.run.smithi115.stdout:3731 read 0x70776aa thru 0x707b811 (0x4168 bytes) 2024-09-17T22:42:20.124 INFO:teuthology.orchestra.run.smithi115.stdout:3733 read 0x135ce88 thru 0x1365df0 (0x8f69 bytes) 2024-09-17T22:42:20.124 INFO:teuthology.orchestra.run.smithi115.stdout:3734 write 0xbbdcac1 thru 0xbbe1bb2 (0x50f2 bytes) 2024-09-17T22:42:20.129 INFO:teuthology.orchestra.run.smithi115.stdout:3736 read 0xb4b25a5 thru 0xb4c1646 (0xf0a2 bytes) 2024-09-17T22:42:20.130 INFO:teuthology.orchestra.run.smithi115.stdout:3738 writesame 0x6162d67 thru 0x61679ab data_size 0x37(0x4c45 bytes) 2024-09-17T22:42:20.132 INFO:teuthology.orchestra.run.smithi115.stdout:3740 read 0x8925626 thru 0x89353e8 (0xfdc3 bytes) 2024-09-17T22:42:20.136 INFO:teuthology.orchestra.run.smithi115.stdout:3742 trunc from 0xda27f45 to 0xa745824 2024-09-17T22:42:20.145 INFO:teuthology.orchestra.run.smithi115.stdout:3743 writesame 0x99e738f thru 0x99eb7e1 data_size 0x1(0x4453 bytes) 2024-09-17T22:42:20.153 INFO:teuthology.orchestra.run.smithi115.stdout:3744 trunc from 0xa745824 to 0x9704706 2024-09-17T22:42:20.162 INFO:teuthology.orchestra.run.smithi115.stdout:3745 writesame 0x4d2561d thru 0x4d321bc data_size 0x90(0xcba0 bytes) 2024-09-17T22:42:20.165 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:20.166 INFO:teuthology.orchestra.run.smithi115.stdout:3747 read 0x23356d2 thru 0x2336954 (0x1283 bytes) 2024-09-17T22:42:20.176 INFO:teuthology.orchestra.run.smithi115.stdout:3749 write 0xdabd5f4 thru 0xdacc26a (0xec77 bytes) 2024-09-17T22:42:20.179 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:20.179 INFO:teuthology.orchestra.run.smithi115.stdout:3751 writesame 0x5a29cbc thru 0x5a2bfe5 data_size 0x7(0x232a bytes) 2024-09-17T22:42:20.182 INFO:teuthology.orchestra.run.smithi115.stdout:3752 write 0xb9f4f84 thru 0xb9ffa35 (0xaab2 bytes) 2024-09-17T22:42:20.185 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:20.182+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:20.185 INFO:teuthology.orchestra.run.smithi115.stdout:3753 writesame 0x7313cd6 thru 0x7323345 data_size 0x8(0xf670 bytes) 2024-09-17T22:42:20.191 INFO:teuthology.orchestra.run.smithi115.stdout:3755 read 0xfa89b5 thru 0xfb7bde (0xf22a bytes) 2024-09-17T22:42:20.191 INFO:teuthology.orchestra.run.smithi115.stdout:3756 write 0x9e1208f thru 0x9e20d90 (0xed02 bytes) 2024-09-17T22:42:20.197 INFO:teuthology.orchestra.run.smithi115.stdout:3757 writesame 0x45a200f thru 0x45a4ff3 data_size 0x3d(0x2fe5 bytes) 2024-09-17T22:42:20.205 INFO:teuthology.orchestra.run.smithi115.stdout:3759 writesame 0xe5c7c3a thru 0xe5d07e3 data_size 0x65(0x8baa bytes) 2024-09-17T22:42:20.206 INFO:teuthology.orchestra.run.smithi115.stdout:3760 write 0xc64d3be thru 0xc65a128 (0xcd6b bytes) 2024-09-17T22:42:20.208 INFO:teuthology.orchestra.run.smithi115.stdout:3762 writesame 0x9197ca9 thru 0x919cfce data_size 0x1d(0x5326 bytes) 2024-09-17T22:42:20.211 INFO:teuthology.orchestra.run.smithi115.stdout:3763 write 0xd501831 thru 0xd50bfd2 (0xa7a2 bytes) 2024-09-17T22:42:20.213 INFO:teuthology.orchestra.run.smithi115.stdout:3771 write 0x7e673b3 thru 0x7e73adf (0xc72d bytes) 2024-09-17T22:42:20.216 INFO:teuthology.orchestra.run.smithi115.stdout:3772 writesame 0x6a8d49c thru 0x6a97f14 data_size 0x27(0xaa79 bytes) 2024-09-17T22:42:20.219 INFO:teuthology.orchestra.run.smithi115.stdout:3773 writesame 0x8d56d0e thru 0x8d5b9dc data_size 0x7(0x4ccf bytes) 2024-09-17T22:42:20.222 INFO:teuthology.orchestra.run.smithi115.stdout:3774 read 0x49383e4 thru 0x49421bb (0x9dd8 bytes) 2024-09-17T22:42:20.222 INFO:teuthology.orchestra.run.smithi115.stdout:3775 write 0xb8ea3d4 thru 0xb8ea86f (0x49c bytes) 2024-09-17T22:42:20.224 INFO:teuthology.orchestra.run.smithi115.stdout:3776 writesame 0xddb40db thru 0xddbf00e data_size 0x2(0xaf34 bytes) 2024-09-17T22:42:20.231 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:20.232 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:20.237 INFO:teuthology.orchestra.run.smithi115.stdout:3777 write 0x3090109 thru 0x309b50b (0xb403 bytes) 2024-09-17T22:42:20.240 INFO:teuthology.orchestra.run.smithi115.stdout:3778 read 0x6cbd08a thru 0x6cc7b63 (0xaada bytes) 2024-09-17T22:42:20.240 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:20.241 INFO:teuthology.orchestra.run.smithi115.stdout:3780 write 0x5a17b81 thru 0x5a1b6de (0x3b5e bytes) 2024-09-17T22:42:20.242 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:42:20.243 INFO:teuthology.orchestra.run.smithi115.stdout:3781 read 0xa29becc thru 0xa2a2ab8 (0x6bed bytes) 2024-09-17T22:42:20.244 INFO:teuthology.orchestra.run.smithi115.stdout:3782 punch from 0x2977bfe to 0x297ad73, (0x3175 bytes) 2024-09-17T22:42:20.244 INFO:teuthology.orchestra.run.smithi115.stdout:3783 read 0x831698b thru 0x83231bf (0xc835 bytes) 2024-09-17T22:42:20.244 INFO:teuthology.orchestra.run.smithi115.stdout:3784 write 0x5a7e912 thru 0x5a81bdd (0x32cc bytes) 2024-09-17T22:42:20.246 INFO:teuthology.orchestra.run.smithi115.stdout:3786 read 0x3aabb99 thru 0x3ab4560 (0x89c8 bytes) 2024-09-17T22:42:20.246 INFO:teuthology.orchestra.run.smithi115.stdout:3787 punch from 0x9645f05 to 0x9649efc, (0x3ff7 bytes) 2024-09-17T22:42:20.246 INFO:teuthology.orchestra.run.smithi115.stdout:3788 read 0xb74f297 thru 0xb7573cc (0x8136 bytes) 2024-09-17T22:42:20.246 INFO:teuthology.orchestra.run.smithi115.stdout:3789 read 0x5feb6df thru 0x5ff8828 (0xd14a bytes) 2024-09-17T22:42:20.246 INFO:teuthology.orchestra.run.smithi115.stdout:3790 read 0xbb22b6d thru 0xbb2abf7 (0x808b bytes) 2024-09-17T22:42:20.246 INFO:teuthology.orchestra.run.smithi115.stdout:3792 read 0xe593c29 thru 0xe596f8f (0x3367 bytes) 2024-09-17T22:42:20.257 INFO:teuthology.orchestra.run.smithi115.stdout:3793 trunc from 0xe5d07e4 to 0x468cd84 2024-09-17T22:42:20.271 INFO:teuthology.orchestra.run.smithi115.stdout:3794 read 0xd9fdb1 thru 0xdacff4 (0xd244 bytes) 2024-09-17T22:42:20.271 INFO:teuthology.orchestra.run.smithi115.stdout:3796 read 0x1f48697 thru 0x1f487fb (0x165 bytes) 2024-09-17T22:42:20.286 INFO:teuthology.orchestra.run.smithi115.stdout:3797 writesame 0xc6de47a thru 0xc6e595c data_size 0x1(0x74e3 bytes) 2024-09-17T22:42:20.286 INFO:teuthology.orchestra.run.smithi115.stdout:3798 clone 32 order 21 su 262144 sc 11 2024-09-17T22:42:20.286 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:20.283+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:20.353 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:20.353 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:20.362 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:42:20.403 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:20.400+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:20.488 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:20.485+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:20.504 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:20.504 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:20.513 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:42:20.589 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:20.586+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:20.616 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:20.617 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:20.625 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:42:20.709 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:20.706+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:20.756 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:42:20.768 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:20.768 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:20.777 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:42:20.819 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:20.815+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:20.859 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:20.876 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:20.876 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:20.893 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:20.889+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:20.893 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:20.893 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:20.908 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:20.908 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:20.921 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:20.921 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:20.924 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:20.924 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:20.930 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:42:20.941 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:20.941 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:20.957 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:20.957 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:20.973 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:20.974 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:20.976 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:20.977 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:21.014 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:21.011+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:21.074 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:21.074 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:21.074 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:21.074 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:21.083 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:42:21.094 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:21.092+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:21.195 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:21.192+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:21.227 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:21.227 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:21.227 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:21.227 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:21.228 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:21.236 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:42:21.264 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:42:21.295 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:21.292+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:21.310 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone31 from 0xc6e595d (overlap 0xaf5c90) to 0x118941 2024-09-17T22:42:21.356 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:21.356 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:21.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157229 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4073472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.206678647, 2024-09-17T22:42:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041957959000000003, 2024-09-17T22:42:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.357 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.358 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157229 2024-09-17T22:42:21.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.358 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:21.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:21.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146207+0000", 2024-09-17T22:42:21.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0299999999999999e-07 2024-09-17T22:42:21.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146215+0000", 2024-09-17T22:42:21.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3059999999999998e-06 2024-09-17T22:42:21.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146222+0000", 2024-09-17T22:42:21.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1180000000000003e-06 2024-09-17T22:42:21.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146227+0000", 2024-09-17T22:42:21.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1749999999999998e-06 2024-09-17T22:42:21.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.187985+0000", 2024-09-17T22:42:21.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041758391999999998 2024-09-17T22:42:21.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188065+0000", 2024-09-17T22:42:21.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0290000000000005e-05 2024-09-17T22:42:21.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188164+0000", 2024-09-17T22:42:21.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9074999999999996e-05 2024-09-17T22:42:21.361 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.361 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.361 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.362 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157231 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4079616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.362 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:21.362 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.206541023, 2024-09-17T22:42:21.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042034581000000001, 2024-09-17T22:42:21.362 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.362 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.362 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.362 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.362 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.362 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.362 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157231 2024-09-17T22:42:21.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.363 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:21.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:21.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146345+0000", 2024-09-17T22:42:21.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:42:21.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146347+0000", 2024-09-17T22:42:21.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.357e-06 2024-09-17T22:42:21.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146352+0000", 2024-09-17T22:42:21.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T22:42:21.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146361+0000", 2024-09-17T22:42:21.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0219999999999993e-06 2024-09-17T22:42:21.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188199+0000", 2024-09-17T22:42:21.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041838130000000001 2024-09-17T22:42:21.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188285+0000", 2024-09-17T22:42:21.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5160000000000005e-05 2024-09-17T22:42:21.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188379+0000", 2024-09-17T22:42:21.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3949000000000005e-05 2024-09-17T22:42:21.367 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.367 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.367 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.367 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157233 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4085760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.367 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:21.367 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.206324874, 2024-09-17T22:42:21.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042008960999999997, 2024-09-17T22:42:21.367 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.367 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.367 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.368 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.368 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.368 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.368 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157233 2024-09-17T22:42:21.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.368 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:21.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:21.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146561+0000", 2024-09-17T22:42:21.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6300000000000005e-07 2024-09-17T22:42:21.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146564+0000", 2024-09-17T22:42:21.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-17T22:42:21.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146575+0000", 2024-09-17T22:42:21.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0891e-05 2024-09-17T22:42:21.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146579+0000", 2024-09-17T22:42:21.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.284e-06 2024-09-17T22:42:21.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188407+0000", 2024-09-17T22:42:21.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041828390999999999 2024-09-17T22:42:21.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188476+0000", 2024-09-17T22:42:21.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9129999999999997e-05 2024-09-17T22:42:21.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188569+0000", 2024-09-17T22:42:21.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2669000000000003e-05 2024-09-17T22:42:21.373 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.373 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.373 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.373 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157235 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4091904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.374 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:21.374 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.206130877, 2024-09-17T22:42:21.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042047517, 2024-09-17T22:42:21.374 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.374 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.374 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.374 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.374 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.374 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.375 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157235 2024-09-17T22:42:21.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.375 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:21.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:21.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146755+0000", 2024-09-17T22:42:21.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0900000000000001e-07 2024-09-17T22:42:21.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146758+0000", 2024-09-17T22:42:21.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.399e-06 2024-09-17T22:42:21.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146768+0000", 2024-09-17T22:42:21.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4730000000000006e-06 2024-09-17T22:42:21.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146774+0000", 2024-09-17T22:42:21.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6939999999999998e-06 2024-09-17T22:42:21.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188616+0000", 2024-09-17T22:42:21.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041841481999999999 2024-09-17T22:42:21.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188700+0000", 2024-09-17T22:42:21.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4227999999999994e-05 2024-09-17T22:42:21.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188802+0000", 2024-09-17T22:42:21.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101532 2024-09-17T22:42:21.380 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.380 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.381 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.381 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157237 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4098048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.381 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:21.381 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.206100606, 2024-09-17T22:42:21.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165452999999999, 2024-09-17T22:42:21.382 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.382 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.382 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.382 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.382 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.382 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.382 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157237 2024-09-17T22:42:21.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.383 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:21.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:21.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146785+0000", 2024-09-17T22:42:21.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6800000000000001e-07 2024-09-17T22:42:21.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146787+0000", 2024-09-17T22:42:21.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.046e-06 2024-09-17T22:42:21.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146794+0000", 2024-09-17T22:42:21.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9249999999999998e-06 2024-09-17T22:42:21.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146799+0000", 2024-09-17T22:42:21.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9289999999999997e-06 2024-09-17T22:42:21.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188844+0000", 2024-09-17T22:42:21.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042045386999999997 2024-09-17T22:42:21.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188888+0000", 2024-09-17T22:42:21.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3760000000000001e-05 2024-09-17T22:42:21.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188950+0000", 2024-09-17T22:42:21.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1938000000000002e-05 2024-09-17T22:42:21.387 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.387 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.387 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.387 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157239 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4104192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.387 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:21.387 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.206072219999999, 2024-09-17T22:42:21.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042347193999999998, 2024-09-17T22:42:21.388 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.388 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.388 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.389 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.389 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.389 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157239 2024-09-17T22:42:21.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.389 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:21.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:21.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:21.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:42:21.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146816+0000", 2024-09-17T22:42:21.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4700000000000001e-06 2024-09-17T22:42:21.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146822+0000", 2024-09-17T22:42:21.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.579e-06 2024-09-17T22:42:21.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146826+0000", 2024-09-17T22:42:21.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3780000000000001e-06 2024-09-17T22:42:21.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188991+0000", 2024-09-17T22:42:21.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165553000000001 2024-09-17T22:42:21.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189068+0000", 2024-09-17T22:42:21.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6548e-05 2024-09-17T22:42:21.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189160+0000", 2024-09-17T22:42:21.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2163000000000005e-05 2024-09-17T22:42:21.394 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.394 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.394 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.394 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157242 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4110336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.395 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:21.395 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.206034433999999, 2024-09-17T22:42:21.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042499783999999999, 2024-09-17T22:42:21.395 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.395 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.395 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.395 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157242 2024-09-17T22:42:21.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.396 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:21.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:21.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:21.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:42:21.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146852+0000", 2024-09-17T22:42:21.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.37e-06 2024-09-17T22:42:21.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146856+0000", 2024-09-17T22:42:21.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1690000000000002e-06 2024-09-17T22:42:21.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146863+0000", 2024-09-17T22:42:21.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9920000000000001e-06 2024-09-17T22:42:21.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189193+0000", 2024-09-17T22:42:21.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042329355999999999 2024-09-17T22:42:21.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189246+0000", 2024-09-17T22:42:21.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3643000000000001e-05 2024-09-17T22:42:21.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189350+0000", 2024-09-17T22:42:21.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010394 2024-09-17T22:42:21.400 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.400 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.400 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.400 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157244 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4116480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.400 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:21.400 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.206015223000001, 2024-09-17T22:42:21.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042675684999999998, 2024-09-17T22:42:21.401 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.401 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.401 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.401 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157244 2024-09-17T22:42:21.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.401 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:21.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:21.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:21.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1e-07 2024-09-17T22:42:21.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146871+0000", 2024-09-17T22:42:21.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.189e-06 2024-09-17T22:42:21.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146877+0000", 2024-09-17T22:42:21.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8100000000000003e-06 2024-09-17T22:42:21.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146880+0000", 2024-09-17T22:42:21.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.83e-06 2024-09-17T22:42:21.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189388+0000", 2024-09-17T22:42:21.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042507637000000001 2024-09-17T22:42:21.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189443+0000", 2024-09-17T22:42:21.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5742999999999998e-05 2024-09-17T22:42:21.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189545+0000", 2024-09-17T22:42:21.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102166 2024-09-17T22:42:21.406 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.406 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.406 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.406 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157246 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4122624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.406 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:21.406 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.205986170999999, 2024-09-17T22:42:21.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042847083000000001, 2024-09-17T22:42:21.406 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.406 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.406 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.407 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.407 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.407 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.407 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157246 2024-09-17T22:42:21.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.407 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:21.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:21.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:21.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0100000000000001e-07 2024-09-17T22:42:21.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146901+0000", 2024-09-17T22:42:21.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.531e-06 2024-09-17T22:42:21.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146905+0000", 2024-09-17T22:42:21.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0759999999999996e-06 2024-09-17T22:42:21.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146908+0000", 2024-09-17T22:42:21.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.083e-06 2024-09-17T22:42:21.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189585+0000", 2024-09-17T22:42:21.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042677009000000002 2024-09-17T22:42:21.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189656+0000", 2024-09-17T22:42:21.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0712999999999999e-05 2024-09-17T22:42:21.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189746+0000", 2024-09-17T22:42:21.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0370000000000004e-05 2024-09-17T22:42:21.410 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.410 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.410 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.411 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157247 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4128768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.411 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:21.411 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.205914334999999, 2024-09-17T22:42:21.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043016916000000002, 2024-09-17T22:42:21.411 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.411 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.411 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.411 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.411 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.411 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.411 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157247 2024-09-17T22:42:21.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.412 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:21.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:21.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147071+0000", 2024-09-17T22:42:21.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100769 2024-09-17T22:42:21.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147081+0000", 2024-09-17T22:42:21.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9229999999999997e-06 2024-09-17T22:42:21.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147090+0000", 2024-09-17T22:42:21.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.935e-06 2024-09-17T22:42:21.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147097+0000", 2024-09-17T22:42:21.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T22:42:21.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189787+0000", 2024-09-17T22:42:21.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042690525 2024-09-17T22:42:21.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189884+0000", 2024-09-17T22:42:21.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6858e-05 2024-09-17T22:42:21.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189988+0000", 2024-09-17T22:42:21.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103336 2024-09-17T22:42:21.415 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.415 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.415 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.415 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157250 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4134912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.416 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:21.416 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.205770873999999, 2024-09-17T22:42:21.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043099446, 2024-09-17T22:42:21.416 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.416 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.416 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.416 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.416 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.416 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.416 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157250 2024-09-17T22:42:21.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.416 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:21.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:21.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147235+0000", 2024-09-17T22:42:21.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120975 2024-09-17T22:42:21.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147238+0000", 2024-09-17T22:42:21.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5289999999999998e-06 2024-09-17T22:42:21.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147244+0000", 2024-09-17T22:42:21.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6050000000000003e-06 2024-09-17T22:42:21.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147249+0000", 2024-09-17T22:42:21.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4939999999999997e-06 2024-09-17T22:42:21.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190020+0000", 2024-09-17T22:42:21.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042771300999999998 2024-09-17T22:42:21.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190102+0000", 2024-09-17T22:42:21.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1600000000000005e-05 2024-09-17T22:42:21.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190214+0000", 2024-09-17T22:42:21.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011194200000000001 2024-09-17T22:42:21.420 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.420 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.420 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.420 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157252 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4141056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.420 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:21.420 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.205628205, 2024-09-17T22:42:21.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043165508999999998, 2024-09-17T22:42:21.420 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.420 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.420 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.420 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.420 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.421 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.421 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157252 2024-09-17T22:42:21.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.421 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:21.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:21.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147402+0000", 2024-09-17T22:42:21.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014475600000000001 2024-09-17T22:42:21.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147411+0000", 2024-09-17T22:42:21.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4350000000000003e-06 2024-09-17T22:42:21.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147420+0000", 2024-09-17T22:42:21.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4790000000000002e-06 2024-09-17T22:42:21.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147427+0000", 2024-09-17T22:42:21.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5400000000000001e-06 2024-09-17T22:42:21.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190254+0000", 2024-09-17T22:42:21.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042826731 2024-09-17T22:42:21.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190326+0000", 2024-09-17T22:42:21.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1798999999999995e-05 2024-09-17T22:42:21.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190422+0000", 2024-09-17T22:42:21.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6768999999999995e-05 2024-09-17T22:42:21.424 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.424 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.424 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.424 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157253 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4147200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.424 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:21.424 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.205441838999999, 2024-09-17T22:42:21.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043157752000000001, 2024-09-17T22:42:21.425 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.425 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.425 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.425 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.425 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.425 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.425 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157253 2024-09-17T22:42:21.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.425 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:21.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:21.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147590+0000", 2024-09-17T22:42:21.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014709 2024-09-17T22:42:21.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147593+0000", 2024-09-17T22:42:21.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5840000000000002e-06 2024-09-17T22:42:21.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147598+0000", 2024-09-17T22:42:21.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2469999999999999e-06 2024-09-17T22:42:21.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147602+0000", 2024-09-17T22:42:21.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0049999999999998e-06 2024-09-17T22:42:21.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190475+0000", 2024-09-17T22:42:21.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872767999999999 2024-09-17T22:42:21.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190528+0000", 2024-09-17T22:42:21.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2889999999999997e-05 2024-09-17T22:42:21.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190601+0000", 2024-09-17T22:42:21.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3168000000000001e-05 2024-09-17T22:42:21.429 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.429 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.429 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.429 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157255 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4153344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.430 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:21.430 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.205275858, 2024-09-17T22:42:21.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043132251000000003, 2024-09-17T22:42:21.430 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.430 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.430 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.430 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.430 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.430 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.430 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157255 2024-09-17T22:42:21.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.430 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:21.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:21.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147820+0000", 2024-09-17T22:42:21.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021123600000000001 2024-09-17T22:42:21.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147824+0000", 2024-09-17T22:42:21.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6780000000000002e-06 2024-09-17T22:42:21.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147833+0000", 2024-09-17T22:42:21.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4250000000000004e-06 2024-09-17T22:42:21.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147842+0000", 2024-09-17T22:42:21.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0679999999999997e-06 2024-09-17T22:42:21.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190632+0000", 2024-09-17T22:42:21.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042790863999999998 2024-09-17T22:42:21.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190674+0000", 2024-09-17T22:42:21.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1804000000000002e-05 2024-09-17T22:42:21.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190741+0000", 2024-09-17T22:42:21.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7175999999999995e-05 2024-09-17T22:42:21.433 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.434 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.434 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.434 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157257 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4159488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.434 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:21.434 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.205035368000001, 2024-09-17T22:42:21.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.04307035, 2024-09-17T22:42:21.434 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.434 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.434 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.434 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.434 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.435 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.435 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157257 2024-09-17T22:42:21.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.435 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:21.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:21.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148844+0000", 2024-09-17T22:42:21.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00099455599999999996 2024-09-17T22:42:21.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148848+0000", 2024-09-17T22:42:21.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0180000000000003e-06 2024-09-17T22:42:21.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148858+0000", 2024-09-17T22:42:21.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.825e-06 2024-09-17T22:42:21.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148865+0000", 2024-09-17T22:42:21.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8290000000000002e-06 2024-09-17T22:42:21.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190789+0000", 2024-09-17T22:42:21.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041924630999999997 2024-09-17T22:42:21.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190860+0000", 2024-09-17T22:42:21.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0557999999999998e-05 2024-09-17T22:42:21.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190920+0000", 2024-09-17T22:42:21.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9932999999999997e-05 2024-09-17T22:42:21.438 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.438 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.438 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.438 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157259 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4165632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.438 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:21.438 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.204009349, 2024-09-17T22:42:21.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042261411999999998, 2024-09-17T22:42:21.439 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.439 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.439 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.439 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.439 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.439 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.439 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157259 2024-09-17T22:42:21.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.439 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:21.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:21.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:21.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:42:21.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.440 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148878+0000", 2024-09-17T22:42:21.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:42:21.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148885+0000", 2024-09-17T22:42:21.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7290000000000003e-06 2024-09-17T22:42:21.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148893+0000", 2024-09-17T22:42:21.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8110000000000003e-06 2024-09-17T22:42:21.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190967+0000", 2024-09-17T22:42:21.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042073948999999999 2024-09-17T22:42:21.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191069+0000", 2024-09-17T22:42:21.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102075 2024-09-17T22:42:21.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191137+0000", 2024-09-17T22:42:21.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8281000000000006e-05 2024-09-17T22:42:21.442 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.442 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.442 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.443 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157261 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4171776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.443 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:21.443 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.203959741999999, 2024-09-17T22:42:21.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042437255, 2024-09-17T22:42:21.443 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.443 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.443 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.443 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157261 2024-09-17T22:42:21.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.443 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:21.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:21.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148926+0000", 2024-09-17T22:42:21.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T22:42:21.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148928+0000", 2024-09-17T22:42:21.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4329999999999998e-06 2024-09-17T22:42:21.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148936+0000", 2024-09-17T22:42:21.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6969999999999993e-06 2024-09-17T22:42:21.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148942+0000", 2024-09-17T22:42:21.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9479999999999996e-06 2024-09-17T22:42:21.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191185+0000", 2024-09-17T22:42:21.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042243147000000002 2024-09-17T22:42:21.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191266+0000", 2024-09-17T22:42:21.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1186000000000005e-05 2024-09-17T22:42:21.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191362+0000", 2024-09-17T22:42:21.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6384000000000004e-05 2024-09-17T22:42:21.447 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.447 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.447 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.447 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157263 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4177920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.447 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:21.447 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.203935343000001, 2024-09-17T22:42:21.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042636395000000001, 2024-09-17T22:42:21.447 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.448 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.448 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.448 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.448 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.448 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.448 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157263 2024-09-17T22:42:21.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.448 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:21.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:21.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:21.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.37e-07 2024-09-17T22:42:21.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148952+0000", 2024-09-17T22:42:21.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.88e-06 2024-09-17T22:42:21.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148959+0000", 2024-09-17T22:42:21.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5649999999999998e-06 2024-09-17T22:42:21.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148963+0000", 2024-09-17T22:42:21.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8570000000000003e-06 2024-09-17T22:42:21.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191408+0000", 2024-09-17T22:42:21.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042444707999999998 2024-09-17T22:42:21.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191489+0000", 2024-09-17T22:42:21.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0612999999999995e-05 2024-09-17T22:42:21.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191586+0000", 2024-09-17T22:42:21.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7235e-05 2024-09-17T22:42:21.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.451 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.452 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.452 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157265 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4184064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.452 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:21.452 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.203913794000002, 2024-09-17T22:42:21.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042791830000000003, 2024-09-17T22:42:21.452 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.452 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.452 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.453 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157265 2024-09-17T22:42:21.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.453 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:21.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:21.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148972+0000", 2024-09-17T22:42:21.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:42:21.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148973+0000", 2024-09-17T22:42:21.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.894e-06 2024-09-17T22:42:21.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148985+0000", 2024-09-17T22:42:21.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1995e-05 2024-09-17T22:42:21.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148990+0000", 2024-09-17T22:42:21.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8269999999999997e-06 2024-09-17T22:42:21.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191630+0000", 2024-09-17T22:42:21.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042639639 2024-09-17T22:42:21.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191686+0000", 2024-09-17T22:42:21.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6066000000000002e-05 2024-09-17T22:42:21.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191763+0000", 2024-09-17T22:42:21.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7035999999999999e-05 2024-09-17T22:42:21.456 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.456 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.456 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.456 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157267 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4190208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:21.456 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:21.456 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.203887981000001, 2024-09-17T22:42:21.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043028813999999999, 2024-09-17T22:42:21.456 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.457 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.457 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.457 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157267 2024-09-17T22:42:21.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.457 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:21.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:21.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:21.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:42:21.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148999+0000", 2024-09-17T22:42:21.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.432e-06 2024-09-17T22:42:21.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149002+0000", 2024-09-17T22:42:21.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7850000000000002e-06 2024-09-17T22:42:21.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149005+0000", 2024-09-17T22:42:21.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:42:21.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191878+0000", 2024-09-17T22:42:21.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872312000000003 2024-09-17T22:42:21.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191961+0000", 2024-09-17T22:42:21.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3418000000000007e-05 2024-09-17T22:42:21.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.192026+0000", 2024-09-17T22:42:21.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4689000000000004e-05 2024-09-17T22:42:21.460 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.460 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.460 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.460 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.461 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.461 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:21.463 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:42:21.463 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:21.459+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:21.463 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-17T22:42:21.463 INFO:teuthology.orchestra.run.smithi115.stdout:3800 read 0x5bbafe5 thru 0x5bc7fe0 (0xcffc bytes) 2024-09-17T22:42:21.463 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:21.464 INFO:teuthology.orchestra.run.smithi115.stdout:3803 read 0xb0151df thru 0xb01a04d (0x4e6f bytes) 2024-09-17T22:42:21.464 INFO:teuthology.orchestra.run.smithi115.stdout:3804 read 0x427b756 thru 0x4283a94 (0x833f bytes) 2024-09-17T22:42:21.464 INFO:teuthology.orchestra.run.smithi115.stdout:3805 punch from 0xddee7f to 0xde0429, (0x15aa bytes) 2024-09-17T22:42:21.464 INFO:teuthology.orchestra.run.smithi115.stdout:3809 write 0x2c4b3a8 thru 0x2c5a0de (0xed37 bytes) 2024-09-17T22:42:21.464 INFO:teuthology.orchestra.run.smithi115.stdout:3810 writesame 0xe546748 thru 0xe5523fa data_size 0x7(0xbcb3 bytes) 2024-09-17T22:42:21.465 INFO:teuthology.orchestra.run.smithi115.stdout:3811 trunc from 0xe5523fb to 0x43f09a4 2024-09-17T22:42:21.487 INFO:teuthology.orchestra.run.smithi115.stdout:3812 write 0x41b23d9 thru 0x41b5685 (0x32ad bytes) 2024-09-17T22:42:21.491 INFO:teuthology.orchestra.run.smithi115.stdout:3814 read 0x3994bd3 thru 0x39a0753 (0xbb81 bytes) 2024-09-17T22:42:21.491 INFO:teuthology.orchestra.run.smithi115.stdout:3815 writesame 0x181dd3e thru 0x1820d44 data_size 0x1(0x3007 bytes) 2024-09-17T22:42:21.499 INFO:teuthology.orchestra.run.smithi115.stdout:3816 punch from 0x20e795a to 0x20f3e4b, (0xc4f1 bytes) 2024-09-17T22:42:21.504 INFO:teuthology.orchestra.run.smithi115.stdout:3817 trunc from 0x43f09a4 to 0xbc6d25 2024-09-17T22:42:21.557 INFO:teuthology.orchestra.run.smithi115.stdout:3818 writesame 0x83b1788 thru 0x83bba8b data_size 0x2(0xa304 bytes) 2024-09-17T22:42:21.557 INFO:teuthology.orchestra.run.smithi115.stdout:3819 write 0x2c30536 thru 0x2c39fc9 (0x9a94 bytes) 2024-09-17T22:42:21.564 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:21.562+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:21.566 INFO:teuthology.orchestra.run.smithi115.stdout:3820 trunc from 0x83bba8c to 0x2553517 2024-09-17T22:42:21.587 INFO:teuthology.orchestra.run.smithi115.stdout:3821 write 0x59d198b thru 0x59d9b82 (0x81f8 bytes) 2024-09-17T22:42:21.591 INFO:teuthology.orchestra.run.smithi115.stdout:3822 read 0x197b1df thru 0x1988d83 (0xdba5 bytes) 2024-09-17T22:42:21.591 INFO:teuthology.orchestra.run.smithi115.stdout:3824 read 0xd03c92 thru 0xd071d3 (0x3542 bytes) 2024-09-17T22:42:21.599 INFO:teuthology.orchestra.run.smithi115.stdout:3825 write 0x6feaf00 thru 0x6fed087 (0x2188 bytes) 2024-09-17T22:42:21.607 INFO:teuthology.orchestra.run.smithi115.stdout:3826 writesame 0x7701a24 thru 0x770f46b data_size 0x7f(0xda48 bytes) 2024-09-17T22:42:21.611 INFO:teuthology.orchestra.run.smithi115.stdout:3827 trunc from 0x770f46c to 0x4405861 2024-09-17T22:42:21.621 INFO:teuthology.orchestra.run.smithi115.stdout:3828 write 0x215f3d3 thru 0x2166ff4 (0x7c22 bytes) 2024-09-17T22:42:21.622 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:21.622 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:21.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.622 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:206009 3.12 3:4f5b6197:::rbd_object_map.115f32a0c059:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T22:42:21.622 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:21.619497+0000", 2024-09-17T22:42:21.622 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.00058786299999999999, 2024-09-17T22:42:21.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00060694299999999998, 2024-09-17T22:42:21.622 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.622 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.622 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "waiting for sub ops", 2024-09-17T22:42:21.623 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.623 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.623 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.623 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 206009 2024-09-17T22:42:21.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.623 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:21.619497+0000", 2024-09-17T22:42:21.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:21.619497+0000", 2024-09-17T22:42:21.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:21.619498+0000", 2024-09-17T22:42:21.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9299999999999997e-07 2024-09-17T22:42:21.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:21.619502+0000", 2024-09-17T22:42:21.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1359999999999999e-06 2024-09-17T22:42:21.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:21.619512+0000", 2024-09-17T22:42:21.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7790000000000007e-06 2024-09-17T22:42:21.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:21.619517+0000", 2024-09-17T22:42:21.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0270000000000003e-06 2024-09-17T22:42:21.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:21.619562+0000", 2024-09-17T22:42:21.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5025999999999999e-05 2024-09-17T22:42:21.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:21.619645+0000", 2024-09-17T22:42:21.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3045000000000002e-05 2024-09-17T22:42:21.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 3", 2024-09-17T22:42:21.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:21.620007+0000", 2024-09-17T22:42:21.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00036165399999999999 2024-09-17T22:42:21.628 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.628 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.628 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.628 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.628 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-17T22:42:21.628 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 1 2024-09-17T22:42:21.628 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:21.629 INFO:teuthology.orchestra.run.smithi115.stdout:3829 write 0x1312441 thru 0x1319f1f (0x7adf bytes) 2024-09-17T22:42:21.631 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:42:21.638 INFO:teuthology.orchestra.run.smithi115.stdout:3832 write 0x9daa8d9 thru 0x9db51dc (0xa904 bytes) 2024-09-17T22:42:21.643 INFO:teuthology.orchestra.run.smithi115.stdout:3834 write 0x96cfe33 thru 0x96d5194 (0x5362 bytes) 2024-09-17T22:42:21.654 INFO:teuthology.orchestra.run.smithi115.stdout:3835 write 0xb3f1fe6 thru 0xb3ff630 (0xd64b bytes) 2024-09-17T22:42:21.657 INFO:teuthology.orchestra.run.smithi115.stdout:3836 writesame 0x8b9291f thru 0x8b936b0 data_size 0x9(0xd92 bytes) 2024-09-17T22:42:21.661 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:21.670 INFO:teuthology.orchestra.run.smithi115.stdout:3838 write 0xde44531 thru 0xde4df64 (0x9a34 bytes) 2024-09-17T22:42:21.673 INFO:teuthology.orchestra.run.smithi115.stdout:3841 read 0x9ad1597 thru 0x9ae0e2d (0xf897 bytes) 2024-09-17T22:42:21.674 INFO:teuthology.orchestra.run.smithi115.stdout:3842 writesame 0x7da3df0 thru 0x7da8dd1 data_size 0x19(0x4fe2 bytes) 2024-09-17T22:42:21.675 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:21.672+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:21.677 INFO:teuthology.orchestra.run.smithi115.stdout:3844 punch from 0x944509a to 0x9452ff8, (0xdf5e bytes) 2024-09-17T22:42:21.677 INFO:teuthology.orchestra.run.smithi115.stdout:3845 read 0x5cc1237 thru 0x5cc1d41 (0xb0b bytes) 2024-09-17T22:42:21.677 INFO:teuthology.orchestra.run.smithi115.stdout:3847 read 0x63ba45d thru 0x63c124c (0x6df0 bytes) 2024-09-17T22:42:21.678 INFO:teuthology.orchestra.run.smithi115.stdout:3848 write 0xa279b44 thru 0xa27a415 (0x8d2 bytes) 2024-09-17T22:42:21.680 INFO:teuthology.orchestra.run.smithi115.stdout:3849 read 0xa4961c0 thru 0xa4a2005 (0xbe46 bytes) 2024-09-17T22:42:21.680 INFO:teuthology.orchestra.run.smithi115.stdout:3850 read 0xc598b22 thru 0xc59edf7 (0x62d6 bytes) 2024-09-17T22:42:21.680 INFO:teuthology.orchestra.run.smithi115.stdout:3851 punch from 0xb59b30a to 0xb5a7dcd, (0xcac3 bytes) 2024-09-17T22:42:21.680 INFO:teuthology.orchestra.run.smithi115.stdout:3852 write 0xd47f8c3 thru 0xd484772 (0x4eb0 bytes) 2024-09-17T22:42:21.690 INFO:teuthology.orchestra.run.smithi115.stdout:3853 trunc from 0xde4df65 to 0x8a670de 2024-09-17T22:42:21.701 INFO:teuthology.orchestra.run.smithi115.stdout:3854 punch from 0x4bf0a65 to 0x4bfffda, (0xf575 bytes) 2024-09-17T22:42:21.701 INFO:teuthology.orchestra.run.smithi115.stdout:3856 read 0xa2b80a thru 0xa39b49 (0xe340 bytes) 2024-09-17T22:42:21.702 INFO:teuthology.orchestra.run.smithi115.stdout:3857 write 0x6e69d53 thru 0x6e7420d (0xa4bb bytes) 2024-09-17T22:42:21.705 INFO:teuthology.orchestra.run.smithi115.stdout:3858 read 0x325420e thru 0x3256bc5 (0x29b8 bytes) 2024-09-17T22:42:21.705 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:21.711 INFO:teuthology.orchestra.run.smithi115.stdout:3860 trunc from 0x8a670de to 0x33b19dd 2024-09-17T22:42:21.741 INFO:teuthology.orchestra.run.smithi115.stdout:3861 write 0xd7c8e45 thru 0xd7d2d17 (0x9ed3 bytes) 2024-09-17T22:42:21.743 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:21.743 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:21.743 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:21.743 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:21.743 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:21.744 INFO:teuthology.orchestra.run.smithi115.stdout:3863 write 0x8645272 thru 0x8646180 (0xf0f bytes) 2024-09-17T22:42:21.746 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:21.746 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:21.746 INFO:teuthology.orchestra.run.smithi115.stdout:3866 punch from 0x8a63c91 to 0x8a711e0, (0xd54f bytes) 2024-09-17T22:42:21.747 INFO:teuthology.orchestra.run.smithi115.stdout:3868 read 0x888471e thru 0x8888952 (0x4235 bytes) 2024-09-17T22:42:21.747 INFO:teuthology.orchestra.run.smithi115.stdout:3869 read 0x5bb41f4 thru 0x5bbc95c (0x8769 bytes) 2024-09-17T22:42:21.750 INFO:teuthology.orchestra.run.smithi115.stdout:3871 trunc from 0xd7d2d18 to 0xa1b25a1 2024-09-17T22:42:21.753 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:42:21.760 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:21.770 INFO:teuthology.orchestra.run.smithi115.stdout:3874 writesame 0xe1510be thru 0xe15b55a data_size 0xb(0xa49d bytes) 2024-09-17T22:42:21.781 INFO:teuthology.orchestra.run.smithi115.stdout:3875 trunc from 0xe15b55b to 0x533a5c3 2024-09-17T22:42:21.795 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:21.791+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:21.806 INFO:teuthology.orchestra.run.smithi115.stdout:3876 write 0xdae8281 thru 0xdaf22dc (0xa05c bytes) 2024-09-17T22:42:21.810 INFO:teuthology.orchestra.run.smithi115.stdout:3878 write 0x9604409 thru 0x9608c1c (0x4814 bytes) 2024-09-17T22:42:21.813 INFO:teuthology.orchestra.run.smithi115.stdout:3879 punch from 0xb6c863d to 0xb6cd8c4, (0x5287 bytes) 2024-09-17T22:42:21.813 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:21.813 INFO:teuthology.orchestra.run.smithi115.stdout:3882 punch from 0x8bc86a3 to 0x8bd7092, (0xe9ef bytes) 2024-09-17T22:42:21.829 INFO:teuthology.orchestra.run.smithi115.stdout:3883 trunc from 0xdaf22dd to 0x546bcd 2024-09-17T22:42:21.855 INFO:teuthology.orchestra.run.smithi115.stdout:3884 write 0x29c1dc9 thru 0x29c428c (0x24c4 bytes) 2024-09-17T22:42:21.860 INFO:teuthology.orchestra.run.smithi115.stdout:3885 trunc from 0x29c428d to 0x691fe3 2024-09-17T22:42:21.866 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:21.863+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:21.876 INFO:teuthology.orchestra.run.smithi115.stdout:3886 trunc from 0x691fe3 to 0x31d0ee8 2024-09-17T22:42:21.882 INFO:teuthology.orchestra.run.smithi115.stdout:3887 punch from 0x2ae4fda to 0x2af0995, (0xb9bb bytes) 2024-09-17T22:42:21.882 INFO:teuthology.orchestra.run.smithi115.stdout:3888 punch from 0x22a6bc to 0x232aaa, (0x83ee bytes) 2024-09-17T22:42:21.883 INFO:teuthology.orchestra.run.smithi115.stdout:3894 trunc from 0x31d0ee8 to 0x24b5d83 2024-09-17T22:42:21.889 INFO:teuthology.orchestra.run.smithi115.stdout:3895 read 0x175885b thru 0x175f9ff (0x71a5 bytes) 2024-09-17T22:42:21.903 INFO:teuthology.orchestra.run.smithi115.stdout:3896 writesame 0x98e420a thru 0x98e81ad data_size 0x2(0x3fa4 bytes) 2024-09-17T22:42:21.903 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:21.904 INFO:teuthology.orchestra.run.smithi115.stdout:3898 read 0x96dec87 thru 0x96e714f (0x84c9 bytes) 2024-09-17T22:42:21.904 INFO:teuthology.orchestra.run.smithi115.stdout:3900 trunc from 0x98e81ae to 0xa603c15 2024-09-17T22:42:21.905 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:21.906 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:21.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:21.906 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:21.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.906 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188693 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5881856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:21.906 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:21.906 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.075542370000001, 2024-09-17T22:42:21.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028193223, 2024-09-17T22:42:21.906 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.906 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.906 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.906 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.906 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.906 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.906 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188693 2024-09-17T22:42:21.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.907 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:21.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:21.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:21.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:42:21.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827090+0000", 2024-09-17T22:42:21.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2170000000000001e-06 2024-09-17T22:42:21.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827098+0000", 2024-09-17T22:42:21.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1510000000000004e-06 2024-09-17T22:42:21.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827103+0000", 2024-09-17T22:42:21.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0839999999999999e-06 2024-09-17T22:42:21.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855158+0000", 2024-09-17T22:42:21.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028055217 2024-09-17T22:42:21.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855201+0000", 2024-09-17T22:42:21.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3272000000000002e-05 2024-09-17T22:42:21.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855280+0000", 2024-09-17T22:42:21.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8845999999999997e-05 2024-09-17T22:42:21.910 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.910 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.910 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.911 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188694 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5971968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:21.911 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:21.911 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.075518483, 2024-09-17T22:42:21.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028312278, 2024-09-17T22:42:21.911 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.911 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.911 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.912 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.912 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.912 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.912 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188694 2024-09-17T22:42:21.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.912 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:21.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:21.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:21.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:42:21.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827114+0000", 2024-09-17T22:42:21.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3029999999999998e-06 2024-09-17T22:42:21.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827121+0000", 2024-09-17T22:42:21.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3699999999999997e-06 2024-09-17T22:42:21.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827126+0000", 2024-09-17T22:42:21.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7350000000000004e-06 2024-09-17T22:42:21.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855307+0000", 2024-09-17T22:42:21.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028180851999999999 2024-09-17T22:42:21.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855346+0000", 2024-09-17T22:42:21.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9274999999999999e-05 2024-09-17T22:42:21.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855423+0000", 2024-09-17T22:42:21.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7294999999999998e-05 2024-09-17T22:42:21.917 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.917 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.917 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.917 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188695 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:21.917 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:21.917 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.075497705, 2024-09-17T22:42:21.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028435876999999998, 2024-09-17T22:42:21.917 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.917 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.918 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.918 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188695 2024-09-17T22:42:21.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.920 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:21.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:21.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:21.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:42:21.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827138+0000", 2024-09-17T22:42:21.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2580000000000001e-06 2024-09-17T22:42:21.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827145+0000", 2024-09-17T22:42:21.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0129999999999996e-06 2024-09-17T22:42:21.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827150+0000", 2024-09-17T22:42:21.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3749999999999996e-06 2024-09-17T22:42:21.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855447+0000", 2024-09-17T22:42:21.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028297224999999999 2024-09-17T22:42:21.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855487+0000", 2024-09-17T22:42:21.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0454000000000003e-05 2024-09-17T22:42:21.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855567+0000", 2024-09-17T22:42:21.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0149999999999997e-05 2024-09-17T22:42:21.924 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.924 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.924 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.924 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188696 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:21.924 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:21.924 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.075473921, 2024-09-17T22:42:21.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028543570000000001, 2024-09-17T22:42:21.925 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.925 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.925 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.925 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.925 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188696 2024-09-17T22:42:21.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.925 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:21.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:21.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827156+0000", 2024-09-17T22:42:21.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T22:42:21.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827158+0000", 2024-09-17T22:42:21.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.875e-06 2024-09-17T22:42:21.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827164+0000", 2024-09-17T22:42:21.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7950000000000002e-06 2024-09-17T22:42:21.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827169+0000", 2024-09-17T22:42:21.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2880000000000003e-06 2024-09-17T22:42:21.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855582+0000", 2024-09-17T22:42:21.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028412929 2024-09-17T22:42:21.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855624+0000", 2024-09-17T22:42:21.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2484000000000002e-05 2024-09-17T22:42:21.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855699+0000", 2024-09-17T22:42:21.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4869000000000004e-05 2024-09-17T22:42:21.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.930 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.931 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188697 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6242304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:21.931 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:21.931 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 13.075442391999999, 2024-09-17T22:42:21.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028658598, 2024-09-17T22:42:21.931 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.931 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.931 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.931 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188697 2024-09-17T22:42:21.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.932 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:21.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:21.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:21.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8800000000000003e-07 2024-09-17T22:42:21.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827190+0000", 2024-09-17T22:42:21.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.351e-06 2024-09-17T22:42:21.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827198+0000", 2024-09-17T22:42:21.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.909e-06 2024-09-17T22:42:21.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827203+0000", 2024-09-17T22:42:21.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4020000000000001e-06 2024-09-17T22:42:21.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855720+0000", 2024-09-17T22:42:21.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028516909999999999 2024-09-17T22:42:21.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855762+0000", 2024-09-17T22:42:21.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2184999999999997e-05 2024-09-17T22:42:21.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855845+0000", 2024-09-17T22:42:21.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3353e-05 2024-09-17T22:42:21.935 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.935 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.935 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.935 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202883 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2342912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:21.935 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:42:21.935 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.1778060110000004, 2024-09-17T22:42:21.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028098636, 2024-09-17T22:42:21.936 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.936 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.936 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.936 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202883 2024-09-17T22:42:21.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.936 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:42:21.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:42:21.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724824+0000", 2024-09-17T22:42:21.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:42:21.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724826+0000", 2024-09-17T22:42:21.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.086e-06 2024-09-17T22:42:21.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724832+0000", 2024-09-17T22:42:21.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9100000000000002e-06 2024-09-17T22:42:21.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724837+0000", 2024-09-17T22:42:21.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9439999999999996e-06 2024-09-17T22:42:21.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752790+0000", 2024-09-17T22:42:21.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027953657999999999 2024-09-17T22:42:21.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752848+0000", 2024-09-17T22:42:21.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7677999999999999e-05 2024-09-17T22:42:21.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752922+0000", 2024-09-17T22:42:21.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3957999999999999e-05 2024-09-17T22:42:21.940 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.940 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.940 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.940 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202884 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2473984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:21.940 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:42:21.940 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.177777786, 2024-09-17T22:42:21.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028206208999999999, 2024-09-17T22:42:21.940 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.940 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.940 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.941 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.941 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.941 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.941 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202884 2024-09-17T22:42:21.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.941 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:42:21.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:42:21.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724852+0000", 2024-09-17T22:42:21.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2599999999999998e-07 2024-09-17T22:42:21.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724854+0000", 2024-09-17T22:42:21.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.26e-06 2024-09-17T22:42:21.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724871+0000", 2024-09-17T22:42:21.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6589999999999999e-05 2024-09-17T22:42:21.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724877+0000", 2024-09-17T22:42:21.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.054e-06 2024-09-17T22:42:21.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752951+0000", 2024-09-17T22:42:21.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028073747 2024-09-17T22:42:21.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752993+0000", 2024-09-17T22:42:21.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2082000000000001e-05 2024-09-17T22:42:21.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753058+0000", 2024-09-17T22:42:21.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5049999999999996e-05 2024-09-17T22:42:21.944 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.944 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.944 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.945 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202885 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:21.945 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:21.945 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.1777444829999997, 2024-09-17T22:42:21.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028333028, 2024-09-17T22:42:21.945 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.945 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.945 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.945 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.945 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.945 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.945 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202885 2024-09-17T22:42:21.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.946 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:21.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:21.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:21.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3300000000000002e-07 2024-09-17T22:42:21.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724892+0000", 2024-09-17T22:42:21.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2779999999999999e-06 2024-09-17T22:42:21.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724899+0000", 2024-09-17T22:42:21.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9110000000000004e-06 2024-09-17T22:42:21.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724904+0000", 2024-09-17T22:42:21.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2360000000000004e-06 2024-09-17T22:42:21.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753094+0000", 2024-09-17T22:42:21.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02819033 2024-09-17T22:42:21.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753142+0000", 2024-09-17T22:42:21.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.829e-05 2024-09-17T22:42:21.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753218+0000", 2024-09-17T22:42:21.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5450000000000004e-05 2024-09-17T22:42:21.949 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.949 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.949 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.949 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202886 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:21.949 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:21.949 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.1777192850000002, 2024-09-17T22:42:21.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028442512, 2024-09-17T22:42:21.950 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.950 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.950 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.950 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.950 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.950 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.950 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202886 2024-09-17T22:42:21.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.950 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:21.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:21.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:21.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:42:21.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724912+0000", 2024-09-17T22:42:21.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.967e-06 2024-09-17T22:42:21.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724918+0000", 2024-09-17T22:42:21.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8069999999999996e-06 2024-09-17T22:42:21.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724923+0000", 2024-09-17T22:42:21.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.386e-06 2024-09-17T22:42:21.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753244+0000", 2024-09-17T22:42:21.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028321567999999998 2024-09-17T22:42:21.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753289+0000", 2024-09-17T22:42:21.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4620000000000003e-05 2024-09-17T22:42:21.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753353+0000", 2024-09-17T22:42:21.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3795999999999997e-05 2024-09-17T22:42:21.953 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.954 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.954 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.954 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202887 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:21.954 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:21.954 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.1777013719999996, 2024-09-17T22:42:21.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028565838999999999, 2024-09-17T22:42:21.954 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.954 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.954 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.954 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.954 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.954 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.955 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202887 2024-09-17T22:42:21.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.955 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:21.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:21.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:21.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:42:21.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724930+0000", 2024-09-17T22:42:21.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8980000000000001e-06 2024-09-17T22:42:21.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724936+0000", 2024-09-17T22:42:21.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7849999999999997e-06 2024-09-17T22:42:21.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724942+0000", 2024-09-17T22:42:21.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9649999999999996e-06 2024-09-17T22:42:21.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753380+0000", 2024-09-17T22:42:21.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028438049 2024-09-17T22:42:21.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753425+0000", 2024-09-17T22:42:21.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5163000000000001e-05 2024-09-17T22:42:21.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753494+0000", 2024-09-17T22:42:21.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8588000000000001e-05 2024-09-17T22:42:21.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.958 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.958 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202888 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:21.959 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:42:21.959 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.1776818740000001, 2024-09-17T22:42:21.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028703314000000001, 2024-09-17T22:42:21.959 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.959 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.959 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.959 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202888 2024-09-17T22:42:21.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.959 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:42:21.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:42:21.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724948+0000", 2024-09-17T22:42:21.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:42:21.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724950+0000", 2024-09-17T22:42:21.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8029999999999999e-06 2024-09-17T22:42:21.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724955+0000", 2024-09-17T22:42:21.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8760000000000002e-06 2024-09-17T22:42:21.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724960+0000", 2024-09-17T22:42:21.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5979999999999999e-06 2024-09-17T22:42:21.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753527+0000", 2024-09-17T22:42:21.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028566864000000001 2024-09-17T22:42:21.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753568+0000", 2024-09-17T22:42:21.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0775000000000002e-05 2024-09-17T22:42:21.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753651+0000", 2024-09-17T22:42:21.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3039000000000002e-05 2024-09-17T22:42:21.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.963 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.963 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202889 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:21.963 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:21.963 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.1776643020000002, 2024-09-17T22:42:21.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028820964000000001, 2024-09-17T22:42:21.963 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.963 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.963 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.964 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202889 2024-09-17T22:42:21.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.964 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:21.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:21.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:21.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2599999999999998e-07 2024-09-17T22:42:21.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724967+0000", 2024-09-17T22:42:21.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T22:42:21.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724973+0000", 2024-09-17T22:42:21.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6060000000000002e-06 2024-09-17T22:42:21.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724978+0000", 2024-09-17T22:42:21.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8230000000000002e-06 2024-09-17T22:42:21.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753687+0000", 2024-09-17T22:42:21.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028709495000000002 2024-09-17T22:42:21.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753727+0000", 2024-09-17T22:42:21.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9953999999999997e-05 2024-09-17T22:42:21.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753786+0000", 2024-09-17T22:42:21.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8941999999999999e-05 2024-09-17T22:42:21.967 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.967 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.967 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.968 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202890 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:21.968 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:21.968 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.1776467339999996, 2024-09-17T22:42:21.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028933838, 2024-09-17T22:42:21.968 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.968 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.968 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.968 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.968 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202890 2024-09-17T22:42:21.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.968 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:21.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:21.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:21.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5999999999999999e-07 2024-09-17T22:42:21.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724985+0000", 2024-09-17T22:42:21.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8169999999999999e-06 2024-09-17T22:42:21.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724991+0000", 2024-09-17T22:42:21.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:42:21.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724995+0000", 2024-09-17T22:42:21.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4530000000000004e-06 2024-09-17T22:42:21.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753811+0000", 2024-09-17T22:42:21.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028815584000000002 2024-09-17T22:42:21.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753854+0000", 2024-09-17T22:42:21.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3356999999999998e-05 2024-09-17T22:42:21.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753916+0000", 2024-09-17T22:42:21.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2488000000000002e-05 2024-09-17T22:42:21.972 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.972 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.972 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.972 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202891 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:21.972 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:21.972 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.1776292770000003, 2024-09-17T22:42:21.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029067554999999998, 2024-09-17T22:42:21.973 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.973 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.973 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.973 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.973 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202891 2024-09-17T22:42:21.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.973 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:21.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:21.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:21.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:42:21.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725002+0000", 2024-09-17T22:42:21.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:42:21.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725008+0000", 2024-09-17T22:42:21.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6629999999999998e-06 2024-09-17T22:42:21.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.975 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725016+0000", 2024-09-17T22:42:21.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8269999999999998e-06 2024-09-17T22:42:21.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.975 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753949+0000", 2024-09-17T22:42:21.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028932625999999999 2024-09-17T22:42:21.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753994+0000", 2024-09-17T22:42:21.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5717000000000003e-05 2024-09-17T22:42:21.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.976 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754068+0000", 2024-09-17T22:42:21.976 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3312000000000006e-05 2024-09-17T22:42:21.976 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.977 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.977 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.977 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202892 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:21.977 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:21.977 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.1775865699999999, 2024-09-17T22:42:21.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029159939999999999, 2024-09-17T22:42:21.977 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.977 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.977 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.977 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.978 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202892 2024-09-17T22:42:21.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.978 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:21.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:21.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:21.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7999999999999995e-07 2024-09-17T22:42:21.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725045+0000", 2024-09-17T22:42:21.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-17T22:42:21.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725052+0000", 2024-09-17T22:42:21.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3629999999999999e-06 2024-09-17T22:42:21.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725056+0000", 2024-09-17T22:42:21.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3250000000000001e-06 2024-09-17T22:42:21.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754094+0000", 2024-09-17T22:42:21.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029038270000000001 2024-09-17T22:42:21.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754134+0000", 2024-09-17T22:42:21.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9722999999999999e-05 2024-09-17T22:42:21.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.981 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.981 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754203+0000", 2024-09-17T22:42:21.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8524999999999999e-05 2024-09-17T22:42:21.981 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.981 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.981 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.981 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202893 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:21.981 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:42:21.982 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.1775669930000001, 2024-09-17T22:42:21.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029341651, 2024-09-17T22:42:21.982 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.982 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.982 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.982 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202893 2024-09-17T22:42:21.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.982 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:42:21.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:42:21.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725063+0000", 2024-09-17T22:42:21.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6699999999999999e-07 2024-09-17T22:42:21.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725065+0000", 2024-09-17T22:42:21.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2120000000000002e-06 2024-09-17T22:42:21.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725071+0000", 2024-09-17T22:42:21.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5169999999999996e-06 2024-09-17T22:42:21.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725076+0000", 2024-09-17T22:42:21.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.78e-06 2024-09-17T22:42:21.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754233+0000", 2024-09-17T22:42:21.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029156277000000001 2024-09-17T22:42:21.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754298+0000", 2024-09-17T22:42:21.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5320000000000005e-05 2024-09-17T22:42:21.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754404+0000", 2024-09-17T22:42:21.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106078 2024-09-17T22:42:21.986 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.986 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.986 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.986 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202894 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:21.986 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:42:21.986 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.1775471140000002, 2024-09-17T22:42:21.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029456572, 2024-09-17T22:42:21.986 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.986 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.986 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.987 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.987 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.987 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202894 2024-09-17T22:42:21.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.987 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:42:21.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:42:21.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725083+0000", 2024-09-17T22:42:21.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5900000000000002e-07 2024-09-17T22:42:21.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725085+0000", 2024-09-17T22:42:21.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.979e-06 2024-09-17T22:42:21.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725091+0000", 2024-09-17T22:42:21.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1480000000000003e-06 2024-09-17T22:42:21.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725095+0000", 2024-09-17T22:42:21.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1339999999999997e-06 2024-09-17T22:42:21.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754429+0000", 2024-09-17T22:42:21.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029333758000000001 2024-09-17T22:42:21.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754471+0000", 2024-09-17T22:42:21.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2274999999999998e-05 2024-09-17T22:42:21.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.990 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.990 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754539+0000", 2024-09-17T22:42:21.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7818999999999996e-05 2024-09-17T22:42:21.990 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.990 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.990 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.991 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202895 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:21.991 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:21.991 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.1775287179999996, 2024-09-17T22:42:21.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029635578999999999, 2024-09-17T22:42:21.991 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.991 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.991 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.991 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202895 2024-09-17T22:42:21.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.991 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:21.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:21.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:21.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.08e-07 2024-09-17T22:42:21.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725107+0000", 2024-09-17T22:42:21.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9020000000000004e-06 2024-09-17T22:42:21.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725113+0000", 2024-09-17T22:42:21.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7400000000000001e-06 2024-09-17T22:42:21.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725117+0000", 2024-09-17T22:42:21.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:42:21.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754567+0000", 2024-09-17T22:42:21.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029449411000000002 2024-09-17T22:42:21.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754638+0000", 2024-09-17T22:42:21.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1432000000000003e-05 2024-09-17T22:42:21.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754736+0000", 2024-09-17T22:42:21.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8066000000000006e-05 2024-09-17T22:42:21.995 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.995 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.995 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.995 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202896 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:21.995 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:42:21.995 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.1775069870000001, 2024-09-17T22:42:21.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029760534000000002, 2024-09-17T22:42:21.995 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:21.996 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:21.996 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:21.996 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:21.996 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:21.996 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:21.996 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202896 2024-09-17T22:42:21.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.996 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:21.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.996 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:21.996 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:42:21.996 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.996 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:21.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:42:21.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:21.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:21.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725123+0000", 2024-09-17T22:42:21.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:42:21.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.997 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.997 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:21.997 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725125+0000", 2024-09-17T22:42:21.997 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.863e-06 2024-09-17T22:42:21.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:21.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725130+0000", 2024-09-17T22:42:21.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7250000000000002e-06 2024-09-17T22:42:21.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:21.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725135+0000", 2024-09-17T22:42:21.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:42:21.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.998 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.998 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:21.998 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754763+0000", 2024-09-17T22:42:21.998 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029628202999999999 2024-09-17T22:42:21.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:21.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754805+0000", 2024-09-17T22:42:21.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2370000000000003e-05 2024-09-17T22:42:21.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:21.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:21.999 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:21.999 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754883+0000", 2024-09-17T22:42:21.999 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7565999999999996e-05 2024-09-17T22:42:21.999 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:21.999 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:21.999 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.000 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.000 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202897 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.000 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:22.000 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.1774896269999999, 2024-09-17T22:42:22.000 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029947972999999999, 2024-09-17T22:42:22.000 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.000 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.000 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.000 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.000 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.000 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.000 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202897 2024-09-17T22:42:22.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.001 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:22.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.001 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.001 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:22.001 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.001 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:22.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:42:22.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725146+0000", 2024-09-17T22:42:22.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6030000000000004e-06 2024-09-17T22:42:22.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.002 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.002 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.002 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725152+0000", 2024-09-17T22:42:22.002 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9970000000000004e-06 2024-09-17T22:42:22.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725156+0000", 2024-09-17T22:42:22.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-06 2024-09-17T22:42:22.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754909+0000", 2024-09-17T22:42:22.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029752958999999999 2024-09-17T22:42:22.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.003 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.003 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.003 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754987+0000", 2024-09-17T22:42:22.003 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8208999999999996e-05 2024-09-17T22:42:22.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.004 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.004 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.004 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.755088+0000", 2024-09-17T22:42:22.004 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100348 2024-09-17T22:42:22.004 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.004 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.004 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.004 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.004 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.004 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:22.006 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:42:22.008 INFO:teuthology.orchestra.run.smithi115.stdout:3901 write 0x34ce7d5 thru 0x34d1c4b (0x3477 bytes) 2024-09-17T22:42:22.008 INFO:teuthology.orchestra.run.smithi115.stdout:3902 punch from 0x7f1321a to 0x7f17296, (0x407c bytes) 2024-09-17T22:42:22.008 INFO:teuthology.orchestra.run.smithi115.stdout:3903 write 0x6d33477 thru 0x6d42afe (0xf688 bytes) 2024-09-17T22:42:22.008 INFO:teuthology.orchestra.run.smithi115.stdout:3904 punch from 0x325c9d1 to 0x325de43, (0x1472 bytes) 2024-09-17T22:42:22.008 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.008 INFO:teuthology.orchestra.run.smithi115.stdout:3906 writesame 0x887ace6 thru 0x88874a1 data_size 0x6(0xc7bc bytes) 2024-09-17T22:42:22.008 INFO:teuthology.orchestra.run.smithi115.stdout:3907 writesame 0x856334 thru 0x8654e7 data_size 0x3e(0xf1b4 bytes) 2024-09-17T22:42:22.008 INFO:teuthology.orchestra.run.smithi115.stdout:3910 write 0xe7f1a38 thru 0xe7fe420 (0xc9e9 bytes) 2024-09-17T22:42:22.008 INFO:teuthology.orchestra.run.smithi115.stdout:3911 trunc from 0xe7fe421 to 0x492cb34 2024-09-17T22:42:22.008 INFO:teuthology.orchestra.run.smithi115.stdout:3912 write 0xc2bcb12 thru 0xc2c921c (0xc70b bytes) 2024-09-17T22:42:22.009 INFO:teuthology.orchestra.run.smithi115.stdout:3916 read 0x716051f thru 0x716f971 (0xf453 bytes) 2024-09-17T22:42:22.009 INFO:teuthology.orchestra.run.smithi115.stdout:3917 read 0xa5c13f7 thru 0xa5c5919 (0x4523 bytes) 2024-09-17T22:42:22.009 INFO:teuthology.orchestra.run.smithi115.stdout:3918 punch from 0x4bb9404 to 0x4bc47bb, (0xb3b7 bytes) 2024-09-17T22:42:22.009 INFO:teuthology.orchestra.run.smithi115.stdout:3919 write 0x56dde56 thru 0x56ea1c0 (0xc36b bytes) 2024-09-17T22:42:22.009 INFO:teuthology.orchestra.run.smithi115.stdout:3922 writesame 0x7184ae5 thru 0x7188644 data_size 0x10(0x3b60 bytes) 2024-09-17T22:42:22.009 INFO:teuthology.orchestra.run.smithi115.stdout:3923 writesame 0xb7b6a0a thru 0xb7bd53c data_size 0x1(0x6b33 bytes) 2024-09-17T22:42:22.009 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:22.003+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:22.011 INFO:teuthology.orchestra.run.smithi115.stdout:3924 write 0x6c7afb1 thru 0x6c7cb11 (0x1b61 bytes) 2024-09-17T22:42:22.013 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.014 INFO:teuthology.orchestra.run.smithi115.stdout:3927 read 0x3b94739 thru 0x3b9b108 (0x69d0 bytes) 2024-09-17T22:42:22.014 INFO:teuthology.orchestra.run.smithi115.stdout:3928 read 0x875cce2 thru 0x876420b (0x752a bytes) 2024-09-17T22:42:22.014 INFO:teuthology.orchestra.run.smithi115.stdout:3929 write 0x432e271 thru 0x4334f5b (0x6ceb bytes) 2024-09-17T22:42:22.016 INFO:teuthology.orchestra.run.smithi115.stdout:3930 write 0x7927c6f thru 0x792b58e (0x3920 bytes) 2024-09-17T22:42:22.019 INFO:teuthology.orchestra.run.smithi115.stdout:3931 read 0xb20cf8b thru 0xb21c108 (0xf17e bytes) 2024-09-17T22:42:22.029 INFO:teuthology.orchestra.run.smithi115.stdout:3932 writesame 0xe92b17f thru 0xe92e1e1 data_size 0x1(0x3063 bytes) 2024-09-17T22:42:22.030 INFO:teuthology.orchestra.run.smithi115.stdout:3936 write 0x92ce99f thru 0x92de5d5 (0xfc37 bytes) 2024-09-17T22:42:22.033 INFO:teuthology.orchestra.run.smithi115.stdout:3938 write 0xe0749d4 thru 0xe07c02a (0x7657 bytes) 2024-09-17T22:42:22.036 INFO:teuthology.orchestra.run.smithi115.stdout:3939 write 0xd08b931 thru 0xd09b58e (0xfc5e bytes) 2024-09-17T22:42:22.039 INFO:teuthology.orchestra.run.smithi115.stdout:3941 read 0x90552ea thru 0x90597fc (0x4513 bytes) 2024-09-17T22:42:22.039 INFO:teuthology.orchestra.run.smithi115.stdout:3942 read 0x7005b02 thru 0x700bf9e (0x649d bytes) 2024-09-17T22:42:22.039 INFO:teuthology.orchestra.run.smithi115.stdout:3943 read 0x73efc2a thru 0x73f7ee3 (0x82ba bytes) 2024-09-17T22:42:22.041 INFO:teuthology.orchestra.run.smithi115.stdout:3944 trunc from 0xe92e1e2 to 0xcfb775e 2024-09-17T22:42:22.052 INFO:teuthology.orchestra.run.smithi115.stdout:3945 write 0x6788d14 thru 0x67905f0 (0x78dd bytes) 2024-09-17T22:42:22.055 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.063 INFO:teuthology.orchestra.run.smithi115.stdout:3947 write 0xec49802 thru 0xec54bcd (0xb3cc bytes) 2024-09-17T22:42:22.066 INFO:teuthology.orchestra.run.smithi115.stdout:3949 read 0x6f52cec thru 0x6f5dd3a (0xb04f bytes) 2024-09-17T22:42:22.067 INFO:teuthology.orchestra.run.smithi115.stdout:3950 trunc from 0xec54bce to 0xdcccf2e 2024-09-17T22:42:22.077 INFO:teuthology.orchestra.run.smithi115.stdout:3952 write 0x26c5e57 thru 0x26c75b0 (0x175a bytes) 2024-09-17T22:42:22.081 INFO:teuthology.orchestra.run.smithi115.stdout:3953 trunc from 0xdcccf2e to 0xce62874 2024-09-17T22:42:22.089 INFO:teuthology.orchestra.run.smithi115.stdout:3955 write 0xac6c07e thru 0xac6ec01 (0x2b84 bytes) 2024-09-17T22:42:22.093 INFO:teuthology.orchestra.run.smithi115.stdout:3957 write 0x34a5137 thru 0x34afd6f (0xac39 bytes) 2024-09-17T22:42:22.097 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.097 INFO:teuthology.orchestra.run.smithi115.stdout:3960 read 0x5ebf8e6 thru 0x5ec5c4f (0x636a bytes) 2024-09-17T22:42:22.098 INFO:teuthology.orchestra.run.smithi115.stdout:3963 read 0xbeac7b thru 0xbf114d (0x64d3 bytes) 2024-09-17T22:42:22.098 INFO:teuthology.orchestra.run.smithi115.stdout:3964 read 0x9ecb589 thru 0x9ecd295 (0x1d0d bytes) 2024-09-17T22:42:22.101 INFO:teuthology.orchestra.run.smithi115.stdout:3965 trunc from 0xce62874 to 0xaac794d 2024-09-17T22:42:22.106 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:22.103+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:22.113 INFO:teuthology.orchestra.run.smithi115.stdout:3967 read 0x17389f0 thru 0x173d1a6 (0x47b7 bytes) 2024-09-17T22:42:22.113 INFO:teuthology.orchestra.run.smithi115.stdout:3971 read 0xaa78ffb thru 0xaa80a0e (0x7a14 bytes) 2024-09-17T22:42:22.120 INFO:teuthology.orchestra.run.smithi115.stdout:3972 write 0xb9b5b01 thru 0xb9c1c69 (0xc169 bytes) 2024-09-17T22:42:22.120 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:22.120 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:22.120 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:22.120 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:22.124 INFO:teuthology.orchestra.run.smithi115.stdout:3973 punch from 0x7ae7bf8 to 0x7aeb0d4, (0x34dc bytes) 2024-09-17T22:42:22.124 INFO:teuthology.orchestra.run.smithi115.stdout:3974 punch from 0xf06519 to 0xf0d954, (0x743b bytes) 2024-09-17T22:42:22.130 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:42:22.131 INFO:teuthology.orchestra.run.smithi115.stdout:3975 trunc from 0xb9c1c6a to 0x5b7af8a 2024-09-17T22:42:22.148 INFO:teuthology.orchestra.run.smithi115.stdout:3976 trunc from 0x5b7af8a to 0xb93089f 2024-09-17T22:42:22.156 INFO:teuthology.orchestra.run.smithi115.stdout:3977 trunc from 0xb93089f to 0xa06ff39 2024-09-17T22:42:22.161 INFO:teuthology.orchestra.run.smithi115.stdout:3978 read 0x39ce3a8 thru 0x39d7d68 (0x99c1 bytes) 2024-09-17T22:42:22.168 INFO:teuthology.orchestra.run.smithi115.stdout:3979 write 0xab778e1 thru 0xab7ccaf (0x53cf bytes) 2024-09-17T22:42:22.171 INFO:teuthology.orchestra.run.smithi115.stdout:3980 read 0x27db496 thru 0x27dc867 (0x13d2 bytes) 2024-09-17T22:42:22.171 INFO:teuthology.orchestra.run.smithi115.stdout:3982 trunc from 0xab7ccb0 to 0xa78ddba 2024-09-17T22:42:22.177 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.177 INFO:teuthology.orchestra.run.smithi115.stdout:3984 writesame 0x95f2f0b thru 0x95ffb90 data_size 0x2(0xcc86 bytes) 2024-09-17T22:42:22.187 INFO:teuthology.orchestra.run.smithi115.stdout:3985 read 0x90ae23d thru 0x90b43fe (0x61c2 bytes) 2024-09-17T22:42:22.196 INFO:teuthology.orchestra.run.smithi115.stdout:3986 trunc from 0xa78ddba to 0x2a35863 2024-09-17T22:42:22.213 INFO:teuthology.orchestra.run.smithi115.stdout:3987 write 0x3e13b67 thru 0x3e16727 (0x2bc1 bytes) 2024-09-17T22:42:22.214 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:22.211+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:22.225 INFO:teuthology.orchestra.run.smithi115.stdout:3988 write 0x6ce9a17 thru 0x6cf18c6 (0x7eb0 bytes) 2024-09-17T22:42:22.242 INFO:teuthology.orchestra.run.smithi115.stdout:3989 write 0xd46ad64 thru 0xd47789e (0xcb3b bytes) 2024-09-17T22:42:22.246 INFO:teuthology.orchestra.run.smithi115.stdout:3990 read 0x673c261 thru 0x6748c20 (0xc9c0 bytes) 2024-09-17T22:42:22.246 INFO:teuthology.orchestra.run.smithi115.stdout:3991 write 0x7bf1bf7 thru 0x7bf390a (0x1d14 bytes) 2024-09-17T22:42:22.246 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is set 2024-09-17T22:42:22.251 INFO:teuthology.orchestra.run.smithi115.stdout:3992 read 0x343a84f thru 0x3447bea (0xd39c bytes) 2024-09-17T22:42:22.251 INFO:teuthology.orchestra.run.smithi115.stdout:3994 punch from 0x1a66e68 to 0x1a6b406, (0x459e bytes) 2024-09-17T22:42:22.251 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.251 INFO:teuthology.orchestra.run.smithi115.stdout:3997 writesame 0x95701fb thru 0x9574fd0 data_size 0x29(0x4dd6 bytes) 2024-09-17T22:42:22.264 INFO:teuthology.orchestra.run.smithi115.stdout:3998 writesame 0xe4556fe thru 0xe45b2e0 data_size 0x1(0x5be3 bytes) 2024-09-17T22:42:22.264 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.264 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.264 INFO:teuthology.orchestra.run.smithi115.stdout:4001 writesame 0x29f2558 thru 0x29f5b7c data_size 0x1(0x3625 bytes) 2024-09-17T22:42:22.284 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:22.284 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:22.284 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:22.284 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:22.284 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:22.287 INFO:teuthology.orchestra.run.smithi115.stdout:4002 trunc from 0xe45b2e1 to 0x1f25851 2024-09-17T22:42:22.294 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:42:22.303 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.303 INFO:teuthology.orchestra.run.smithi115.stdout:4004 punch from 0x17a9e9 to 0x1832dc, (0x88f3 bytes) 2024-09-17T22:42:22.304 INFO:teuthology.orchestra.run.smithi115.stdout:4005 read 0x621b16 thru 0x627b4f (0x603a bytes) 2024-09-17T22:42:22.308 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:22.305+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:22.322 INFO:teuthology.orchestra.run.smithi115.stdout:4006 writesame 0xa6ac556 thru 0xa6b96ed data_size 0x26(0xd198 bytes) 2024-09-17T22:42:22.322 INFO:teuthology.orchestra.run.smithi115.stdout:4008 writesame 0xfa2248 thru 0xfa8d66 data_size 0x21(0x6b1f bytes) 2024-09-17T22:42:22.323 INFO:teuthology.orchestra.run.smithi115.stdout:4009 read 0x7a1a3b8 thru 0x7a24d6d (0xa9b6 bytes) 2024-09-17T22:42:22.324 INFO:teuthology.orchestra.run.smithi115.stdout:4010 read 0xa38aaab thru 0xa3902d9 (0x582f bytes) 2024-09-17T22:42:22.324 INFO:teuthology.orchestra.run.smithi115.stdout:4011 writesame 0x491a60e thru 0x491c3f9 data_size 0x2(0x1dec bytes) 2024-09-17T22:42:22.328 INFO:teuthology.orchestra.run.smithi115.stdout:4012 read 0x4ecbf82 thru 0x4ed2ebb (0x6f3a bytes) 2024-09-17T22:42:22.328 INFO:teuthology.orchestra.run.smithi115.stdout:4014 write 0x1ebe1e9 thru 0x1ec3a0c (0x5824 bytes) 2024-09-17T22:42:22.329 INFO:teuthology.orchestra.run.smithi115.stdout:4015 punch from 0x49eda61 to 0x49ee3c0, (0x95f bytes) 2024-09-17T22:42:22.329 INFO:teuthology.orchestra.run.smithi115.stdout:4016 punch from 0x756d052 to 0x75783d0, (0xb37e bytes) 2024-09-17T22:42:22.329 INFO:teuthology.orchestra.run.smithi115.stdout:4017 write 0x32f6ff7 thru 0x32f80f4 (0x10fe bytes) 2024-09-17T22:42:22.332 INFO:teuthology.orchestra.run.smithi115.stdout:4018 write 0x4616e88 thru 0x4620717 (0x9890 bytes) 2024-09-17T22:42:22.335 INFO:teuthology.orchestra.run.smithi115.stdout:4019 read 0x8bce0b6 thru 0x8bdc7ff (0xe74a bytes) 2024-09-17T22:42:22.345 INFO:teuthology.orchestra.run.smithi115.stdout:4020 write 0xcfe3769 thru 0xcfed5b4 (0x9e4c bytes) 2024-09-17T22:42:22.360 INFO:teuthology.orchestra.run.smithi115.stdout:4021 trunc from 0xcfed5b5 to 0x226afc0 2024-09-17T22:42:22.386 INFO:teuthology.orchestra.run.smithi115.stdout:4022 writesame 0x7a59940 thru 0x7a65c0d data_size 0x5(0xc2ce bytes) 2024-09-17T22:42:22.399 INFO:teuthology.orchestra.run.smithi115.stdout:4023 write 0xd589eb6 thru 0xd58ca65 (0x2bb0 bytes) 2024-09-17T22:42:22.408 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:22.406+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:22.416 INFO:teuthology.orchestra.run.smithi115.stdout:4024 trunc from 0xd58ca66 to 0x1f21a8c 2024-09-17T22:42:22.446 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:22.446 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:22.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:22.446 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:22.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.446 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202974 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.447 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725913+0000", 2024-09-17T22:42:22.447 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7170168710000002, 2024-09-17T22:42:22.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022636643000000001, 2024-09-17T22:42:22.447 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.447 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.447 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.447 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.447 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.447 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.447 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202974 2024-09-17T22:42:22.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.447 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725913+0000", 2024-09-17T22:42:22.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725913+0000", 2024-09-17T22:42:22.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725913+0000", 2024-09-17T22:42:22.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5199999999999997e-07 2024-09-17T22:42:22.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725916+0000", 2024-09-17T22:42:22.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8100000000000002e-06 2024-09-17T22:42:22.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725924+0000", 2024-09-17T22:42:22.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1289999999999996e-06 2024-09-17T22:42:22.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725930+0000", 2024-09-17T22:42:22.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3469999999999998e-06 2024-09-17T22:42:22.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748360+0000", 2024-09-17T22:42:22.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022430410000000001 2024-09-17T22:42:22.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748443+0000", 2024-09-17T22:42:22.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3355000000000004e-05 2024-09-17T22:42:22.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748550+0000", 2024-09-17T22:42:22.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010603999999999999 2024-09-17T22:42:22.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.451 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.451 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202975 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.451 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725937+0000", 2024-09-17T22:42:22.451 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7169928319999999, 2024-09-17T22:42:22.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022787498, 2024-09-17T22:42:22.451 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.452 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.452 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.452 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202975 2024-09-17T22:42:22.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.452 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725937+0000", 2024-09-17T22:42:22.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725937+0000", 2024-09-17T22:42:22.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725937+0000", 2024-09-17T22:42:22.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4299999999999998e-07 2024-09-17T22:42:22.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725939+0000", 2024-09-17T22:42:22.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0669999999999999e-06 2024-09-17T22:42:22.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725946+0000", 2024-09-17T22:42:22.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6390000000000003e-06 2024-09-17T22:42:22.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725951+0000", 2024-09-17T22:42:22.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0089999999999998e-06 2024-09-17T22:42:22.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748593+0000", 2024-09-17T22:42:22.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022642343999999998 2024-09-17T22:42:22.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748657+0000", 2024-09-17T22:42:22.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3133999999999996e-05 2024-09-17T22:42:22.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748724+0000", 2024-09-17T22:42:22.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7861999999999995e-05 2024-09-17T22:42:22.455 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.455 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.456 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.456 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202976 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.456 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725958+0000", 2024-09-17T22:42:22.456 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7169716509999997, 2024-09-17T22:42:22.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022890631000000002, 2024-09-17T22:42:22.456 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.456 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.456 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.456 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.456 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.456 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.456 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202976 2024-09-17T22:42:22.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.457 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725958+0000", 2024-09-17T22:42:22.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725958+0000", 2024-09-17T22:42:22.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725959+0000", 2024-09-17T22:42:22.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7199999999999999e-07 2024-09-17T22:42:22.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725961+0000", 2024-09-17T22:42:22.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9860000000000001e-06 2024-09-17T22:42:22.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725967+0000", 2024-09-17T22:42:22.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6630000000000004e-06 2024-09-17T22:42:22.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725972+0000", 2024-09-17T22:42:22.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0549999999999999e-06 2024-09-17T22:42:22.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748753+0000", 2024-09-17T22:42:22.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022780959 2024-09-17T22:42:22.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748792+0000", 2024-09-17T22:42:22.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8581999999999998e-05 2024-09-17T22:42:22.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748849+0000", 2024-09-17T22:42:22.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6913999999999998e-05 2024-09-17T22:42:22.460 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.460 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.460 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.460 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202977 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.460 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:42:22.460 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7169506940000003, 2024-09-17T22:42:22.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023068750999999998, 2024-09-17T22:42:22.461 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.461 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.461 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.461 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202977 2024-09-17T22:42:22.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.461 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:42:22.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:42:22.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725980+0000", 2024-09-17T22:42:22.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:42:22.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725982+0000", 2024-09-17T22:42:22.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0669999999999999e-06 2024-09-17T22:42:22.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725988+0000", 2024-09-17T22:42:22.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6610000000000003e-06 2024-09-17T22:42:22.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725993+0000", 2024-09-17T22:42:22.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:42:22.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748888+0000", 2024-09-17T22:42:22.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022894930000000001 2024-09-17T22:42:22.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748951+0000", 2024-09-17T22:42:22.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3638000000000003e-05 2024-09-17T22:42:22.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749048+0000", 2024-09-17T22:42:22.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6343999999999998e-05 2024-09-17T22:42:22.464 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.465 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.465 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202978 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2211840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.465 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:22.465 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7169301820000005, 2024-09-17T22:42:22.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023238050999999999, 2024-09-17T22:42:22.465 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.465 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.465 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.465 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.466 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.466 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.466 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202978 2024-09-17T22:42:22.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.466 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:22.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:22.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:22.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.58e-07 2024-09-17T22:42:22.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726002+0000", 2024-09-17T22:42:22.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.193e-06 2024-09-17T22:42:22.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726009+0000", 2024-09-17T22:42:22.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8000000000000001e-06 2024-09-17T22:42:22.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726014+0000", 2024-09-17T22:42:22.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9840000000000001e-06 2024-09-17T22:42:22.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749090+0000", 2024-09-17T22:42:22.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023075917000000001 2024-09-17T22:42:22.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749163+0000", 2024-09-17T22:42:22.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3219999999999994e-05 2024-09-17T22:42:22.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749238+0000", 2024-09-17T22:42:22.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4479000000000003e-05 2024-09-17T22:42:22.469 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.469 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.469 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.469 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202979 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2342912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.470 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:42:22.470 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7169087310000002, 2024-09-17T22:42:22.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023393536999999999, 2024-09-17T22:42:22.470 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.470 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.470 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.470 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202979 2024-09-17T22:42:22.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.470 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:42:22.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:42:22.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726022+0000", 2024-09-17T22:42:22.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:42:22.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726024+0000", 2024-09-17T22:42:22.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1969999999999999e-06 2024-09-17T22:42:22.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726030+0000", 2024-09-17T22:42:22.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.691e-06 2024-09-17T22:42:22.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726035+0000", 2024-09-17T22:42:22.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8029999999999996e-06 2024-09-17T22:42:22.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749280+0000", 2024-09-17T22:42:22.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023244668 2024-09-17T22:42:22.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749344+0000", 2024-09-17T22:42:22.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4418000000000006e-05 2024-09-17T22:42:22.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749415+0000", 2024-09-17T22:42:22.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0272999999999996e-05 2024-09-17T22:42:22.474 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.474 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.474 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.474 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202980 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2473984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.474 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:22.474 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7168879620000004, 2024-09-17T22:42:22.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023562221000000001, 2024-09-17T22:42:22.474 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.474 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.474 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.474 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.475 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.475 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.475 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202980 2024-09-17T22:42:22.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.475 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:22.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:22.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:22.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:42:22.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726044+0000", 2024-09-17T22:42:22.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9489999999999999e-06 2024-09-17T22:42:22.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726066+0000", 2024-09-17T22:42:22.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1508000000000001e-05 2024-09-17T22:42:22.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726072+0000", 2024-09-17T22:42:22.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3450000000000002e-06 2024-09-17T22:42:22.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749455+0000", 2024-09-17T22:42:22.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023382824999999999 2024-09-17T22:42:22.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749527+0000", 2024-09-17T22:42:22.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1728000000000002e-05 2024-09-17T22:42:22.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749604+0000", 2024-09-17T22:42:22.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7428999999999993e-05 2024-09-17T22:42:22.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.478 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.479 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202981 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.479 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:42:22.479 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7168496869999998, 2024-09-17T22:42:22.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023677800999999998, 2024-09-17T22:42:22.479 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.479 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.479 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.479 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202981 2024-09-17T22:42:22.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.480 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:42:22.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:42:22.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726081+0000", 2024-09-17T22:42:22.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5499999999999998e-07 2024-09-17T22:42:22.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726083+0000", 2024-09-17T22:42:22.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1780000000000002e-06 2024-09-17T22:42:22.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726090+0000", 2024-09-17T22:42:22.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8510000000000001e-06 2024-09-17T22:42:22.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726101+0000", 2024-09-17T22:42:22.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.133e-05 2024-09-17T22:42:22.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749639+0000", 2024-09-17T22:42:22.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023537679999999998 2024-09-17T22:42:22.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749688+0000", 2024-09-17T22:42:22.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9548000000000001e-05 2024-09-17T22:42:22.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749758+0000", 2024-09-17T22:42:22.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9758999999999994e-05 2024-09-17T22:42:22.483 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.483 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.483 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.483 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202982 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.483 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:22.483 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7168221040000002, 2024-09-17T22:42:22.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023841469000000001, 2024-09-17T22:42:22.484 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.484 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.484 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.484 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.484 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.484 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.484 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202982 2024-09-17T22:42:22.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.484 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:22.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:22.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:22.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:42:22.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726110+0000", 2024-09-17T22:42:22.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.902e-06 2024-09-17T22:42:22.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726117+0000", 2024-09-17T22:42:22.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6189999999999996e-06 2024-09-17T22:42:22.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726122+0000", 2024-09-17T22:42:22.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.198e-06 2024-09-17T22:42:22.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749795+0000", 2024-09-17T22:42:22.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023673397999999998 2024-09-17T22:42:22.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749877+0000", 2024-09-17T22:42:22.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1910999999999996e-05 2024-09-17T22:42:22.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749949+0000", 2024-09-17T22:42:22.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2001000000000004e-05 2024-09-17T22:42:22.487 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.488 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.488 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.488 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202983 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.488 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:22.488 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7168010450000004, 2024-09-17T22:42:22.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024014081, 2024-09-17T22:42:22.488 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.488 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.488 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.489 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.489 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202983 2024-09-17T22:42:22.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.489 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:22.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:22.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:22.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:42:22.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726131+0000", 2024-09-17T22:42:22.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9429999999999999e-06 2024-09-17T22:42:22.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726138+0000", 2024-09-17T22:42:22.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T22:42:22.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726143+0000", 2024-09-17T22:42:22.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0629999999999996e-06 2024-09-17T22:42:22.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749991+0000", 2024-09-17T22:42:22.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023847671000000001 2024-09-17T22:42:22.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750065+0000", 2024-09-17T22:42:22.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4569999999999999e-05 2024-09-17T22:42:22.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750143+0000", 2024-09-17T22:42:22.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7649000000000001e-05 2024-09-17T22:42:22.492 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.492 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.492 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.492 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202984 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.493 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:42:22.493 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.716780344, 2024-09-17T22:42:22.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024177605000000001, 2024-09-17T22:42:22.493 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.493 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.493 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.493 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.493 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.493 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.493 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202984 2024-09-17T22:42:22.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.493 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:42:22.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:42:22.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726150+0000", 2024-09-17T22:42:22.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T22:42:22.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726152+0000", 2024-09-17T22:42:22.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:42:22.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726162+0000", 2024-09-17T22:42:22.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7839999999999998e-06 2024-09-17T22:42:22.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726167+0000", 2024-09-17T22:42:22.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5690000000000004e-06 2024-09-17T22:42:22.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750184+0000", 2024-09-17T22:42:22.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024016909999999999 2024-09-17T22:42:22.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750255+0000", 2024-09-17T22:42:22.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1137999999999996e-05 2024-09-17T22:42:22.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750327+0000", 2024-09-17T22:42:22.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1903000000000006e-05 2024-09-17T22:42:22.497 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.497 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.497 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.497 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202985 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.498 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:22.498 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.716754957, 2024-09-17T22:42:22.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024280500999999999, 2024-09-17T22:42:22.498 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.498 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.498 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.498 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.498 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.498 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.498 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202985 2024-09-17T22:42:22.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.498 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:22.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:22.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:22.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8100000000000003e-07 2024-09-17T22:42:22.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726182+0000", 2024-09-17T22:42:22.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:42:22.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726190+0000", 2024-09-17T22:42:22.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:42:22.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726195+0000", 2024-09-17T22:42:22.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2809999999999999e-06 2024-09-17T22:42:22.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750366+0000", 2024-09-17T22:42:22.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024171635 2024-09-17T22:42:22.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750410+0000", 2024-09-17T22:42:22.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3823999999999999e-05 2024-09-17T22:42:22.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750455+0000", 2024-09-17T22:42:22.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5039e-05 2024-09-17T22:42:22.502 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.502 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.502 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.502 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202986 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.502 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:42:22.502 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7167273769999998, 2024-09-17T22:42:22.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024453077, 2024-09-17T22:42:22.502 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.502 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.502 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.503 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.503 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.503 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.503 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202986 2024-09-17T22:42:22.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.503 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:42:22.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:42:22.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726203+0000", 2024-09-17T22:42:22.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T22:42:22.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726205+0000", 2024-09-17T22:42:22.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0250000000000001e-06 2024-09-17T22:42:22.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726211+0000", 2024-09-17T22:42:22.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1340000000000001e-06 2024-09-17T22:42:22.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726222+0000", 2024-09-17T22:42:22.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1105e-05 2024-09-17T22:42:22.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750520+0000", 2024-09-17T22:42:22.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024297462999999998 2024-09-17T22:42:22.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750585+0000", 2024-09-17T22:42:22.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5940999999999998e-05 2024-09-17T22:42:22.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750655+0000", 2024-09-17T22:42:22.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0012000000000007e-05 2024-09-17T22:42:22.506 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.506 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.506 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.507 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202987 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.507 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:22.507 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7167009970000002, 2024-09-17T22:42:22.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024622569, 2024-09-17T22:42:22.507 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.507 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.507 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.507 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.507 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.507 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.507 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202987 2024-09-17T22:42:22.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.507 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:22.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:22.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:22.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:42:22.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726238+0000", 2024-09-17T22:42:22.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7430000000000008e-06 2024-09-17T22:42:22.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726245+0000", 2024-09-17T22:42:22.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6640000000000001e-06 2024-09-17T22:42:22.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726249+0000", 2024-09-17T22:42:22.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9119999999999997e-06 2024-09-17T22:42:22.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750694+0000", 2024-09-17T22:42:22.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024444105000000001 2024-09-17T22:42:22.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750767+0000", 2024-09-17T22:42:22.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3577000000000004e-05 2024-09-17T22:42:22.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750851+0000", 2024-09-17T22:42:22.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4189000000000003e-05 2024-09-17T22:42:22.511 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.511 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.511 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.511 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202988 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.511 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:22.511 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7166741349999999, 2024-09-17T22:42:22.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024808883, 2024-09-17T22:42:22.511 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.511 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.512 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.512 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.512 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.512 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.512 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202988 2024-09-17T22:42:22.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.512 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:22.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:22.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:22.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5499999999999999e-07 2024-09-17T22:42:22.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726258+0000", 2024-09-17T22:42:22.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8020000000000001e-06 2024-09-17T22:42:22.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726272+0000", 2024-09-17T22:42:22.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3828e-05 2024-09-17T22:42:22.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726277+0000", 2024-09-17T22:42:22.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2379999999999997e-06 2024-09-17T22:42:22.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750895+0000", 2024-09-17T22:42:22.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024618543 2024-09-17T22:42:22.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750978+0000", 2024-09-17T22:42:22.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3030999999999998e-05 2024-09-17T22:42:22.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751065+0000", 2024-09-17T22:42:22.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6086000000000002e-05 2024-09-17T22:42:22.515 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.515 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.515 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.516 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202989 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.516 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:42:22.516 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7166452620000001, 2024-09-17T22:42:22.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024976788, 2024-09-17T22:42:22.516 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.516 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.516 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.516 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.516 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.516 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.516 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202989 2024-09-17T22:42:22.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.517 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:42:22.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:42:22.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726285+0000", 2024-09-17T22:42:22.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T22:42:22.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726287+0000", 2024-09-17T22:42:22.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9860000000000001e-06 2024-09-17T22:42:22.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726294+0000", 2024-09-17T22:42:22.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2849999999999997e-06 2024-09-17T22:42:22.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726299+0000", 2024-09-17T22:42:22.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0150000000000003e-06 2024-09-17T22:42:22.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751109+0000", 2024-09-17T22:42:22.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024810175 2024-09-17T22:42:22.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751184+0000", 2024-09-17T22:42:22.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4640000000000004e-05 2024-09-17T22:42:22.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751261+0000", 2024-09-17T22:42:22.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7317000000000005e-05 2024-09-17T22:42:22.520 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.520 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.520 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.520 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202990 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.520 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:22.520 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.716625047, 2024-09-17T22:42:22.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025156767999999999, 2024-09-17T22:42:22.521 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.521 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.521 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.521 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.521 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.521 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.521 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202990 2024-09-17T22:42:22.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.521 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:22.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:22.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:22.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:42:22.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726307+0000", 2024-09-17T22:42:22.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7090000000000001e-06 2024-09-17T22:42:22.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726312+0000", 2024-09-17T22:42:22.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4909999999999996e-06 2024-09-17T22:42:22.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726317+0000", 2024-09-17T22:42:22.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3800000000000004e-06 2024-09-17T22:42:22.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751308+0000", 2024-09-17T22:42:22.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024991143 2024-09-17T22:42:22.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751385+0000", 2024-09-17T22:42:22.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6793999999999997e-05 2024-09-17T22:42:22.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751461+0000", 2024-09-17T22:42:22.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6891999999999995e-05 2024-09-17T22:42:22.525 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.525 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.525 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.525 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202991 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.525 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:22.525 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.716607013, 2024-09-17T22:42:22.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025289477000000001, 2024-09-17T22:42:22.525 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.525 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.525 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.525 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.526 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.526 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.526 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202991 2024-09-17T22:42:22.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.526 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:22.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:22.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:22.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.27e-07 2024-09-17T22:42:22.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726325+0000", 2024-09-17T22:42:22.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1459999999999999e-06 2024-09-17T22:42:22.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726333+0000", 2024-09-17T22:42:22.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1289999999999996e-06 2024-09-17T22:42:22.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726339+0000", 2024-09-17T22:42:22.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2530000000000004e-06 2024-09-17T22:42:22.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751499+0000", 2024-09-17T22:42:22.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025160109 2024-09-17T22:42:22.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751543+0000", 2024-09-17T22:42:22.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4317000000000003e-05 2024-09-17T22:42:22.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751612+0000", 2024-09-17T22:42:22.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9096000000000004e-05 2024-09-17T22:42:22.529 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.529 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.529 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.529 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202992 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.529 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:22.530 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7165849489999996, 2024-09-17T22:42:22.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025420585999999998, 2024-09-17T22:42:22.530 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.530 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.530 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.530 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.530 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202992 2024-09-17T22:42:22.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.530 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:22.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:22.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:22.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5199999999999998e-07 2024-09-17T22:42:22.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726347+0000", 2024-09-17T22:42:22.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9939999999999999e-06 2024-09-17T22:42:22.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726358+0000", 2024-09-17T22:42:22.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0421999999999999e-05 2024-09-17T22:42:22.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726363+0000", 2024-09-17T22:42:22.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:42:22.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751644+0000", 2024-09-17T22:42:22.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025280727999999999 2024-09-17T22:42:22.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751712+0000", 2024-09-17T22:42:22.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8207999999999995e-05 2024-09-17T22:42:22.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751765+0000", 2024-09-17T22:42:22.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3533999999999999e-05 2024-09-17T22:42:22.534 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.534 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.534 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.535 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202993 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:22.536 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:22.536 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 4.7165609579999996, 2024-09-17T22:42:22.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025516259999999999, 2024-09-17T22:42:22.536 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:22.536 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:22.536 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:22.536 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:22.536 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:22.536 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.536 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202993 2024-09-17T22:42:22.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.536 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:22.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:22.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:22.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:22.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:22.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:22.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:22.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:22.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:42:22.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:22.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726371+0000", 2024-09-17T22:42:22.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T22:42:22.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:22.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726380+0000", 2024-09-17T22:42:22.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9109999999999999e-06 2024-09-17T22:42:22.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726385+0000", 2024-09-17T22:42:22.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7380000000000002e-06 2024-09-17T22:42:22.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:22.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751785+0000", 2024-09-17T22:42:22.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025400526 2024-09-17T22:42:22.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:22.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751824+0000", 2024-09-17T22:42:22.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8989999999999998e-05 2024-09-17T22:42:22.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:22.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:22.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:22.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751885+0000", 2024-09-17T22:42:22.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0937000000000003e-05 2024-09-17T22:42:22.540 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.540 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.540 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.540 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:22.540 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:22.540 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:22.542 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:22.538+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:22.543 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:42:22.544 INFO:teuthology.orchestra.run.smithi115.stdout:4025 writesame 0xc690aaa thru 0xc69ae0d data_size 0x2(0xa364 bytes) 2024-09-17T22:42:22.544 INFO:teuthology.orchestra.run.smithi115.stdout:4026 punch from 0xa97b6cb to 0xa98ab29, (0xf45e bytes) 2024-09-17T22:42:22.544 INFO:teuthology.orchestra.run.smithi115.stdout:4027 write 0xab8d00f thru 0xab99869 (0xc85b bytes) 2024-09-17T22:42:22.544 INFO:teuthology.orchestra.run.smithi115.stdout:4031 trunc from 0xc69ae0e to 0x5af17d4 2024-09-17T22:42:22.544 INFO:teuthology.orchestra.run.smithi115.stdout:4032 read 0x1fbe00b thru 0x1fc25e8 (0x45de bytes) 2024-09-17T22:42:22.544 INFO:teuthology.orchestra.run.smithi115.stdout:4033 trunc from 0x5af17d4 to 0xa1596c9 2024-09-17T22:42:22.544 INFO:teuthology.orchestra.run.smithi115.stdout:4034 write 0xeaac6f2 thru 0xeab7c7a (0xb589 bytes) 2024-09-17T22:42:22.544 INFO:teuthology.orchestra.run.smithi115.stdout:4035 write 0x3a6112b thru 0x3a6721d (0x60f3 bytes) 2024-09-17T22:42:22.544 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.544 INFO:teuthology.orchestra.run.smithi115.stdout:4037 write 0xce65bb4 thru 0xce74d7a (0xf1c7 bytes) 2024-09-17T22:42:22.545 INFO:teuthology.orchestra.run.smithi115.stdout:4039 punch from 0x3f6a758 to 0x3f77280, (0xcb28 bytes) 2024-09-17T22:42:22.545 INFO:teuthology.orchestra.run.smithi115.stdout:4040 write 0x912495d thru 0x912ef29 (0xa5cd bytes) 2024-09-17T22:42:22.545 INFO:teuthology.orchestra.run.smithi115.stdout:4042 write 0xa6e4497 thru 0xa6ee406 (0x9f70 bytes) 2024-09-17T22:42:22.545 INFO:teuthology.orchestra.run.smithi115.stdout:4043 write 0x88ea422 thru 0x88efa3f (0x561e bytes) 2024-09-17T22:42:22.545 INFO:teuthology.orchestra.run.smithi115.stdout:4044 trunc from 0xeab7c7b to 0x3bb0af4 2024-09-17T22:42:22.545 INFO:teuthology.orchestra.run.smithi115.stdout:4045 read 0x82080d thru 0x829296 (0x8a8a bytes) 2024-09-17T22:42:22.547 INFO:teuthology.orchestra.run.smithi115.stdout:4048 writesame 0xac90e29 thru 0xac9b973 data_size 0x2f(0xab4b bytes) 2024-09-17T22:42:22.547 INFO:teuthology.orchestra.run.smithi115.stdout:4049 writesame 0x7e30fac thru 0x7e34904 data_size 0x1(0x3959 bytes) 2024-09-17T22:42:22.554 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.554 INFO:teuthology.orchestra.run.smithi115.stdout:truncating to largest ever: 0xed7caa2 2024-09-17T22:42:22.558 INFO:teuthology.orchestra.run.smithi115.stdout:4051 trunc from 0xac9b974 to 0xed7caa2 2024-09-17T22:42:22.564 INFO:teuthology.orchestra.run.smithi115.stdout:4052 write 0xe1c1988 thru 0xe1c829f (0x6918 bytes) 2024-09-17T22:42:22.571 INFO:teuthology.orchestra.run.smithi115.stdout:4053 trunc from 0xed7caa2 to 0xb7ecbc0 2024-09-17T22:42:22.580 INFO:teuthology.orchestra.run.smithi115.stdout:truncating to largest ever: 0xee460d9 2024-09-17T22:42:22.584 INFO:teuthology.orchestra.run.smithi115.stdout:4054 trunc from 0xb7ecbc0 to 0xee460d9 2024-09-17T22:42:22.589 INFO:teuthology.orchestra.run.smithi115.stdout:4056 read 0x883b3d0 thru 0x8841cb0 (0x68e1 bytes) 2024-09-17T22:42:22.589 INFO:teuthology.orchestra.run.smithi115.stdout:4058 writesame 0xb46cffb thru 0xb472011 data_size 0x1d(0x5017 bytes) 2024-09-17T22:42:22.593 INFO:teuthology.orchestra.run.smithi115.stdout:4059 write 0x2d0049a thru 0x2d0dcc1 (0xd828 bytes) 2024-09-17T22:42:22.596 INFO:teuthology.orchestra.run.smithi115.stdout:4060 read 0x17bfa70 thru 0x17cd6d4 (0xdc65 bytes) 2024-09-17T22:42:22.596 INFO:teuthology.orchestra.run.smithi115.stdout:4061 read 0x3a43aa1 thru 0x3a47158 (0x36b8 bytes) 2024-09-17T22:42:22.596 INFO:teuthology.orchestra.run.smithi115.stdout:4062 read 0x753339b thru 0x7539ea4 (0x6b0a bytes) 2024-09-17T22:42:22.596 INFO:teuthology.orchestra.run.smithi115.stdout:4064 writesame 0x512e15 thru 0x520b78 data_size 0x6(0xdd64 bytes) 2024-09-17T22:42:22.602 INFO:teuthology.orchestra.run.smithi115.stdout:4065 write 0x693f260 thru 0x694addb (0xbb7c bytes) 2024-09-17T22:42:22.606 INFO:teuthology.orchestra.run.smithi115.stdout:4066 write 0x398a806 thru 0x3997edb (0xd6d6 bytes) 2024-09-17T22:42:22.609 INFO:teuthology.orchestra.run.smithi115.stdout:4068 writesame 0xbb94c62 thru 0xbb954cc data_size 0x1(0x86b bytes) 2024-09-17T22:42:22.612 INFO:teuthology.orchestra.run.smithi115.stdout:4070 punch from 0x27355c3 to 0x2738888, (0x32c5 bytes) 2024-09-17T22:42:22.613 INFO:teuthology.orchestra.run.smithi115.stdout:4071 write 0x6600ccd thru 0x6605ec9 (0x51fd bytes) 2024-09-17T22:42:22.616 INFO:teuthology.orchestra.run.smithi115.stdout:4072 read 0x539a8e4 thru 0x539b71d (0xe3a bytes) 2024-09-17T22:42:22.633 INFO:teuthology.orchestra.run.smithi115.stdout:4073 trunc from 0xee460d9 to 0x91aaba 2024-09-17T22:42:22.642 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:22.640+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:22.658 INFO:teuthology.orchestra.run.smithi115.stdout:4074 write 0x66a8c3c thru 0x66b2d05 (0xa0ca bytes) 2024-09-17T22:42:22.660 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:22.660 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:22.660 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:22.660 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:22.661 INFO:teuthology.orchestra.run.smithi115.stdout:4075 trunc from 0x66b2d06 to 0x65cd23b 2024-09-17T22:42:22.670 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:42:22.675 INFO:teuthology.orchestra.run.smithi115.stdout:4076 writesame 0x8d1719b thru 0x8d24f27 data_size 0x1(0xdd8d bytes) 2024-09-17T22:42:22.675 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.676 INFO:teuthology.orchestra.run.smithi115.stdout:4079 read 0x5669364 thru 0x56744c3 (0xb160 bytes) 2024-09-17T22:42:22.676 INFO:teuthology.orchestra.run.smithi115.stdout:4080 writesame 0x5f28b14 thru 0x5f36340 data_size 0x2b(0xd82d bytes) 2024-09-17T22:42:22.678 INFO:teuthology.orchestra.run.smithi115.stdout:4081 punch from 0x3cf768f to 0x3d03405, (0xbd76 bytes) 2024-09-17T22:42:22.679 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.679 INFO:teuthology.orchestra.run.smithi115.stdout:4084 read 0x6a3da4b thru 0x6a43988 (0x5f3e bytes) 2024-09-17T22:42:22.679 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.689 INFO:teuthology.orchestra.run.smithi115.stdout:4086 write 0xde1e408 thru 0xde20bcc (0x27c5 bytes) 2024-09-17T22:42:22.692 INFO:teuthology.orchestra.run.smithi115.stdout:4088 read 0x7608539 thru 0x760909e (0xb66 bytes) 2024-09-17T22:42:22.692 INFO:teuthology.orchestra.run.smithi115.stdout:4089 read 0x8f7d4ba thru 0x8f85a73 (0x85ba bytes) 2024-09-17T22:42:22.692 INFO:teuthology.orchestra.run.smithi115.stdout:4090 write 0x720332 thru 0x72fd2a (0xf9f9 bytes) 2024-09-17T22:42:22.694 INFO:teuthology.orchestra.run.smithi115.stdout:4091 write 0x69edfb2 thru 0x69ef1dd (0x122c bytes) 2024-09-17T22:42:22.697 INFO:teuthology.orchestra.run.smithi115.stdout:4092 write 0x9a8d189 thru 0x9a9a8f6 (0xd76e bytes) 2024-09-17T22:42:22.700 INFO:teuthology.orchestra.run.smithi115.stdout:4094 read 0xc567e95 thru 0xc57730e (0xf47a bytes) 2024-09-17T22:42:22.701 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.701 INFO:teuthology.orchestra.run.smithi115.stdout:4097 write 0x5a94059 thru 0x5a9bb53 (0x7afb bytes) 2024-09-17T22:42:22.711 INFO:teuthology.orchestra.run.smithi115.stdout:4098 trunc from 0xde20bcd to 0x71dd8a3 2024-09-17T22:42:22.721 INFO:teuthology.orchestra.run.smithi115.stdout:4099 write 0x5bd4863 thru 0x5bd9b4a (0x52e8 bytes) 2024-09-17T22:42:22.735 INFO:teuthology.orchestra.run.smithi115.stdout:4100 writesame 0xb4f22a8 thru 0xb4fe860 data_size 0x7(0xc5b9 bytes) 2024-09-17T22:42:22.735 INFO:teuthology.orchestra.run.smithi115.stdout:4101 writesame 0x788c4cc thru 0x78905a1 data_size 0x2b(0x40d6 bytes) 2024-09-17T22:42:22.743 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:22.740+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:22.746 INFO:teuthology.orchestra.run.smithi115.stdout:4103 writesame 0xcfc4c23 thru 0xcfcfd84 data_size 0x26(0xb162 bytes) 2024-09-17T22:42:22.755 INFO:teuthology.orchestra.run.smithi115.stdout:4104 write 0xdf5db6f thru 0xdf68c5d (0xb0ef bytes) 2024-09-17T22:42:22.758 INFO:teuthology.orchestra.run.smithi115.stdout:4105 writesame 0xcbe43a9 thru 0xcbf00a7 data_size 0x1(0xbcff bytes) 2024-09-17T22:42:22.776 INFO:teuthology.orchestra.run.smithi115.stdout:4106 read 0x21119c5 thru 0x21160a8 (0x46e4 bytes) 2024-09-17T22:42:22.776 INFO:teuthology.orchestra.run.smithi115.stdout:4107 read 0x3a556be thru 0x3a652e7 (0xfc2a bytes) 2024-09-17T22:42:22.776 INFO:teuthology.orchestra.run.smithi115.stdout:4108 write 0x7660d2c thru 0x766f41e (0xe6f3 bytes) 2024-09-17T22:42:22.779 INFO:teuthology.orchestra.run.smithi115.stdout:4109 read 0x2509663 thru 0x250c91c (0x32ba bytes) 2024-09-17T22:42:22.779 INFO:teuthology.orchestra.run.smithi115.stdout:4110 read 0x57d0605 thru 0x57d6da4 (0x67a0 bytes) 2024-09-17T22:42:22.779 INFO:teuthology.orchestra.run.smithi115.stdout:4112 write 0x6db6017 thru 0x6dc550a (0xf4f4 bytes) 2024-09-17T22:42:22.782 INFO:teuthology.orchestra.run.smithi115.stdout:4113 write 0x48676eb thru 0x4874e6a (0xd780 bytes) 2024-09-17T22:42:22.784 INFO:teuthology.orchestra.run.smithi115.stdout:4114 read 0xbf41b4f thru 0xbf4c58c (0xaa3e bytes) 2024-09-17T22:42:22.791 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:22.791 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:22.791 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:22.791 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:22.791 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:22.801 INFO:teuthology.orchestra.run.smithi115.stdout:4115 trunc from 0xdf68c5e to 0x35d6cd 2024-09-17T22:42:22.805 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:42:22.829 INFO:teuthology.orchestra.run.smithi115.stdout:4116 write 0x456d38d thru 0x4572cff (0x5973 bytes) 2024-09-17T22:42:22.832 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.832 INFO:teuthology.orchestra.run.smithi115.stdout:4118 punch from 0x40a582d to 0x40aeb25, (0x92f8 bytes) 2024-09-17T22:42:22.834 INFO:teuthology.orchestra.run.smithi115.stdout:4119 trunc from 0x4572d00 to 0x6099652 2024-09-17T22:42:22.840 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.844 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:22.841+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:22.852 INFO:teuthology.orchestra.run.smithi115.stdout:4121 write 0xaef5f8a thru 0xaf00cc0 (0xad37 bytes) 2024-09-17T22:42:22.858 INFO:teuthology.orchestra.run.smithi115.stdout:4122 writesame 0x1d83ae6 thru 0x1d91c3a data_size 0x5(0xe155 bytes) 2024-09-17T22:42:22.866 INFO:teuthology.orchestra.run.smithi115.stdout:4123 read 0x860546 thru 0x8624e1 (0x1f9c bytes) 2024-09-17T22:42:22.866 INFO:teuthology.orchestra.run.smithi115.stdout:4124 punch from 0xbc05b6 to 0xbc6c6f, (0x66b9 bytes) 2024-09-17T22:42:22.866 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.866 INFO:teuthology.orchestra.run.smithi115.stdout:4130 read 0x3a250cd thru 0x3a33e70 (0xeda4 bytes) 2024-09-17T22:42:22.878 INFO:teuthology.orchestra.run.smithi115.stdout:4132 trunc from 0xaf00cc1 to 0xe88990 2024-09-17T22:42:22.908 INFO:teuthology.orchestra.run.smithi115.stdout:4133 write 0xa2cef59 thru 0xa2dcd4a (0xddf2 bytes) 2024-09-17T22:42:22.911 INFO:teuthology.orchestra.run.smithi115.stdout:4134 read 0x958aec1 thru 0x9591db2 (0x6ef2 bytes) 2024-09-17T22:42:22.911 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.911 INFO:teuthology.orchestra.run.smithi115.stdout:4137 punch from 0x3a5ea05 to 0x3a6ae23, (0xc41e bytes) 2024-09-17T22:42:22.911 INFO:teuthology.orchestra.run.smithi115.stdout:4138 read 0x5841f04 thru 0x58470c8 (0x51c5 bytes) 2024-09-17T22:42:22.911 INFO:teuthology.orchestra.run.smithi115.stdout:4139 read 0x672d23d thru 0x6736fa7 (0x9d6b bytes) 2024-09-17T22:42:22.911 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:22.911 INFO:teuthology.orchestra.run.smithi115.stdout:4141 writesame 0x3d2fdd9 thru 0x3d35f98 data_size 0x44(0x61c0 bytes) 2024-09-17T22:42:22.923 INFO:teuthology.orchestra.run.smithi115.stdout:4143 write 0xb899d15 thru 0xb89a82b (0xb17 bytes) 2024-09-17T22:42:22.926 INFO:teuthology.orchestra.run.smithi115.stdout:4144 read 0xb112959 thru 0xb11b6be (0x8d66 bytes) 2024-09-17T22:42:22.926 INFO:teuthology.orchestra.run.smithi115.stdout:4145 read 0x72d434b thru 0x72d73e9 (0x309f bytes) 2024-09-17T22:42:22.936 INFO:teuthology.orchestra.run.smithi115.stdout:4146 trunc from 0xb89a82c to 0x32c4a69 2024-09-17T22:42:22.945 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:22.941+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:22.948 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:42:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.586095602, 2024-09-17T22:42:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.249158453, 2024-09-17T22:42:22.948 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:42:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:42:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-17T22:42:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358680+0000", 2024-09-17T22:42:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.592e-06 2024-09-17T22:42:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358688+0000", 2024-09-17T22:42:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1610000000000003e-06 2024-09-17T22:42:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358697+0000", 2024-09-17T22:42:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2580000000000001e-06 2024-09-17T22:42:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358706+0000", 2024-09-17T22:42:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1640000000000007e-06 2024-09-17T22:42:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358766+0000", 2024-09-17T22:42:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9893999999999999e-05 2024-09-17T22:42:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:42:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358778+0000", 2024-09-17T22:42:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1148999999999999e-05 2024-09-17T22:42:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606347+0000", 2024-09-17T22:42:22.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.247569971 2024-09-17T22:42:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606533+0000", 2024-09-17T22:42:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018599400000000001 2024-09-17T22:42:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:42:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606752+0000", 2024-09-17T22:42:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000218373 2024-09-17T22:42:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:42:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607455+0000", 2024-09-17T22:42:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070311799999999997 2024-09-17T22:42:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607735+0000", 2024-09-17T22:42:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027972300000000001 2024-09-17T22:42:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:42:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607793+0000", 2024-09-17T22:42:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8529999999999997e-05 2024-09-17T22:42:22.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607837+0000", 2024-09-17T22:42:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3526000000000003e-05 2024-09-17T22:42:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186749 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 11960320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.52145428, 2024-09-17T22:42:22.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029226166000000001, 2024-09-17T22:42:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186749 2024-09-17T22:42:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:22.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-09-17T22:42:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423328+0000", 2024-09-17T22:42:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6790000000000005e-06 2024-09-17T22:42:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423334+0000", 2024-09-17T22:42:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0009999999999998e-06 2024-09-17T22:42:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423338+0000", 2024-09-17T22:42:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6419999999999998e-06 2024-09-17T22:42:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452464+0000", 2024-09-17T22:42:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029125389000000002 2024-09-17T22:42:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452501+0000", 2024-09-17T22:42:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7656e-05 2024-09-17T22:42:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452546+0000", 2024-09-17T22:42:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4468999999999997e-05 2024-09-17T22:42:22.959 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186750 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12050432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.52142849, 2024-09-17T22:42:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029307863999999999, 2024-09-17T22:42:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:22.960 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186750 2024-09-17T22:42:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423346+0000", 2024-09-17T22:42:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.84e-07 2024-09-17T22:42:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423348+0000", 2024-09-17T22:42:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7540000000000001e-06 2024-09-17T22:42:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423353+0000", 2024-09-17T22:42:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:42:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423362+0000", 2024-09-17T22:42:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9220000000000003e-06 2024-09-17T22:42:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452571+0000", 2024-09-17T22:42:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029209174000000001 2024-09-17T22:42:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:22.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452612+0000", 2024-09-17T22:42:22.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1363999999999999e-05 2024-09-17T22:42:22.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:22.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452653+0000", 2024-09-17T22:42:22.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0893999999999998e-05 2024-09-17T22:42:22.964 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186751 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12140544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.521406000000001, 2024-09-17T22:42:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029387488, 2024-09-17T22:42:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:22.965 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186751 2024-09-17T22:42:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:22.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1399999999999997e-07 2024-09-17T22:42:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423375+0000", 2024-09-17T22:42:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8889999999999996e-06 2024-09-17T22:42:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423381+0000", 2024-09-17T22:42:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5690000000000004e-06 2024-09-17T22:42:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423386+0000", 2024-09-17T22:42:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9559999999999996e-06 2024-09-17T22:42:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452679+0000", 2024-09-17T22:42:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029292820000000001 2024-09-17T22:42:22.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452718+0000", 2024-09-17T22:42:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8961000000000002e-05 2024-09-17T22:42:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452755+0000", 2024-09-17T22:42:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7879000000000001e-05 2024-09-17T22:42:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186752 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12230656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.521374749, 2024-09-17T22:42:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029463269, 2024-09-17T22:42:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186752 2024-09-17T22:42:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:22.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:22.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423400+0000", 2024-09-17T22:42:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:42:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423401+0000", 2024-09-17T22:42:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3599999999999999e-06 2024-09-17T22:42:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423404+0000", 2024-09-17T22:42:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5999999999999998e-06 2024-09-17T22:42:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423407+0000", 2024-09-17T22:42:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6869999999999999e-06 2024-09-17T22:42:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452782+0000", 2024-09-17T22:42:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029375107000000001 2024-09-17T22:42:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452821+0000", 2024-09-17T22:42:22.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9122999999999998e-05 2024-09-17T22:42:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452862+0000", 2024-09-17T22:42:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1019000000000001e-05 2024-09-17T22:42:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186753 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:22.974 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.521362727, 2024-09-17T22:42:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029553297999999999, 2024-09-17T22:42:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186753 2024-09-17T22:42:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:22.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.98e-07 2024-09-17T22:42:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423412+0000", 2024-09-17T22:42:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8700000000000004e-07 2024-09-17T22:42:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423415+0000", 2024-09-17T22:42:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:42:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423418+0000", 2024-09-17T22:42:22.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.283e-06 2024-09-17T22:42:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452893+0000", 2024-09-17T22:42:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029475881999999998 2024-09-17T22:42:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452928+0000", 2024-09-17T22:42:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4109000000000002e-05 2024-09-17T22:42:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:22.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452964+0000", 2024-09-17T22:42:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6937000000000002e-05 2024-09-17T22:42:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186754 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12410880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.52135359, 2024-09-17T22:42:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029656021000000001, 2024-09-17T22:42:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:22.979 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186754 2024-09-17T22:42:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423421+0000", 2024-09-17T22:42:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9999999999999999e-07 2024-09-17T22:42:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423425+0000", 2024-09-17T22:42:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6519999999999997e-06 2024-09-17T22:42:22.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423428+0000", 2024-09-17T22:42:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1370000000000002e-06 2024-09-17T22:42:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423431+0000", 2024-09-17T22:42:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4090000000000001e-06 2024-09-17T22:42:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452992+0000", 2024-09-17T22:42:22.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029561545000000002 2024-09-17T22:42:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453031+0000", 2024-09-17T22:42:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8834000000000001e-05 2024-09-17T22:42:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453076+0000", 2024-09-17T22:42:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5244000000000003e-05 2024-09-17T22:42:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186755 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12500992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.521340308999999, 2024-09-17T22:42:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029745649999999998, 2024-09-17T22:42:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186755 2024-09-17T22:42:22.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.17e-07 2024-09-17T22:42:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423435+0000", 2024-09-17T22:42:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:42:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423438+0000", 2024-09-17T22:42:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8210000000000002e-06 2024-09-17T22:42:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423443+0000", 2024-09-17T22:42:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1070000000000004e-06 2024-09-17T22:42:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453103+0000", 2024-09-17T22:42:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029660399 2024-09-17T22:42:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453141+0000", 2024-09-17T22:42:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.748e-05 2024-09-17T22:42:22.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453179+0000", 2024-09-17T22:42:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8671000000000003e-05 2024-09-17T22:42:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187439 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8437760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.403903211999999, 2024-09-17T22:42:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029278048000000001, 2024-09-17T22:42:22.988 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187439 2024-09-17T22:42:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6199999999999999e-07 2024-09-17T22:42:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540872+0000", 2024-09-17T22:42:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.09e-07 2024-09-17T22:42:22.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540875+0000", 2024-09-17T22:42:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8990000000000001e-06 2024-09-17T22:42:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540877+0000", 2024-09-17T22:42:22.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1789999999999998e-06 2024-09-17T22:42:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570074+0000", 2024-09-17T22:42:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029197358 2024-09-17T22:42:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570111+0000", 2024-09-17T22:42:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6170000000000001e-05 2024-09-17T22:42:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570149+0000", 2024-09-17T22:42:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8170999999999998e-05 2024-09-17T22:42:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187440 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8527872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.403887742, 2024-09-17T22:42:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029359758999999999, 2024-09-17T22:42:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:22.993 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187440 2024-09-17T22:42:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2700000000000001e-07 2024-09-17T22:42:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540888+0000", 2024-09-17T22:42:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-17T22:42:22.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540893+0000", 2024-09-17T22:42:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9110000000000001e-06 2024-09-17T22:42:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540895+0000", 2024-09-17T22:42:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.712e-06 2024-09-17T22:42:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:22.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570168+0000", 2024-09-17T22:42:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029273114999999999 2024-09-17T22:42:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570208+0000", 2024-09-17T22:42:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9227999999999998e-05 2024-09-17T22:42:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570246+0000", 2024-09-17T22:42:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8253000000000001e-05 2024-09-17T22:42:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:22.997 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187441 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8617984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.403874778, 2024-09-17T22:42:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029442211999999999, 2024-09-17T22:42:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187441 2024-09-17T22:42:22.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:22.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:42:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540900+0000", 2024-09-17T22:42:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7999999999999993e-07 2024-09-17T22:42:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540903+0000", 2024-09-17T22:42:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9129999999999999e-06 2024-09-17T22:42:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540909+0000", 2024-09-17T22:42:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4269999999999999e-06 2024-09-17T22:42:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570267+0000", 2024-09-17T22:42:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029358308 2024-09-17T22:42:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570304+0000", 2024-09-17T22:42:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6860999999999998e-05 2024-09-17T22:42:23.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570341+0000", 2024-09-17T22:42:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7509000000000003e-05 2024-09-17T22:42:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187442 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8708096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.403862071000001, 2024-09-17T22:42:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029522606999999999, 2024-09-17T22:42:23.002 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187442 2024-09-17T22:42:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:23.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.22e-07 2024-09-17T22:42:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540917+0000", 2024-09-17T22:42:23.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999998e-06 2024-09-17T22:42:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540920+0000", 2024-09-17T22:42:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4989999999999999e-06 2024-09-17T22:42:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540923+0000", 2024-09-17T22:42:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3870000000000002e-06 2024-09-17T22:42:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:23.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570361+0000", 2024-09-17T22:42:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029438345000000001 2024-09-17T22:42:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570397+0000", 2024-09-17T22:42:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5429000000000002e-05 2024-09-17T22:42:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570434+0000", 2024-09-17T22:42:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7824999999999998e-05 2024-09-17T22:42:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.006 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187443 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8798208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.403848143999999, 2024-09-17T22:42:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029603640000000001, 2024-09-17T22:42:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:23.007 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187443 2024-09-17T22:42:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.05e-07 2024-09-17T22:42:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540927+0000", 2024-09-17T22:42:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:42:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540930+0000", 2024-09-17T22:42:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9229999999999998e-06 2024-09-17T22:42:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540932+0000", 2024-09-17T22:42:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.294e-06 2024-09-17T22:42:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570454+0000", 2024-09-17T22:42:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029522230999999999 2024-09-17T22:42:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570491+0000", 2024-09-17T22:42:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6835999999999998e-05 2024-09-17T22:42:23.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570529+0000", 2024-09-17T22:42:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8167999999999998e-05 2024-09-17T22:42:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187444 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8888320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.403838670000001, 2024-09-17T22:42:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029687584999999999, 2024-09-17T22:42:23.011 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187444 2024-09-17T22:42:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-09-17T22:42:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540936+0000", 2024-09-17T22:42:23.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3900000000000003e-07 2024-09-17T22:42:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540939+0000", 2024-09-17T22:42:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.994e-06 2024-09-17T22:42:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540942+0000", 2024-09-17T22:42:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1739999999999999e-06 2024-09-17T22:42:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570549+0000", 2024-09-17T22:42:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029607652000000002 2024-09-17T22:42:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570585+0000", 2024-09-17T22:42:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5775000000000002e-05 2024-09-17T22:42:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570623+0000", 2024-09-17T22:42:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7839999999999997e-05 2024-09-17T22:42:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.015 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187445 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8978432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.403829782000001, 2024-09-17T22:42:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029772837999999999, 2024-09-17T22:42:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:23.016 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187445 2024-09-17T22:42:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:42:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540945+0000", 2024-09-17T22:42:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3699999999999999e-07 2024-09-17T22:42:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540948+0000", 2024-09-17T22:42:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.898e-06 2024-09-17T22:42:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540950+0000", 2024-09-17T22:42:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2249999999999999e-06 2024-09-17T22:42:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570643+0000", 2024-09-17T22:42:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029692463999999998 2024-09-17T22:42:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570680+0000", 2024-09-17T22:42:23.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6772e-05 2024-09-17T22:42:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570717+0000", 2024-09-17T22:42:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7438999999999998e-05 2024-09-17T22:42:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187446 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9068544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:23.020 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.403821044000001, 2024-09-17T22:42:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029856321000000002, 2024-09-17T22:42:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187446 2024-09-17T22:42:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:23.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:42:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:23.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540958+0000", 2024-09-17T22:42:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3449999999999996e-06 2024-09-17T22:42:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540962+0000", 2024-09-17T22:42:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3529999999999999e-06 2024-09-17T22:42:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540964+0000", 2024-09-17T22:42:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3810000000000002e-06 2024-09-17T22:42:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570737+0000", 2024-09-17T22:42:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029772824 2024-09-17T22:42:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570770+0000", 2024-09-17T22:42:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2835e-05 2024-09-17T22:42:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570809+0000", 2024-09-17T22:42:23.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9379999999999999e-05 2024-09-17T22:42:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187447 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9158656~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.403807076, 2024-09-17T22:42:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02993262, 2024-09-17T22:42:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:23.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187447 2024-09-17T22:42:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:23.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-09-17T22:42:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540968+0000", 2024-09-17T22:42:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6500000000000008e-07 2024-09-17T22:42:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540971+0000", 2024-09-17T22:42:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9869999999999999e-06 2024-09-17T22:42:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540973+0000", 2024-09-17T22:42:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3120000000000001e-06 2024-09-17T22:42:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570829+0000", 2024-09-17T22:42:23.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029855742000000001 2024-09-17T22:42:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570861+0000", 2024-09-17T22:42:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2152e-05 2024-09-17T22:42:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570899+0000", 2024-09-17T22:42:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8241999999999997e-05 2024-09-17T22:42:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.030 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187448 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9248768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:23.030 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:23.030 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.403798046, 2024-09-17T22:42:23.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030014865000000002, 2024-09-17T22:42:23.030 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:23.030 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:23.030 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:23.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:23.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:23.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187448 2024-09-17T22:42:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:23.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9600000000000001e-07 2024-09-17T22:42:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540977+0000", 2024-09-17T22:42:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.16e-07 2024-09-17T22:42:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:23.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540980+0000", 2024-09-17T22:42:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.892e-06 2024-09-17T22:42:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540982+0000", 2024-09-17T22:42:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3410000000000001e-06 2024-09-17T22:42:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570919+0000", 2024-09-17T22:42:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029936800999999999 2024-09-17T22:42:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570953+0000", 2024-09-17T22:42:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3939000000000002e-05 2024-09-17T22:42:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570991+0000", 2024-09-17T22:42:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7780000000000001e-05 2024-09-17T22:42:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.034 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187449 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9338880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.403788866999999, 2024-09-17T22:42:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030110238000000001, 2024-09-17T22:42:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187449 2024-09-17T22:42:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:23.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1500000000000001e-07 2024-09-17T22:42:23.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540986+0000", 2024-09-17T22:42:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9500000000000001e-07 2024-09-17T22:42:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540989+0000", 2024-09-17T22:42:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7949999999999999e-06 2024-09-17T22:42:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:23.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540994+0000", 2024-09-17T22:42:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0359999999999997e-06 2024-09-17T22:42:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571011+0000", 2024-09-17T22:42:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030016915000000002 2024-09-17T22:42:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571056+0000", 2024-09-17T22:42:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5457999999999997e-05 2024-09-17T22:42:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571095+0000", 2024-09-17T22:42:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8924000000000002e-05 2024-09-17T22:42:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187450 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9428992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.40377722, 2024-09-17T22:42:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030188204999999999, 2024-09-17T22:42:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187450 2024-09-17T22:42:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.16e-07 2024-09-17T22:42:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:23.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540998+0000", 2024-09-17T22:42:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9000000000000005e-07 2024-09-17T22:42:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.541001+0000", 2024-09-17T22:42:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0680000000000001e-06 2024-09-17T22:42:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.541006+0000", 2024-09-17T22:42:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4720000000000003e-06 2024-09-17T22:42:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571110+0000", 2024-09-17T22:42:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030103431 2024-09-17T22:42:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571147+0000", 2024-09-17T22:42:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6748999999999997e-05 2024-09-17T22:42:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571185+0000", 2024-09-17T22:42:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8278999999999997e-05 2024-09-17T22:42:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.043 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:23.044 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.044 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:23.044 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:23.044 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:23.044 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:42:23.045 INFO:teuthology.orchestra.run.smithi115.stdout:4147 read 0xff23be thru 0x1001b30 (0xf773 bytes) 2024-09-17T22:42:23.045 INFO:teuthology.orchestra.run.smithi115.stdout:4149 trunc from 0x32c4a69 to 0x7232116 2024-09-17T22:42:23.046 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:42:23.048 INFO:teuthology.orchestra.run.smithi115.stdout:4150 trunc from 0x7232116 to 0x6233ee6 2024-09-17T22:42:23.048 INFO:teuthology.orchestra.run.smithi115.stdout:4151 read 0x3d9d395 thru 0x3da3b64 (0x67d0 bytes) 2024-09-17T22:42:23.048 INFO:teuthology.orchestra.run.smithi115.stdout:4153 write 0x13a17dc thru 0x13a317f (0x19a4 bytes) 2024-09-17T22:42:23.048 INFO:teuthology.orchestra.run.smithi115.stdout:4154 write 0xa03bbca thru 0xa04b0d8 (0xf50f bytes) 2024-09-17T22:42:23.048 INFO:teuthology.orchestra.run.smithi115.stdout:4156 trunc from 0xa04b0d9 to 0x9b884ab 2024-09-17T22:42:23.048 INFO:teuthology.orchestra.run.smithi115.stdout:4157 write 0x857a96c thru 0x85870e3 (0xc778 bytes) 2024-09-17T22:42:23.048 INFO:teuthology.orchestra.run.smithi115.stdout:4159 read 0x5ba669b thru 0x5ba8a00 (0x2366 bytes) 2024-09-17T22:42:23.048 INFO:teuthology.orchestra.run.smithi115.stdout:4160 write 0xe58990c thru 0xe592736 (0x8e2b bytes) 2024-09-17T22:42:23.048 INFO:teuthology.orchestra.run.smithi115.stdout:4161 read 0x5a53154 thru 0x5a5d334 (0xa1e1 bytes) 2024-09-17T22:42:23.049 INFO:teuthology.orchestra.run.smithi115.stdout:4162 writesame 0x6e9c760 thru 0x6e9e70f data_size 0x18(0x1fb0 bytes) 2024-09-17T22:42:23.049 INFO:teuthology.orchestra.run.smithi115.stdout:4163 writesame 0x771b8b2 thru 0x772406e data_size 0x51(0x87bd bytes) 2024-09-17T22:42:23.049 INFO:teuthology.orchestra.run.smithi115.stdout:4164 read 0xc81d051 thru 0xc82ace3 (0xdc93 bytes) 2024-09-17T22:42:23.049 INFO:teuthology.orchestra.run.smithi115.stdout:4165 writesame 0x4631143 thru 0x463260c data_size 0x3(0x14ca bytes) 2024-09-17T22:42:23.049 INFO:teuthology.orchestra.run.smithi115.stdout:4166 read 0x2cdddca thru 0x2ce5392 (0x75c9 bytes) 2024-09-17T22:42:23.049 INFO:teuthology.orchestra.run.smithi115.stdout:4167 write 0x901572e thru 0x9019f4b (0x481e bytes) 2024-09-17T22:42:23.049 INFO:teuthology.orchestra.run.smithi115.stdout:4168 trunc from 0xe592737 to 0x6bb79b5 2024-09-17T22:42:23.049 INFO:teuthology.orchestra.run.smithi115.stdout:4170 trunc from 0x6bb79b5 to 0x64b237c 2024-09-17T22:42:23.049 INFO:teuthology.orchestra.run.smithi115.stdout:4171 read 0x213b193 thru 0x2140dc5 (0x5c33 bytes) 2024-09-17T22:42:23.050 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:23.048+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:23.051 INFO:teuthology.orchestra.run.smithi115.stdout:4172 trunc from 0x64b237c to 0xe01b575 2024-09-17T22:42:23.057 INFO:teuthology.orchestra.run.smithi115.stdout:4174 trunc from 0xe01b575 to 0xd7e57d5 2024-09-17T22:42:23.063 INFO:teuthology.orchestra.run.smithi115.stdout:4175 read 0x4522205 thru 0x4524fa8 (0x2da4 bytes) 2024-09-17T22:42:23.063 INFO:teuthology.orchestra.run.smithi115.stdout:4176 write 0xd01cf47 thru 0xd0268ee (0x99a8 bytes) 2024-09-17T22:42:23.067 INFO:teuthology.orchestra.run.smithi115.stdout:4177 read 0x8613d7f thru 0x86141a5 (0x427 bytes) 2024-09-17T22:42:23.067 INFO:teuthology.orchestra.run.smithi115.stdout:4178 read 0x524a6f4 thru 0x524e030 (0x393d bytes) 2024-09-17T22:42:23.067 INFO:teuthology.orchestra.run.smithi115.stdout:4179 write 0xaa1d1b1 thru 0xaa22f30 (0x5d80 bytes) 2024-09-17T22:42:23.070 INFO:teuthology.orchestra.run.smithi115.stdout:4180 read 0xe21122 thru 0xe2e766 (0xd645 bytes) 2024-09-17T22:42:23.076 INFO:teuthology.orchestra.run.smithi115.stdout:4181 writesame 0xdc4ffde thru 0xdc5dbaf data_size 0x6(0xdbd2 bytes) 2024-09-17T22:42:23.076 INFO:teuthology.orchestra.run.smithi115.stdout:4182 punch from 0x576db1e to 0x5773d67, (0x6249 bytes) 2024-09-17T22:42:23.076 INFO:teuthology.orchestra.run.smithi115.stdout:4183 write 0x39af39e thru 0x39bd449 (0xe0ac bytes) 2024-09-17T22:42:23.079 INFO:teuthology.orchestra.run.smithi115.stdout:4184 read 0x7dc3559 thru 0x7dd0669 (0xd111 bytes) 2024-09-17T22:42:23.080 INFO:teuthology.orchestra.run.smithi115.stdout:4185 writesame 0xae76c07 thru 0xae783cd data_size 0x1(0x17c7 bytes) 2024-09-17T22:42:23.088 INFO:teuthology.orchestra.run.smithi115.stdout:4186 trunc from 0xdc5dbb0 to 0xa7c833c 2024-09-17T22:42:23.101 INFO:teuthology.orchestra.run.smithi115.stdout:4187 read 0x686bc72 thru 0x6878060 (0xc3ef bytes) 2024-09-17T22:42:23.101 INFO:teuthology.orchestra.run.smithi115.stdout:4189 punch from 0x30354a4 to 0x303db15, (0x8671 bytes) 2024-09-17T22:42:23.101 INFO:teuthology.orchestra.run.smithi115.stdout:4190 read 0x34da9fc thru 0x34df714 (0x4d19 bytes) 2024-09-17T22:42:23.111 INFO:teuthology.orchestra.run.smithi115.stdout:4191 writesame 0xcf1f113 thru 0xcf2b09f data_size 0x1(0xbf8d bytes) 2024-09-17T22:42:23.111 INFO:teuthology.orchestra.run.smithi115.stdout:4194 punch from 0x5fe8bc7 to 0x5fea008, (0x1441 bytes) 2024-09-17T22:42:23.111 INFO:teuthology.orchestra.run.smithi115.stdout:4197 punch from 0x35a7923 to 0x35ab70d, (0x3dea bytes) 2024-09-17T22:42:23.111 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:23.111 INFO:teuthology.orchestra.run.smithi115.stdout:4199 write 0x1ba5240 thru 0x1ba7b54 (0x2915 bytes) 2024-09-17T22:42:23.114 INFO:teuthology.orchestra.run.smithi115.stdout:4200 write 0x4cc1438 thru 0x4cce074 (0xcc3d bytes) 2024-09-17T22:42:23.133 INFO:teuthology.orchestra.run.smithi115.stdout:4201 trunc from 0xcf2b0a0 to 0x1b8f049 2024-09-17T22:42:23.145 INFO:teuthology.orchestra.run.smithi115.stdout:4203 read 0x3f1c07 thru 0x3f907e (0x7478 bytes) 2024-09-17T22:42:23.151 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:23.147+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:23.155 INFO:teuthology.orchestra.run.smithi115.stdout:4204 write 0x3c4b0c8 thru 0x3c5204c (0x6f85 bytes) 2024-09-17T22:42:23.158 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:23.158 INFO:teuthology.orchestra.run.smithi115.stdout:4207 punch from 0xca15a5 to 0xca710a, (0x5b65 bytes) 2024-09-17T22:42:23.158 INFO:teuthology.orchestra.run.smithi115.stdout:4209 write 0x28996ff thru 0x289d021 (0x3923 bytes) 2024-09-17T22:42:23.166 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:23.172 INFO:teuthology.orchestra.run.smithi115.stdout:4210 write 0x729ffb3 thru 0x72a76ff (0x774d bytes) 2024-09-17T22:42:23.176 INFO:teuthology.orchestra.run.smithi115.stdout:4214 writesame 0x4783dd1 thru 0x4791f85 data_size 0x1(0xe1b5 bytes) 2024-09-17T22:42:23.181 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:23.181 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:23.181 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:23.182 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:23.184 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:23.184 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:23.191 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:42:23.202 INFO:teuthology.orchestra.run.smithi115.stdout:4216 trunc from 0x72a7700 to 0x27ae676 2024-09-17T22:42:23.202 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:23.203 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:23.223 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:23.224 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:23.227 INFO:teuthology.orchestra.run.smithi115.stdout:4217 write 0x9c72d03 thru 0x9c73ce7 (0xfe5 bytes) 2024-09-17T22:42:23.231 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:23.231 INFO:teuthology.orchestra.run.smithi115.stdout:4221 write 0x496d3b0 thru 0x497bfb3 (0xec04 bytes) 2024-09-17T22:42:23.236 INFO:teuthology.orchestra.run.smithi115.stdout:4222 trunc from 0x9c73ce8 to 0xa80451d 2024-09-17T22:42:23.242 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:23.242 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:23.251 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:23.249+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:23.253 INFO:teuthology.orchestra.run.smithi115.stdout:4223 trunc from 0xa80451d to 0x2873474 2024-09-17T22:42:23.254 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:42:23.254 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:42:23.254 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-17T22:42:23.254 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T22:42:23.254 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T22:42:23.255 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-17T22:42:23.267 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:23.267 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:23.281 INFO:teuthology.orchestra.run.smithi115.stdout:4224 write 0x8d2cba1 thru 0x8d3713c (0xa59c bytes) 2024-09-17T22:42:23.284 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:23.285 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:23.286 INFO:teuthology.orchestra.run.smithi115.stdout:4225 trunc from 0x8d3713d to 0x9a6cc7e 2024-09-17T22:42:23.292 INFO:teuthology.orchestra.run.smithi115.stdout:4226 writesame 0x80911ec thru 0x80a0a66 data_size 0x1(0xf87b bytes) 2024-09-17T22:42:23.307 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:23.307 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:23.311 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:23.312 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:23.322 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:23.322 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:23.327 INFO:teuthology.orchestra.run.smithi115.stdout:4227 write 0x8459331 thru 0x845a1c4 (0xe94 bytes) 2024-09-17T22:42:23.331 INFO:teuthology.orchestra.run.smithi115.stdout:4228 punch from 0x4be055d to 0x4be3423, (0x2ec6 bytes) 2024-09-17T22:42:23.331 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:23.331 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:23.331 INFO:teuthology.orchestra.run.smithi115.stdout:4232 read 0x10d69d2 thru 0x10e0dc4 (0xa3f3 bytes) 2024-09-17T22:42:23.332 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:42:23.332 INFO:teuthology.orchestra.run.smithi115.stdout:4233 writesame 0x57020e2 thru 0x5711804 data_size 0x1(0xf723 bytes) 2024-09-17T22:42:23.362 INFO:teuthology.orchestra.run.smithi115.stdout:4234 writesame 0x34cef9e thru 0x34dc221 data_size 0x36(0xd284 bytes) 2024-09-17T22:42:23.362 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:23.359+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:23.364 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:23.364 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "" 2024-09-17T22:42:23.364 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:23.365 INFO:teuthology.orchestra.run.smithi115.stdout:4235 write 0x9460cc6 thru 0x9468355 (0x7690 bytes) 2024-09-17T22:42:23.368 INFO:teuthology.orchestra.run.smithi115.stdout:4236 read 0x3c2a4a6 thru 0x3c2de2d (0x3988 bytes) 2024-09-17T22:42:23.368 INFO:teuthology.orchestra.run.smithi115.stdout:4239 read 0x32f3cee thru 0x3301529 (0xd83c bytes) 2024-09-17T22:42:23.375 INFO:teuthology.orchestra.run.smithi115.stdout:4240 write 0xb1aaa02 thru 0xb1b4178 (0x9777 bytes) 2024-09-17T22:42:23.379 INFO:teuthology.orchestra.run.smithi115.stdout:4241 punch from 0x4cf2d36 to 0x4cfb491, (0x875b bytes) 2024-09-17T22:42:23.379 INFO:teuthology.orchestra.run.smithi115.stdout:4243 writesame 0x60c1c03 thru 0x60c5033 data_size 0x1(0x3431 bytes) 2024-09-17T22:42:23.387 INFO:teuthology.orchestra.run.smithi115.stdout:4244 write 0x53e2053 thru 0x53e761d (0x55cb bytes) 2024-09-17T22:42:23.399 INFO:teuthology.orchestra.run.smithi115.stdout:4245 trunc from 0xb1b4179 to 0x384f4b3 2024-09-17T22:42:23.411 INFO:teuthology.orchestra.run.smithi115.stdout:4246 write 0x2f46609 thru 0x2f4cc23 (0x661b bytes) 2024-09-17T22:42:23.412 INFO:teuthology.orchestra.run.smithi115.stdout:4247 read 0x1ee47fe thru 0x1ee5d4f (0x1552 bytes) 2024-09-17T22:42:23.429 INFO:teuthology.orchestra.run.smithi115.stdout:4248 write 0xbf11f7a thru 0xbf16708 (0x478f bytes) 2024-09-17T22:42:23.432 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:23.432 INFO:teuthology.orchestra.run.smithi115.stdout:4250 write 0xadc6fb8 thru 0xadd6aac (0xfaf5 bytes) 2024-09-17T22:42:23.436 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:23.443 INFO:teuthology.orchestra.run.smithi115.stdout:4252 write 0xc037b92 thru 0xc045e91 (0xe300 bytes) 2024-09-17T22:42:23.450 INFO:teuthology.orchestra.run.smithi115.stdout:4254 read 0x2769480 thru 0x277603f (0xcbc0 bytes) 2024-09-17T22:42:23.451 INFO:teuthology.orchestra.run.smithi115.stdout:4255 read 0x202c56a thru 0x203ba80 (0xf517 bytes) 2024-09-17T22:42:23.453 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:23.450+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:23.458 INFO:teuthology.orchestra.run.smithi115.stdout:4256 trunc from 0xc045e92 to 0x5afd1d4 2024-09-17T22:42:23.477 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:23.477 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:23.482 INFO:teuthology.orchestra.run.smithi115.stdout:4257 write 0x7fbb389 thru 0x7fc9298 (0xdf10 bytes) 2024-09-17T22:42:23.487 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:42:23.497 INFO:teuthology.orchestra.run.smithi115.stdout:4258 write 0xc35001a thru 0xc353c25 (0x3c0c bytes) 2024-09-17T22:42:23.508 INFO:teuthology.orchestra.run.smithi115.stdout:4260 writesame 0xebe8ee8 thru 0xebed72c data_size 0x7(0x4845 bytes) 2024-09-17T22:42:23.508 INFO:teuthology.orchestra.run.smithi115.stdout:4261 read 0x16e67ff thru 0x16e9fe8 (0x37ea bytes) 2024-09-17T22:42:23.508 INFO:teuthology.orchestra.run.smithi115.stdout:4263 writesame 0x2183119 thru 0x21889b1 data_size 0x1(0x5899 bytes) 2024-09-17T22:42:23.520 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:23.520 INFO:teuthology.orchestra.run.smithi115.stdout:4267 read 0x7b3e6ae thru 0x7b43c5a (0x55ad bytes) 2024-09-17T22:42:23.520 INFO:teuthology.orchestra.run.smithi115.stdout:4269 write 0xe28fa6a thru 0xe29551c (0x5ab3 bytes) 2024-09-17T22:42:23.523 INFO:teuthology.orchestra.run.smithi115.stdout:4270 write 0xb8dc319 thru 0xb8e22b6 (0x5f9e bytes) 2024-09-17T22:42:23.527 INFO:teuthology.orchestra.run.smithi115.stdout:4271 write 0xa2579d6 thru 0xa26457c (0xcba7 bytes) 2024-09-17T22:42:23.531 INFO:teuthology.orchestra.run.smithi115.stdout:4276 punch from 0x96c5c39 to 0x96d0f77, (0xb33e bytes) 2024-09-17T22:42:23.531 INFO:teuthology.orchestra.run.smithi115.stdout:4278 write 0x9d1915c thru 0x9d1d91c (0x47c1 bytes) 2024-09-17T22:42:23.534 INFO:teuthology.orchestra.run.smithi115.stdout:4279 punch from 0x26d12cd to 0x26d56ff, (0x4432 bytes) 2024-09-17T22:42:23.534 INFO:teuthology.orchestra.run.smithi115.stdout:4280 write 0x7d0b9ce thru 0x7d195b3 (0xdbe6 bytes) 2024-09-17T22:42:23.540 INFO:teuthology.orchestra.run.smithi115.stdout:4281 trunc from 0xebed72d to 0xb131e90 2024-09-17T22:42:23.554 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:23.552+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:23.561 INFO:teuthology.orchestra.run.smithi115.stdout:4282 writesame 0xd4200e2 thru 0xd42832e data_size 0x1(0x824d bytes) 2024-09-17T22:42:23.561 INFO:teuthology.orchestra.run.smithi115.stdout:4283 write 0x5930663 thru 0x59320ff (0x1a9d bytes) 2024-09-17T22:42:23.563 INFO:teuthology.orchestra.run.smithi115.stdout:4284 writesame 0x738943e thru 0x738c6c2 data_size 0x1(0x3285 bytes) 2024-09-17T22:42:23.570 INFO:teuthology.orchestra.run.smithi115.stdout:4285 write 0x2b724f8 thru 0x2b74428 (0x1f31 bytes) 2024-09-17T22:42:23.574 INFO:teuthology.orchestra.run.smithi115.stdout:4286 trunc from 0xd42832f to 0xb7ad59f 2024-09-17T22:42:23.589 INFO:teuthology.orchestra.run.smithi115.stdout:4287 write 0xdcab404 thru 0xdcab8ca (0x4c7 bytes) 2024-09-17T22:42:23.591 INFO:teuthology.orchestra.run.smithi115.stdout:4288 write 0xd4d1762 thru 0xd4d75da (0x5e79 bytes) 2024-09-17T22:42:23.594 INFO:teuthology.orchestra.run.smithi115.stdout:4290 read 0x8da2d5a thru 0x8dadee8 (0xb18f bytes) 2024-09-17T22:42:23.594 INFO:teuthology.orchestra.run.smithi115.stdout:4291 write 0x2c50dd4 thru 0x2c595af (0x87dc bytes) 2024-09-17T22:42:23.598 INFO:teuthology.orchestra.run.smithi115.stdout:4292 write 0x1c18fac thru 0x1c1c3ba (0x340f bytes) 2024-09-17T22:42:23.599 INFO:teuthology.orchestra.run.smithi115.stdout:4294 write 0xaef264f thru 0xaef6b5d (0x450f bytes) 2024-09-17T22:42:23.602 INFO:teuthology.orchestra.run.smithi115.stdout:4296 write 0x7086376 thru 0x709357d (0xd208 bytes) 2024-09-17T22:42:23.606 INFO:teuthology.orchestra.run.smithi115.stdout:4297 write 0x7901554 thru 0x790d9a4 (0xc451 bytes) 2024-09-17T22:42:23.607 INFO:teuthology.orchestra.run.smithi115.stdout:4300 writesame 0x3f50a94 thru 0x3f58039 data_size 0xb(0x75a6 bytes) 2024-09-17T22:42:23.608 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:23.608 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:23.613 INFO:teuthology.orchestra.run.smithi115.stdout:4301 trunc from 0xdcab8cb to 0xb8582d8 2024-09-17T22:42:23.619 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:42:23.627 INFO:teuthology.orchestra.run.smithi115.stdout:4302 write 0x1ed7a8d thru 0x1edeb93 (0x7107 bytes) 2024-09-17T22:42:23.629 INFO:teuthology.orchestra.run.smithi115.stdout:4303 write 0x9362c1c thru 0x93685af (0x5994 bytes) 2024-09-17T22:42:23.631 INFO:teuthology.orchestra.run.smithi115.stdout:4305 writesame 0x627bcd thru 0x62ad8f data_size 0x1(0x31c3 bytes) 2024-09-17T22:42:23.643 INFO:teuthology.orchestra.run.smithi115.stdout:4309 write 0xbc9b19a thru 0xbca612f (0xaf96 bytes) 2024-09-17T22:42:23.646 INFO:teuthology.orchestra.run.smithi115.stdout:4311 read 0x7043c57 thru 0x7049b2a (0x5ed4 bytes) 2024-09-17T22:42:23.647 INFO:teuthology.orchestra.run.smithi115.stdout:4312 read 0x6e3479e thru 0x6e37b82 (0x33e5 bytes) 2024-09-17T22:42:23.647 INFO:teuthology.orchestra.run.smithi115.stdout:4313 read 0xa28898e thru 0xa292ff6 (0xa669 bytes) 2024-09-17T22:42:23.647 INFO:teuthology.orchestra.run.smithi115.stdout:4315 punch from 0x664afab to 0x6654363, (0x93b8 bytes) 2024-09-17T22:42:23.647 INFO:teuthology.orchestra.run.smithi115.stdout:4316 read 0x2ab5542 thru 0x2ab761e (0x20dd bytes) 2024-09-17T22:42:23.647 INFO:teuthology.orchestra.run.smithi115.stdout:4317 writesame 0x221215e thru 0x221949c data_size 0x1(0x733f bytes) 2024-09-17T22:42:23.655 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:23.653+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:23.662 INFO:teuthology.orchestra.run.smithi115.stdout:4318 read 0xb3ab9a8 thru 0xb3abb30 (0x189 bytes) 2024-09-17T22:42:23.663 INFO:teuthology.orchestra.run.smithi115.stdout:4319 punch from 0x7bd7344 to 0x7bd8042, (0xcfe bytes) 2024-09-17T22:42:23.663 INFO:teuthology.orchestra.run.smithi115.stdout:4320 read 0x46f2f18 thru 0x46f91bc (0x62a5 bytes) 2024-09-17T22:42:23.664 INFO:teuthology.orchestra.run.smithi115.stdout:4321 trunc from 0xbca6130 to 0xce8f245 2024-09-17T22:42:23.669 INFO:teuthology.orchestra.run.smithi115.stdout:4322 writesame 0xb86bc9 thru 0xb89322 data_size 0x45(0x275a bytes) 2024-09-17T22:42:23.671 INFO:teuthology.orchestra.run.smithi115.stdout:4323 punch from 0x859a6a6 to 0x85a993b, (0xf295 bytes) 2024-09-17T22:42:23.671 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:23.671 INFO:teuthology.orchestra.run.smithi115.stdout:4328 read 0x13a3d9d thru 0x13aa5b0 (0x6814 bytes) 2024-09-17T22:42:23.671 INFO:teuthology.orchestra.run.smithi115.stdout:4329 write 0xb167626 thru 0xb16f780 (0x815b bytes) 2024-09-17T22:42:23.674 INFO:teuthology.orchestra.run.smithi115.stdout:4330 read 0x642347c thru 0x642b953 (0x84d8 bytes) 2024-09-17T22:42:23.683 INFO:teuthology.orchestra.run.smithi115.stdout:4331 write 0xe90e5a3 thru 0xe91d96a (0xf3c8 bytes) 2024-09-17T22:42:23.686 INFO:teuthology.orchestra.run.smithi115.stdout:4336 write 0xb2114d2 thru 0xb215406 (0x3f35 bytes) 2024-09-17T22:42:23.690 INFO:teuthology.orchestra.run.smithi115.stdout:4337 read 0xa391744 thru 0xa39f88e (0xe14b bytes) 2024-09-17T22:42:23.690 INFO:teuthology.orchestra.run.smithi115.stdout:4340 read 0x368b111 thru 0x3698af8 (0xd9e8 bytes) 2024-09-17T22:42:23.690 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:23.691 INFO:teuthology.orchestra.run.smithi115.stdout:4342 writesame 0xc7567a5 thru 0xc758e0b data_size 0x3(0x2667 bytes) 2024-09-17T22:42:23.694 INFO:teuthology.orchestra.run.smithi115.stdout:4343 read 0x60597c thru 0x60a64c (0x4cd1 bytes) 2024-09-17T22:42:23.695 INFO:teuthology.orchestra.run.smithi115.stdout:4344 read 0x4f8a03b thru 0x4f97b29 (0xdaef bytes) 2024-09-17T22:42:23.695 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:23.695 INFO:teuthology.orchestra.run.smithi115.stdout:4346 read 0xc98ca48 thru 0xc98cebc (0x475 bytes) 2024-09-17T22:42:23.695 INFO:teuthology.orchestra.run.smithi115.stdout:4350 read 0xbe302bf thru 0xbe3730c (0x704e bytes) 2024-09-17T22:42:23.695 INFO:teuthology.orchestra.run.smithi115.stdout:4351 writesame 0x799b10b thru 0x79aaf5c data_size 0x9(0xfe52 bytes) 2024-09-17T22:42:23.700 INFO:teuthology.orchestra.run.smithi115.stdout:4352 read 0xd7d5640 thru 0xd7dd259 (0x7c1a bytes) 2024-09-17T22:42:23.701 INFO:teuthology.orchestra.run.smithi115.stdout:4354 read 0x41f6c59 thru 0x4202028 (0xb3d0 bytes) 2024-09-17T22:42:23.701 INFO:teuthology.orchestra.run.smithi115.stdout:4356 read 0x6c47260 thru 0x6c4c7f5 (0x5596 bytes) 2024-09-17T22:42:23.701 INFO:teuthology.orchestra.run.smithi115.stdout:4357 writesame 0x427d3e7 thru 0x4286461 data_size 0x1(0x907b bytes) 2024-09-17T22:42:23.716 INFO:teuthology.orchestra.run.smithi115.stdout:4358 writesame 0x140edde thru 0x141989c data_size 0x1(0xaabf bytes) 2024-09-17T22:42:23.731 INFO:teuthology.orchestra.run.smithi115.stdout:4359 write 0x2c67d7b thru 0x2c77a7b (0xfd01 bytes) 2024-09-17T22:42:23.737 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:23.737 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:23.738 INFO:teuthology.orchestra.run.smithi115.stdout:4361 trunc from 0xe91d96b to 0xa09ec9c 2024-09-17T22:42:23.748 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:42:23.756 INFO:teuthology.orchestra.run.smithi115.stdout:4362 trunc from 0xa09ec9c to 0xb38dd58 2024-09-17T22:42:23.757 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:23.753+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:23.767 INFO:teuthology.orchestra.run.smithi115.stdout:4363 trunc from 0xb38dd58 to 0x71b1f45 2024-09-17T22:42:23.779 INFO:teuthology.orchestra.run.smithi115.stdout:4365 punch from 0x3ce2238 to 0x3cea522, (0x82ea bytes) 2024-09-17T22:42:23.779 INFO:teuthology.orchestra.run.smithi115.stdout:4366 read 0x5c24458 thru 0x5c27051 (0x2bfa bytes) 2024-09-17T22:42:23.779 INFO:teuthology.orchestra.run.smithi115.stdout:4367 clone 33 order 19 su 16384 sc 7 2024-09-17T22:42:23.857 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:23.854+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:23.860 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:23.860 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:23.868 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:42:23.957 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:23.955+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:23.971 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:23.971 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:23.980 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:42:24.058 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:24.055+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:24.082 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:24.082 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:24.091 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:42:24.175 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:24.172+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:24.234 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:24.234 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:24.242 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:42:24.260 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:24.257+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:24.263 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:42:24.329 INFO:teuthology.orchestra.run.smithi115.stdout:flattening image image_client.0-clone32 2024-09-17T22:42:24.360 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:24.358+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:24.388 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:24.389 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:24.397 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:42:24.439 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-17T22:42:24.443 INFO:teuthology.orchestra.run.smithi115.stdout:4370 punch from 0x189814c to 0x1898387, (0x23b bytes) 2024-09-17T22:42:24.447 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:24.447 INFO:teuthology.orchestra.run.smithi115.stdout:4372 writesame 0x2ba997f thru 0x2bb8864 data_size 0x3(0xeee6 bytes) 2024-09-17T22:42:24.462 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:24.458+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:24.469 INFO:teuthology.orchestra.run.smithi115.stdout:4373 writesame 0x93a1f46 thru 0x93a9a56 data_size 0x1(0x7b11 bytes) 2024-09-17T22:42:24.477 INFO:teuthology.orchestra.run.smithi115.stdout:4374 writesame 0xb09403c thru 0xb09efae data_size 0xd(0xaf73 bytes) 2024-09-17T22:42:24.477 INFO:teuthology.orchestra.run.smithi115.stdout:4376 punch from 0x9bc598 to 0x9c75cf, (0xb037 bytes) 2024-09-17T22:42:24.478 INFO:teuthology.orchestra.run.smithi115.stdout:4377 read 0x440358f thru 0x4409132 (0x5ba4 bytes) 2024-09-17T22:42:24.479 INFO:teuthology.orchestra.run.smithi115.stdout:4378 trunc from 0xb09efaf to 0xce57cb0 2024-09-17T22:42:24.485 INFO:teuthology.orchestra.run.smithi115.stdout:4381 write 0xa4c297a thru 0xa4c9b14 (0x719b bytes) 2024-09-17T22:42:24.492 INFO:teuthology.orchestra.run.smithi115.stdout:4382 writesame 0x79ce464 thru 0x79d8551 data_size 0x1(0xa0ee bytes) 2024-09-17T22:42:24.510 INFO:teuthology.orchestra.run.smithi115.stdout:4384 writesame 0x527c374 thru 0x52884d7 data_size 0x2(0xc164 bytes) 2024-09-17T22:42:24.519 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:24.519 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:24.525 INFO:teuthology.orchestra.run.smithi115.stdout:4385 punch from 0xc890ba4 to 0xc89c963, (0xbdbf bytes) 2024-09-17T22:42:24.525 INFO:teuthology.orchestra.run.smithi115.stdout:4387 write 0xcae0299 thru 0xcae0a93 (0x7fb bytes) 2024-09-17T22:42:24.530 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:42:24.536 INFO:teuthology.orchestra.run.smithi115.stdout:4388 writesame 0xd94ab4a thru 0xd951b5c data_size 0xd(0x7013 bytes) 2024-09-17T22:42:24.536 INFO:teuthology.orchestra.run.smithi115.stdout:4389 writesame 0x3d3ba84 thru 0x3d40434 data_size 0x37(0x49b1 bytes) 2024-09-17T22:42:24.541 INFO:teuthology.orchestra.run.smithi115.stdout:4390 write 0x585342 thru 0x58a403 (0x50c2 bytes) 2024-09-17T22:42:24.550 INFO:teuthology.orchestra.run.smithi115.stdout:4391 write 0x29a0e08 thru 0x29a29f6 (0x1bef bytes) 2024-09-17T22:42:24.557 INFO:teuthology.orchestra.run.smithi115.stdout:4392 trunc from 0xd951b5d to 0xb4ea7a7 2024-09-17T22:42:24.571 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:24.568+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:24.580 INFO:teuthology.orchestra.run.smithi115.stdout:4393 write 0xd66920d thru 0xd676c95 (0xda89 bytes) 2024-09-17T22:42:24.589 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:24.589 INFO:teuthology.orchestra.run.smithi115.stdout:4397 write 0x946ca26 thru 0x947bc03 (0xf1de bytes) 2024-09-17T22:42:24.598 INFO:teuthology.orchestra.run.smithi115.stdout:4398 read 0xd6232e0 thru 0xd625ad7 (0x27f8 bytes) 2024-09-17T22:42:24.598 INFO:teuthology.orchestra.run.smithi115.stdout:4399 punch from 0x1a73f7d to 0x1a78c2d, (0x4cb0 bytes) 2024-09-17T22:42:24.605 INFO:teuthology.orchestra.run.smithi115.stdout:4401 write 0xe0fc7e4 thru 0xe0fe400 (0x1c1d bytes) 2024-09-17T22:42:24.607 INFO:teuthology.orchestra.run.smithi115.stdout:4402 punch from 0xad587e to 0xae3c30, (0xe3b2 bytes) 2024-09-17T22:42:24.607 INFO:teuthology.orchestra.run.smithi115.stdout:4403 writesame 0x9d68c73 thru 0x9d6f573 data_size 0x1(0x6901 bytes) 2024-09-17T22:42:24.624 INFO:teuthology.orchestra.run.smithi115.stdout:4404 write 0xe7c63a2 thru 0xe7cb124 (0x4d83 bytes) 2024-09-17T22:42:24.628 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:24.628 INFO:teuthology.orchestra.run.smithi115.stdout:4408 write 0x2e16028 thru 0x2e222da (0xc2b3 bytes) 2024-09-17T22:42:24.634 INFO:teuthology.orchestra.run.smithi115.stdout:4409 writesame 0x422ce7 thru 0x42422e data_size 0x1(0x1548 bytes) 2024-09-17T22:42:24.642 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:24.642 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:24.642 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:24.642 INFO:teuthology.orchestra.run.smithi115.stdout:4415 punch from 0xc68c13b to 0xc695861, (0x9726 bytes) 2024-09-17T22:42:24.642 INFO:teuthology.orchestra.run.smithi115.stdout:4416 read 0x546422c thru 0x54666c4 (0x2499 bytes) 2024-09-17T22:42:24.642 INFO:teuthology.orchestra.run.smithi115.stdout:4417 read 0xc96c52a thru 0xc97b5e3 (0xf0ba bytes) 2024-09-17T22:42:24.642 INFO:teuthology.orchestra.run.smithi115.stdout:4418 writesame 0x281b0e2 thru 0x282a069 data_size 0xa8(0xef88 bytes) 2024-09-17T22:42:24.654 INFO:teuthology.orchestra.run.smithi115.stdout:4419 write 0x5d2006a thru 0x5d25589 (0x5520 bytes) 2024-09-17T22:42:24.662 INFO:teuthology.orchestra.run.smithi115.stdout:4420 read 0xc41a997 thru 0xc42389b (0x8f05 bytes) 2024-09-17T22:42:24.663 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:24.659+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:24.667 INFO:teuthology.orchestra.run.smithi115.stdout:4421 trunc from 0xe7cb125 to 0x9b7d966 2024-09-17T22:42:24.681 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:24.681 INFO:teuthology.orchestra.run.smithi115.stdout:4424 write 0x3a01302 thru 0x3a0cb00 (0xb7ff bytes) 2024-09-17T22:42:24.690 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:24.690 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:24.693 INFO:teuthology.orchestra.run.smithi115.stdout:4426 punch from 0x4f88262 to 0x4f90973, (0x8711 bytes) 2024-09-17T22:42:24.693 INFO:teuthology.orchestra.run.smithi115.stdout:4427 punch from 0x114b16f to 0x114cbf5, (0x1a86 bytes) 2024-09-17T22:42:24.693 INFO:teuthology.orchestra.run.smithi115.stdout:4428 read 0xc93dea thru 0xca1b85 (0xdd9c bytes) 2024-09-17T22:42:24.693 INFO:teuthology.orchestra.run.smithi115.stdout:4430 writesame 0x8b07521 thru 0x8b125aa data_size 0x3b(0xb08a bytes) 2024-09-17T22:42:24.700 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:42:24.700 INFO:teuthology.orchestra.run.smithi115.stdout:4431 write 0x46de156 thru 0x46de866 (0x711 bytes) 2024-09-17T22:42:24.702 INFO:teuthology.orchestra.run.smithi115.stdout:4432 write 0x44c6356 thru 0x44ccd06 (0x69b1 bytes) 2024-09-17T22:42:24.712 INFO:teuthology.orchestra.run.smithi115.stdout:4433 writesame 0x13cefe1 thru 0x13d6012 data_size 0x3(0x7032 bytes) 2024-09-17T22:42:24.734 INFO:teuthology.orchestra.run.smithi115.stdout:4435 write 0xd8f0c12 thru 0xd8f90f1 (0x84e0 bytes) 2024-09-17T22:42:24.740 INFO:teuthology.orchestra.run.smithi115.stdout:4436 read 0x437b85f thru 0x43826e8 (0x6e8a bytes) 2024-09-17T22:42:24.740 INFO:teuthology.orchestra.run.smithi115.stdout:4437 writesame 0x6ca1ba8 thru 0x6cab355 data_size 0x37(0x97ae bytes) 2024-09-17T22:42:24.758 INFO:teuthology.orchestra.run.smithi115.stdout:4439 trunc from 0xd8f90f2 to 0x307adeb 2024-09-17T22:42:24.763 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:24.761+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:24.797 INFO:teuthology.orchestra.run.smithi115.stdout:4440 writesame 0x9c5e035 thru 0x9c62540 data_size 0x6(0x450c bytes) 2024-09-17T22:42:24.805 INFO:teuthology.orchestra.run.smithi115.stdout:4441 write 0xbc5c430 thru 0xbc5d7ab (0x137c bytes) 2024-09-17T22:42:24.808 INFO:teuthology.orchestra.run.smithi115.stdout:4442 writesame 0x21f178c thru 0x21fadff data_size 0x2(0x9674 bytes) 2024-09-17T22:42:24.820 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:24.820 INFO:teuthology.orchestra.run.smithi115.stdout:4445 read 0x5b677f8 thru 0x5b7683d (0xf046 bytes) 2024-09-17T22:42:24.821 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:24.821 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:24.821 INFO:teuthology.orchestra.run.smithi115.stdout:4449 writesame 0x95f389c thru 0x95fbd23 data_size 0x4(0x8488 bytes) 2024-09-17T22:42:24.836 INFO:teuthology.orchestra.run.smithi115.stdout:4450 trunc from 0xbc5d7ac to 0x6a17ac8 2024-09-17T22:42:24.847 INFO:teuthology.orchestra.run.smithi115.stdout:4451 write 0x12f57ac thru 0x13018f9 (0xc14e bytes) 2024-09-17T22:42:24.856 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:24.856 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:24.857 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:24.857 INFO:teuthology.orchestra.run.smithi115.stdout:4453 punch from 0x819abe to 0x826be3, (0xd125 bytes) 2024-09-17T22:42:24.857 INFO:teuthology.orchestra.run.smithi115.stdout:4455 write 0x455e1e0 thru 0x455ff24 (0x1d45 bytes) 2024-09-17T22:42:24.860 INFO:teuthology.orchestra.run.smithi115.stdout:4456 read 0x5b1d7b5 thru 0x5b2a577 (0xcdc3 bytes) 2024-09-17T22:42:24.860 INFO:teuthology.orchestra.run.smithi115.stdout:4457 read 0x511a90a thru 0x5121baa (0x72a1 bytes) 2024-09-17T22:42:24.860 INFO:teuthology.orchestra.run.smithi115.stdout:4458 punch from 0x98a391 to 0x98f750, (0x53bf bytes) 2024-09-17T22:42:24.860 INFO:teuthology.orchestra.run.smithi115.stdout:4459 write 0x38d2ba7 thru 0x38d6995 (0x3def bytes) 2024-09-17T22:42:24.864 INFO:teuthology.orchestra.run.smithi115.stdout:4460 read 0x23716d9 thru 0x2375e53 (0x477b bytes) 2024-09-17T22:42:24.864 INFO:teuthology.orchestra.run.smithi115.stdout:4461 read 0x52cb3b5 thru 0x52d6dc0 (0xba0c bytes) 2024-09-17T22:42:24.864 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:24.861+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:24.867 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:42:24.878 INFO:teuthology.orchestra.run.smithi115.stdout:4462 write 0xd7f9f37 thru 0xd8036c0 (0x978a bytes) 2024-09-17T22:42:24.882 INFO:teuthology.orchestra.run.smithi115.stdout:4463 read 0x66a7523 thru 0x66ad8df (0x63bd bytes) 2024-09-17T22:42:24.882 INFO:teuthology.orchestra.run.smithi115.stdout:4464 writesame 0x8e7aee7 thru 0x8e8aa4b data_size 0x1(0xfb65 bytes) 2024-09-17T22:42:24.899 INFO:teuthology.orchestra.run.smithi115.stdout:4466 read 0xb9ad9d6 thru 0xb9b4f37 (0x7562 bytes) 2024-09-17T22:42:24.899 INFO:teuthology.orchestra.run.smithi115.stdout:4467 read 0x44e274a thru 0x44e776a (0x5021 bytes) 2024-09-17T22:42:24.899 INFO:teuthology.orchestra.run.smithi115.stdout:4468 read 0xc9a0c2a thru 0xc9a7de8 (0x71bf bytes) 2024-09-17T22:42:24.899 INFO:teuthology.orchestra.run.smithi115.stdout:4469 writesame 0x7104c79 thru 0x7110fec data_size 0xe(0xc374 bytes) 2024-09-17T22:42:24.907 INFO:teuthology.orchestra.run.smithi115.stdout:4470 read 0x8ecb7ce thru 0x8ecfb75 (0x43a8 bytes) 2024-09-17T22:42:24.908 INFO:teuthology.orchestra.run.smithi115.stdout:4472 punch from 0x2b7bedf to 0x2b80451, (0x4572 bytes) 2024-09-17T22:42:24.908 INFO:teuthology.orchestra.run.smithi115.stdout:4473 punch from 0x79e9b1 to 0x7a2626, (0x3c75 bytes) 2024-09-17T22:42:24.908 INFO:teuthology.orchestra.run.smithi115.stdout:4474 read 0x49a9684 thru 0x49b05cf (0x6f4c bytes) 2024-09-17T22:42:24.908 INFO:teuthology.orchestra.run.smithi115.stdout:4477 writesame 0xb1f213c thru 0xb1feaf3 data_size 0x14(0xc9b8 bytes) 2024-09-17T22:42:24.914 INFO:teuthology.orchestra.run.smithi115.stdout:4478 write 0xee9ca thru 0xf0939 (0x1f70 bytes) 2024-09-17T22:42:24.921 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:24.921 INFO:teuthology.orchestra.run.smithi115.stdout:4480 write 0x8ee25d1 thru 0x8ee6021 (0x3a51 bytes) 2024-09-17T22:42:24.925 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:24.925 INFO:teuthology.orchestra.run.smithi115.stdout:4482 writesame 0x2a3c061 thru 0x2a4264f data_size 0x11(0x65ef bytes) 2024-09-17T22:42:24.941 INFO:teuthology.orchestra.run.smithi115.stdout:4483 trunc from 0xd8036c1 to 0x220b9ed 2024-09-17T22:42:24.965 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:24.961+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:24.967 INFO:teuthology.orchestra.run.smithi115.stdout:4484 punch from 0x1677593 to 0x16861f7, (0xec64 bytes) 2024-09-17T22:42:24.967 INFO:teuthology.orchestra.run.smithi115.stdout:4485 read 0x141effc thru 0x142a56f (0xb574 bytes) 2024-09-17T22:42:24.967 INFO:teuthology.orchestra.run.smithi115.stdout:4486 read 0x91a1a9 thru 0x92a0b7 (0xff0f bytes) 2024-09-17T22:42:24.968 INFO:teuthology.orchestra.run.smithi115.stdout:4487 read 0xdd828e thru 0xde2f9e (0xad11 bytes) 2024-09-17T22:42:24.972 INFO:teuthology.orchestra.run.smithi115.stdout:4488 trunc from 0x220b9ed to 0x4fc43fa 2024-09-17T22:42:24.992 INFO:teuthology.orchestra.run.smithi115.stdout:4490 write 0xb9d1909 thru 0xb9d9f84 (0x867c bytes) 2024-09-17T22:42:25.006 INFO:teuthology.orchestra.run.smithi115.stdout:4491 write 0xe292d24 thru 0xe29c5fe (0x98db bytes) 2024-09-17T22:42:25.016 INFO:teuthology.orchestra.run.smithi115.stdout:4493 read 0x4906282 thru 0x49160ae (0xfe2d bytes) 2024-09-17T22:42:25.016 INFO:teuthology.orchestra.run.smithi115.stdout:4494 punch from 0xcf7a8a5 to 0xcf84ef5, (0xa650 bytes) 2024-09-17T22:42:25.016 INFO:teuthology.orchestra.run.smithi115.stdout:4495 punch from 0xdfbab to 0xe69da, (0x6e2f bytes) 2024-09-17T22:42:25.016 INFO:teuthology.orchestra.run.smithi115.stdout:4496 write 0xdf55f11 thru 0xdf65592 (0xf682 bytes) 2024-09-17T22:42:25.024 INFO:teuthology.orchestra.run.smithi115.stdout:4497 read 0x999007 thru 0x999d03 (0xcfd bytes) 2024-09-17T22:42:25.024 INFO:teuthology.orchestra.run.smithi115.stdout:4498 writesame 0x2af65fc thru 0x2afc37e data_size 0x1(0x5d83 bytes) 2024-09-17T22:42:25.028 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:25.028 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:25.036 INFO:teuthology.orchestra.run.smithi115.stdout:4499 read 0x606b3c3 thru 0x606cff9 (0x1c37 bytes) 2024-09-17T22:42:25.037 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:42:25.046 INFO:teuthology.orchestra.run.smithi115.stdout:4500 trunc from 0xe29c5ff to 0x5efc112 2024-09-17T22:42:25.059 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.059 INFO:teuthology.orchestra.run.smithi115.stdout:4502 punch from 0x1a646b4 to 0x1a71edd, (0xd829 bytes) 2024-09-17T22:42:25.059 INFO:teuthology.orchestra.run.smithi115.stdout:4503 write 0x5423ff4 thru 0x542913a (0x5147 bytes) 2024-09-17T22:42:25.068 INFO:teuthology.orchestra.run.smithi115.stdout:4506 trunc from 0x5efc112 to 0x1aa6dae 2024-09-17T22:42:25.078 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:25.075+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:25.117 INFO:teuthology.orchestra.run.smithi115.stdout:4508 write 0xc58d9c0 thru 0xc5939c4 (0x6005 bytes) 2024-09-17T22:42:25.120 INFO:teuthology.orchestra.run.smithi115.stdout:4510 writesame 0x77e97b7 thru 0x77f6297 data_size 0x1(0xcae1 bytes) 2024-09-17T22:42:25.137 INFO:teuthology.orchestra.run.smithi115.stdout:4511 trunc from 0xc5939c5 to 0xe57fe61 2024-09-17T22:42:25.144 INFO:teuthology.orchestra.run.smithi115.stdout:4512 write 0x35b0cc8 thru 0x35b9940 (0x8c79 bytes) 2024-09-17T22:42:25.153 INFO:teuthology.orchestra.run.smithi115.stdout:4513 writesame 0xea9bcd7 thru 0xeaa2eb7 data_size 0x1(0x71e1 bytes) 2024-09-17T22:42:25.154 INFO:teuthology.orchestra.run.smithi115.stdout:4514 writesame 0x1337f5a thru 0x1345cd4 data_size 0x1f(0xdd7b bytes) 2024-09-17T22:42:25.158 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.158 INFO:teuthology.orchestra.run.smithi115.stdout:4518 punch from 0xb675e66 to 0xb682791, (0xc92b bytes) 2024-09-17T22:42:25.159 INFO:teuthology.orchestra.run.smithi115.stdout:4519 writesame 0x3a50c87 thru 0x3a60c30 data_size 0x55(0xffaa bytes) 2024-09-17T22:42:25.166 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:25.163+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:25.168 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.168 INFO:teuthology.orchestra.run.smithi115.stdout:4521 write 0x7120e thru 0x72529 (0x131c bytes) 2024-09-17T22:42:25.174 INFO:teuthology.orchestra.run.smithi115.stdout:4522 write 0xc758666 thru 0xc768661 (0xfffc bytes) 2024-09-17T22:42:25.182 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.183 INFO:teuthology.orchestra.run.smithi115.stdout:4526 read 0x35cb76d thru 0x35dab87 (0xf41b bytes) 2024-09-17T22:42:25.183 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.195 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:25.195 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:25.199 INFO:teuthology.orchestra.run.smithi115.stdout:4529 trunc from 0xeaa2eb8 to 0xfdd81b 2024-09-17T22:42:25.205 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:42:25.227 INFO:teuthology.orchestra.run.smithi115.stdout:4530 read 0x35a786 thru 0x3635f2 (0x8e6d bytes) 2024-09-17T22:42:25.228 INFO:teuthology.orchestra.run.smithi115.stdout:4531 read 0x7ba564 thru 0x7c96c8 (0xf165 bytes) 2024-09-17T22:42:25.229 INFO:teuthology.orchestra.run.smithi115.stdout:4533 read 0x7f7451 thru 0x8002f6 (0x8ea6 bytes) 2024-09-17T22:42:25.236 INFO:teuthology.orchestra.run.smithi115.stdout:4534 write 0x1bba07e thru 0x1bc931c (0xf29f bytes) 2024-09-17T22:42:25.260 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is unset 2024-09-17T22:42:25.261 INFO:teuthology.orchestra.run.smithi115.stdout:4535 write 0xc6a254d thru 0xc6aa3f6 (0x7eaa bytes) 2024-09-17T22:42:25.268 INFO:teuthology.orchestra.run.smithi115.stdout:4537 writesame 0x4fb71ca thru 0x4fc3e04 data_size 0x4d(0xcc3b bytes) 2024-09-17T22:42:25.280 INFO:teuthology.orchestra.run.smithi115.stdout:4538 writesame 0x6c036af thru 0x6c0e96d data_size 0x7(0xb2bf bytes) 2024-09-17T22:42:25.288 INFO:teuthology.orchestra.run.smithi115.stdout:4539 write 0x12ae3bb thru 0x12b893f (0xa585 bytes) 2024-09-17T22:42:25.289 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:25.286+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:25.293 INFO:teuthology.orchestra.run.smithi115.stdout:4540 read 0x75b734d thru 0x75c652c (0xf1e0 bytes) 2024-09-17T22:42:25.293 INFO:teuthology.orchestra.run.smithi115.stdout:4542 write 0x3ace0a8 thru 0x3ad5032 (0x6f8b bytes) 2024-09-17T22:42:25.298 INFO:teuthology.orchestra.run.smithi115.stdout:4545 write 0xb347f82 thru 0xb34e632 (0x66b1 bytes) 2024-09-17T22:42:25.309 INFO:teuthology.orchestra.run.smithi115.stdout:4547 read 0x156b4a5 thru 0x1571aa7 (0x6603 bytes) 2024-09-17T22:42:25.309 INFO:teuthology.orchestra.run.smithi115.stdout:4548 write 0x9886c36 thru 0x9892689 (0xba54 bytes) 2024-09-17T22:42:25.312 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:42:25.318 INFO:teuthology.orchestra.run.smithi115.stdout:4549 punch from 0x587fe8f to 0x588189d, (0x1a0e bytes) 2024-09-17T22:42:25.318 INFO:teuthology.orchestra.run.smithi115.stdout:4550 write 0x1364f0f thru 0x1371982 (0xca74 bytes) 2024-09-17T22:42:25.327 INFO:teuthology.orchestra.run.smithi115.stdout:4552 trunc from 0xc6aa3f7 to 0xb969683 2024-09-17T22:42:25.338 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.338 INFO:teuthology.orchestra.run.smithi115.stdout:4554 write 0x32649d9 thru 0x32716b5 (0xccdd bytes) 2024-09-17T22:42:25.345 INFO:teuthology.orchestra.run.smithi115.stdout:4555 read 0x187cfe4 thru 0x187d6b5 (0x6d2 bytes) 2024-09-17T22:42:25.345 INFO:teuthology.orchestra.run.smithi115.stdout:4556 write 0x1b56cf6 thru 0x1b61a18 (0xad23 bytes) 2024-09-17T22:42:25.350 INFO:teuthology.orchestra.run.smithi115.stdout:4557 read 0x6602d8b thru 0x6604757 (0x19cd bytes) 2024-09-17T22:42:25.350 INFO:teuthology.orchestra.run.smithi115.stdout:4558 write 0x3f56876 thru 0x3f583ed (0x1b78 bytes) 2024-09-17T22:42:25.359 INFO:teuthology.orchestra.run.smithi115.stdout:4559 trunc from 0xb969683 to 0x800ee79 2024-09-17T22:42:25.362 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:25.362 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:25.367 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:25.364+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:25.372 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:42:25.379 INFO:teuthology.orchestra.run.smithi115.stdout:4560 writesame 0xb5f2e41 thru 0xb5f982a data_size 0x3(0x69ea bytes) 2024-09-17T22:42:25.383 INFO:teuthology.orchestra.run.smithi115.stdout:4561 trunc from 0xb5f982b to 0xe2958fc 2024-09-17T22:42:25.388 INFO:teuthology.orchestra.run.smithi115.stdout:4562 write 0x392b0ad thru 0x3937b58 (0xcaac bytes) 2024-09-17T22:42:25.393 INFO:teuthology.orchestra.run.smithi115.stdout:4563 write 0x5b90d9a thru 0x5b98e86 (0x80ed bytes) 2024-09-17T22:42:25.403 INFO:teuthology.orchestra.run.smithi115.stdout:4564 read 0x16b281e thru 0x16c1a10 (0xf1f3 bytes) 2024-09-17T22:42:25.403 INFO:teuthology.orchestra.run.smithi115.stdout:4565 read 0xc0d2905 thru 0xc0d5fe4 (0x36e0 bytes) 2024-09-17T22:42:25.403 INFO:teuthology.orchestra.run.smithi115.stdout:4567 read 0xaf8af74 thru 0xaf8eb30 (0x3bbd bytes) 2024-09-17T22:42:25.403 INFO:teuthology.orchestra.run.smithi115.stdout:4569 write 0x663bd25 thru 0x6648c6b (0xcf47 bytes) 2024-09-17T22:42:25.410 INFO:teuthology.orchestra.run.smithi115.stdout:4571 write 0x81eb575 thru 0x81f3030 (0x7abc bytes) 2024-09-17T22:42:25.416 INFO:teuthology.orchestra.run.smithi115.stdout:4572 write 0x19b5f9c thru 0x19b74e9 (0x154e bytes) 2024-09-17T22:42:25.417 INFO:teuthology.orchestra.run.smithi115.stdout:4573 writesame 0xc264d66 thru 0xc26f567 data_size 0x73(0xa802 bytes) 2024-09-17T22:42:25.422 INFO:teuthology.orchestra.run.smithi115.stdout:4574 write 0xdec6313 thru 0xdecd22d (0x6f1b bytes) 2024-09-17T22:42:25.426 INFO:teuthology.orchestra.run.smithi115.stdout:4575 read 0xa9e5427 thru 0xa9f29f2 (0xd5cc bytes) 2024-09-17T22:42:25.428 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:25.442 INFO:teuthology.orchestra.run.smithi115.stdout:4577 trunc from 0xe2958fc to 0x19978f3 2024-09-17T22:42:25.446 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:25.447 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:25.467 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:25.468 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:25.468 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:25.465+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:25.478 INFO:teuthology.orchestra.run.smithi115.stdout:4578 write 0x97910cf thru 0x9797bed (0x6b1f bytes) 2024-09-17T22:42:25.483 INFO:teuthology.orchestra.run.smithi115.stdout:4581 flatten 2024-09-17T22:42:25.487 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:25.487 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:25.518 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:25.518 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:25.527 INFO:teuthology.orchestra.run.smithi115.stdout:4582 write 0x9db40cb thru 0x9dbef25 (0xae5b bytes) 2024-09-17T22:42:25.531 INFO:teuthology.orchestra.run.smithi115.stdout:4583 punch from 0x6eea3ef to 0x6eee9d9, (0x45ea bytes) 2024-09-17T22:42:25.535 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:25.535 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:25.536 INFO:teuthology.orchestra.run.smithi115.stdout:4585 trunc from 0x9dbef26 to 0x661e64e 2024-09-17T22:42:25.539 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:25.539 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:25.546 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:42:25.546 INFO:teuthology.orchestra.run.smithi115.stdout:4586 writesame 0x244bb5f thru 0x2454d28 data_size 0x1(0x91ca bytes) 2024-09-17T22:42:25.558 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:25.559 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:25.560 INFO:teuthology.orchestra.run.smithi115.stdout:4587 write 0x32769b6 thru 0x327a81c (0x3e67 bytes) 2024-09-17T22:42:25.564 INFO:teuthology.orchestra.run.smithi115.stdout:4588 read 0x5b1646a thru 0x5b1c1c2 (0x5d59 bytes) 2024-09-17T22:42:25.568 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:25.566+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:25.573 INFO:teuthology.orchestra.run.smithi115.stdout:4591 writesame 0x92950d8 thru 0x92a4997 data_size 0xa0(0xf8c0 bytes) 2024-09-17T22:42:25.578 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:25.578 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:25.582 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:25.583 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:25.584 INFO:teuthology.orchestra.run.smithi115.stdout:4592 write 0xcd9d899 thru 0xcdab888 (0xdff0 bytes) 2024-09-17T22:42:25.590 INFO:teuthology.orchestra.run.smithi115.stdout:4594 read 0x9066877 thru 0x906858c (0x1d16 bytes) 2024-09-17T22:42:25.590 INFO:teuthology.orchestra.run.smithi115.stdout:4595 write 0x4599e44 thru 0x45a88ec (0xeaa9 bytes) 2024-09-17T22:42:25.601 INFO:teuthology.orchestra.run.smithi115.stdout:4596 punch from 0x8722ef6 to 0x872a112, (0x721c bytes) 2024-09-17T22:42:25.602 INFO:teuthology.orchestra.run.smithi115.stdout:4597 punch from 0x2afb22c to 0x2b01666, (0x643a bytes) 2024-09-17T22:42:25.602 INFO:teuthology.orchestra.run.smithi115.stdout:4598 read 0x9b75060 thru 0x9b82404 (0xd3a5 bytes) 2024-09-17T22:42:25.602 INFO:teuthology.orchestra.run.smithi115.stdout:4599 read 0x8dc4866 thru 0x8dcdd36 (0x94d1 bytes) 2024-09-17T22:42:25.602 INFO:teuthology.orchestra.run.smithi115.stdout:4600 read 0xa9363ce thru 0xa940640 (0xa273 bytes) 2024-09-17T22:42:25.602 INFO:teuthology.orchestra.run.smithi115.stdout:4601 read 0x5940f97 thru 0x5947f1e (0x6f88 bytes) 2024-09-17T22:42:25.602 INFO:teuthology.orchestra.run.smithi115.stdout:4602 punch from 0xe87400 to 0xe92213, (0xae13 bytes) 2024-09-17T22:42:25.602 INFO:teuthology.orchestra.run.smithi115.stdout:4603 read 0x6735a32 thru 0x673d3cb (0x799a bytes) 2024-09-17T22:42:25.602 INFO:teuthology.orchestra.run.smithi115.stdout:4606 read 0x62ecdd5 thru 0x62f63b8 (0x95e4 bytes) 2024-09-17T22:42:25.602 INFO:teuthology.orchestra.run.smithi115.stdout:4607 write 0x26d11ca thru 0x26dbf90 (0xadc7 bytes) 2024-09-17T22:42:25.606 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.606 INFO:teuthology.orchestra.run.smithi115.stdout:4610 writesame 0x22798e8 thru 0x227d519 data_size 0x43(0x3c32 bytes) 2024-09-17T22:42:25.612 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.612 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.612 INFO:teuthology.orchestra.run.smithi115.stdout:4615 punch from 0x8e95c42 to 0x8e9cb33, (0x6ef1 bytes) 2024-09-17T22:42:25.612 INFO:teuthology.orchestra.run.smithi115.stdout:4616 writesame 0x28e5ced thru 0x28e730f data_size 0x1(0x1623 bytes) 2024-09-17T22:42:25.613 INFO:teuthology.orchestra.run.smithi115.stdout:4617 read 0xb281044 thru 0xb28f994 (0xe951 bytes) 2024-09-17T22:42:25.613 INFO:teuthology.orchestra.run.smithi115.stdout:4618 read 0x942b417 thru 0x942e1b1 (0x2d9b bytes) 2024-09-17T22:42:25.613 INFO:teuthology.orchestra.run.smithi115.stdout:4619 writesame 0x2b7f0e0 thru 0x2b88fcf data_size 0x8(0x9ef0 bytes) 2024-09-17T22:42:25.619 INFO:teuthology.orchestra.run.smithi115.stdout:4621 write 0x3b259fa thru 0x3b2ac54 (0x525b bytes) 2024-09-17T22:42:25.624 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.624 INFO:teuthology.orchestra.run.smithi115.stdout:4626 write 0x142910a thru 0x143604e (0xcf45 bytes) 2024-09-17T22:42:25.626 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.632 INFO:teuthology.orchestra.run.smithi115.stdout:4628 writesame 0xd60b8c6 thru 0xd60d5ac data_size 0x7(0x1ce7 bytes) 2024-09-17T22:42:25.632 INFO:teuthology.orchestra.run.smithi115.stdout:4631 write 0x6b98f1c thru 0x6ba4bda (0xbcbf bytes) 2024-09-17T22:42:25.637 INFO:teuthology.orchestra.run.smithi115.stdout:4632 read 0x23b7cee thru 0x23bcace (0x4de1 bytes) 2024-09-17T22:42:25.637 INFO:teuthology.orchestra.run.smithi115.stdout:4633 write 0xab3ee85 thru 0xab4e4b2 (0xf62e bytes) 2024-09-17T22:42:25.645 INFO:teuthology.orchestra.run.smithi115.stdout:4634 read 0x10ae0d5 thru 0x10b2180 (0x40ac bytes) 2024-09-17T22:42:25.645 INFO:teuthology.orchestra.run.smithi115.stdout:4635 writesame 0x15082e thru 0x151211 data_size 0x6(0x9e4 bytes) 2024-09-17T22:42:25.646 INFO:teuthology.orchestra.run.smithi115.stdout:4638 writesame 0x4a50e3f thru 0x4a5c47b data_size 0x1(0xb63d bytes) 2024-09-17T22:42:25.664 INFO:teuthology.orchestra.run.smithi115.stdout:4639 read 0x63d9e60 thru 0x63e5ae7 (0xbc88 bytes) 2024-09-17T22:42:25.664 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.664 INFO:teuthology.orchestra.run.smithi115.stdout:4641 read 0x6814751 thru 0x6823397 (0xec47 bytes) 2024-09-17T22:42:25.664 INFO:teuthology.orchestra.run.smithi115.stdout:4642 read 0x409fca4 thru 0x40aefbd (0xf31a bytes) 2024-09-17T22:42:25.664 INFO:teuthology.orchestra.run.smithi115.stdout:4644 read 0x5d0c156 thru 0x5d18273 (0xc11e bytes) 2024-09-17T22:42:25.665 INFO:teuthology.orchestra.run.smithi115.stdout:4645 trunc from 0xd60d5ad to 0xbf9f710 2024-09-17T22:42:25.669 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:25.666+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:25.674 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202974 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725913+0000", 2024-09-17T22:42:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9437216990000001, 2024-09-17T22:42:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022636643000000001, 2024-09-17T22:42:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.674 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202974 2024-09-17T22:42:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725913+0000", 2024-09-17T22:42:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725913+0000", 2024-09-17T22:42:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725913+0000", 2024-09-17T22:42:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5199999999999997e-07 2024-09-17T22:42:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725916+0000", 2024-09-17T22:42:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8100000000000002e-06 2024-09-17T22:42:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725924+0000", 2024-09-17T22:42:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1289999999999996e-06 2024-09-17T22:42:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725930+0000", 2024-09-17T22:42:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3469999999999998e-06 2024-09-17T22:42:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748360+0000", 2024-09-17T22:42:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022430410000000001 2024-09-17T22:42:25.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748443+0000", 2024-09-17T22:42:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3355000000000004e-05 2024-09-17T22:42:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748550+0000", 2024-09-17T22:42:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010603999999999999 2024-09-17T22:42:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202975 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725937+0000", 2024-09-17T22:42:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9436976599999998, 2024-09-17T22:42:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022787498, 2024-09-17T22:42:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202975 2024-09-17T22:42:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.679 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725937+0000", 2024-09-17T22:42:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725937+0000", 2024-09-17T22:42:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725937+0000", 2024-09-17T22:42:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4299999999999998e-07 2024-09-17T22:42:25.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725939+0000", 2024-09-17T22:42:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0669999999999999e-06 2024-09-17T22:42:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725946+0000", 2024-09-17T22:42:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6390000000000003e-06 2024-09-17T22:42:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725951+0000", 2024-09-17T22:42:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0089999999999998e-06 2024-09-17T22:42:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748593+0000", 2024-09-17T22:42:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022642343999999998 2024-09-17T22:42:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748657+0000", 2024-09-17T22:42:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3133999999999996e-05 2024-09-17T22:42:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748724+0000", 2024-09-17T22:42:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7861999999999995e-05 2024-09-17T22:42:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202976 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725958+0000", 2024-09-17T22:42:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9436764789999996, 2024-09-17T22:42:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022890631000000002, 2024-09-17T22:42:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.683 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202976 2024-09-17T22:42:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725958+0000", 2024-09-17T22:42:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725958+0000", 2024-09-17T22:42:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725959+0000", 2024-09-17T22:42:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7199999999999999e-07 2024-09-17T22:42:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725961+0000", 2024-09-17T22:42:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9860000000000001e-06 2024-09-17T22:42:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725967+0000", 2024-09-17T22:42:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6630000000000004e-06 2024-09-17T22:42:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725972+0000", 2024-09-17T22:42:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0549999999999999e-06 2024-09-17T22:42:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748753+0000", 2024-09-17T22:42:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022780959 2024-09-17T22:42:25.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748792+0000", 2024-09-17T22:42:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8581999999999998e-05 2024-09-17T22:42:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748849+0000", 2024-09-17T22:42:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6913999999999998e-05 2024-09-17T22:42:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202977 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:42:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9436555220000002, 2024-09-17T22:42:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023068750999999998, 2024-09-17T22:42:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202977 2024-09-17T22:42:25.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:42:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:42:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725980+0000", 2024-09-17T22:42:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:42:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725982+0000", 2024-09-17T22:42:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0669999999999999e-06 2024-09-17T22:42:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725988+0000", 2024-09-17T22:42:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6610000000000003e-06 2024-09-17T22:42:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725993+0000", 2024-09-17T22:42:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:42:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748888+0000", 2024-09-17T22:42:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022894930000000001 2024-09-17T22:42:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748951+0000", 2024-09-17T22:42:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3638000000000003e-05 2024-09-17T22:42:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749048+0000", 2024-09-17T22:42:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6343999999999998e-05 2024-09-17T22:42:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202978 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2211840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:25.692 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9436350100000004, 2024-09-17T22:42:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023238050999999999, 2024-09-17T22:42:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202978 2024-09-17T22:42:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:25.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.58e-07 2024-09-17T22:42:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726002+0000", 2024-09-17T22:42:25.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.193e-06 2024-09-17T22:42:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726009+0000", 2024-09-17T22:42:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8000000000000001e-06 2024-09-17T22:42:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726014+0000", 2024-09-17T22:42:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9840000000000001e-06 2024-09-17T22:42:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749090+0000", 2024-09-17T22:42:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023075917000000001 2024-09-17T22:42:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749163+0000", 2024-09-17T22:42:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3219999999999994e-05 2024-09-17T22:42:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749238+0000", 2024-09-17T22:42:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4479000000000003e-05 2024-09-17T22:42:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.696 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202979 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2342912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:42:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9436135590000001, 2024-09-17T22:42:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023393536999999999, 2024-09-17T22:42:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.697 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202979 2024-09-17T22:42:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:42:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:42:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726022+0000", 2024-09-17T22:42:25.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:42:25.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726024+0000", 2024-09-17T22:42:25.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1969999999999999e-06 2024-09-17T22:42:25.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726030+0000", 2024-09-17T22:42:25.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.691e-06 2024-09-17T22:42:25.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726035+0000", 2024-09-17T22:42:25.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8029999999999996e-06 2024-09-17T22:42:25.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749280+0000", 2024-09-17T22:42:25.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023244668 2024-09-17T22:42:25.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749344+0000", 2024-09-17T22:42:25.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4418000000000006e-05 2024-09-17T22:42:25.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749415+0000", 2024-09-17T22:42:25.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0272999999999996e-05 2024-09-17T22:42:25.701 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.701 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.701 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.701 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202980 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2473984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.701 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:25.701 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9435927900000003, 2024-09-17T22:42:25.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023562221000000001, 2024-09-17T22:42:25.702 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.702 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.702 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.702 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.702 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.702 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.702 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202980 2024-09-17T22:42:25.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.702 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:25.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:25.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:25.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:42:25.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726044+0000", 2024-09-17T22:42:25.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9489999999999999e-06 2024-09-17T22:42:25.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726066+0000", 2024-09-17T22:42:25.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1508000000000001e-05 2024-09-17T22:42:25.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726072+0000", 2024-09-17T22:42:25.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3450000000000002e-06 2024-09-17T22:42:25.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749455+0000", 2024-09-17T22:42:25.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023382824999999999 2024-09-17T22:42:25.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749527+0000", 2024-09-17T22:42:25.705 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1728000000000002e-05 2024-09-17T22:42:25.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.706 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.706 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749604+0000", 2024-09-17T22:42:25.706 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7428999999999993e-05 2024-09-17T22:42:25.706 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.706 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.706 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.706 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.706 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202981 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.706 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:42:25.706 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9435545149999998, 2024-09-17T22:42:25.707 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023677800999999998, 2024-09-17T22:42:25.707 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.707 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.707 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.707 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.707 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.707 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.707 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202981 2024-09-17T22:42:25.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.707 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.707 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.707 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:42:25.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:42:25.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726081+0000", 2024-09-17T22:42:25.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5499999999999998e-07 2024-09-17T22:42:25.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726083+0000", 2024-09-17T22:42:25.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1780000000000002e-06 2024-09-17T22:42:25.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726090+0000", 2024-09-17T22:42:25.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8510000000000001e-06 2024-09-17T22:42:25.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726101+0000", 2024-09-17T22:42:25.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.133e-05 2024-09-17T22:42:25.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749639+0000", 2024-09-17T22:42:25.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023537679999999998 2024-09-17T22:42:25.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749688+0000", 2024-09-17T22:42:25.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9548000000000001e-05 2024-09-17T22:42:25.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749758+0000", 2024-09-17T22:42:25.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9758999999999994e-05 2024-09-17T22:42:25.711 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.711 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.711 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.711 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202982 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.711 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:25.711 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9435269320000002, 2024-09-17T22:42:25.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023841469000000001, 2024-09-17T22:42:25.711 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.711 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.711 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.712 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.712 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.712 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.712 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202982 2024-09-17T22:42:25.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.712 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:25.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:25.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:25.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:42:25.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726110+0000", 2024-09-17T22:42:25.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.902e-06 2024-09-17T22:42:25.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726117+0000", 2024-09-17T22:42:25.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6189999999999996e-06 2024-09-17T22:42:25.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726122+0000", 2024-09-17T22:42:25.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.198e-06 2024-09-17T22:42:25.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749795+0000", 2024-09-17T22:42:25.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023673397999999998 2024-09-17T22:42:25.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749877+0000", 2024-09-17T22:42:25.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1910999999999996e-05 2024-09-17T22:42:25.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749949+0000", 2024-09-17T22:42:25.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2001000000000004e-05 2024-09-17T22:42:25.715 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.715 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.715 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.715 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202983 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.716 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:25.716 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9435058730000003, 2024-09-17T22:42:25.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024014081, 2024-09-17T22:42:25.716 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.716 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.716 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.716 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.716 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.716 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.716 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202983 2024-09-17T22:42:25.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.716 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:25.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:25.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:25.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:42:25.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726131+0000", 2024-09-17T22:42:25.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9429999999999999e-06 2024-09-17T22:42:25.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726138+0000", 2024-09-17T22:42:25.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T22:42:25.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.718 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726143+0000", 2024-09-17T22:42:25.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0629999999999996e-06 2024-09-17T22:42:25.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749991+0000", 2024-09-17T22:42:25.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023847671000000001 2024-09-17T22:42:25.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750065+0000", 2024-09-17T22:42:25.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4569999999999999e-05 2024-09-17T22:42:25.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750143+0000", 2024-09-17T22:42:25.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7649000000000001e-05 2024-09-17T22:42:25.720 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.720 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.720 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.720 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202984 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.720 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:42:25.720 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9434851719999999, 2024-09-17T22:42:25.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024177605000000001, 2024-09-17T22:42:25.720 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.720 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.720 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.720 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.721 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.721 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.721 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202984 2024-09-17T22:42:25.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.721 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:42:25.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:42:25.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726150+0000", 2024-09-17T22:42:25.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T22:42:25.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726152+0000", 2024-09-17T22:42:25.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:42:25.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.722 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.722 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726162+0000", 2024-09-17T22:42:25.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7839999999999998e-06 2024-09-17T22:42:25.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726167+0000", 2024-09-17T22:42:25.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5690000000000004e-06 2024-09-17T22:42:25.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750184+0000", 2024-09-17T22:42:25.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024016909999999999 2024-09-17T22:42:25.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750255+0000", 2024-09-17T22:42:25.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1137999999999996e-05 2024-09-17T22:42:25.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750327+0000", 2024-09-17T22:42:25.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1903000000000006e-05 2024-09-17T22:42:25.724 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.724 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.724 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.725 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202985 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.725 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:25.725 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9434597849999999, 2024-09-17T22:42:25.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024280500999999999, 2024-09-17T22:42:25.725 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.725 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.725 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.725 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.725 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.725 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.725 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202985 2024-09-17T22:42:25.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.725 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:25.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:25.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:25.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8100000000000003e-07 2024-09-17T22:42:25.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726182+0000", 2024-09-17T22:42:25.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:42:25.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726190+0000", 2024-09-17T22:42:25.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:42:25.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.727 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726195+0000", 2024-09-17T22:42:25.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2809999999999999e-06 2024-09-17T22:42:25.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750366+0000", 2024-09-17T22:42:25.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024171635 2024-09-17T22:42:25.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750410+0000", 2024-09-17T22:42:25.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3823999999999999e-05 2024-09-17T22:42:25.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750455+0000", 2024-09-17T22:42:25.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5039e-05 2024-09-17T22:42:25.729 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.729 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.729 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.729 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202986 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.729 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:42:25.729 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9434322049999997, 2024-09-17T22:42:25.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024453077, 2024-09-17T22:42:25.729 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.729 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.729 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.729 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.730 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.730 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.730 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202986 2024-09-17T22:42:25.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.730 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:42:25.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:42:25.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726203+0000", 2024-09-17T22:42:25.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T22:42:25.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726205+0000", 2024-09-17T22:42:25.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0250000000000001e-06 2024-09-17T22:42:25.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.731 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.731 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726211+0000", 2024-09-17T22:42:25.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1340000000000001e-06 2024-09-17T22:42:25.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726222+0000", 2024-09-17T22:42:25.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1105e-05 2024-09-17T22:42:25.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750520+0000", 2024-09-17T22:42:25.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024297462999999998 2024-09-17T22:42:25.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750585+0000", 2024-09-17T22:42:25.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5940999999999998e-05 2024-09-17T22:42:25.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750655+0000", 2024-09-17T22:42:25.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0012000000000007e-05 2024-09-17T22:42:25.733 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.733 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.733 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.733 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202987 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.733 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:25.733 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9434058250000001, 2024-09-17T22:42:25.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024622569, 2024-09-17T22:42:25.734 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.734 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.734 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.734 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.734 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.734 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.734 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202987 2024-09-17T22:42:25.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.734 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:25.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:25.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:25.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:42:25.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726238+0000", 2024-09-17T22:42:25.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7430000000000008e-06 2024-09-17T22:42:25.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726245+0000", 2024-09-17T22:42:25.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6640000000000001e-06 2024-09-17T22:42:25.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726249+0000", 2024-09-17T22:42:25.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9119999999999997e-06 2024-09-17T22:42:25.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.736 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.736 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.736 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750694+0000", 2024-09-17T22:42:25.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024444105000000001 2024-09-17T22:42:25.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750767+0000", 2024-09-17T22:42:25.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3577000000000004e-05 2024-09-17T22:42:25.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750851+0000", 2024-09-17T22:42:25.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4189000000000003e-05 2024-09-17T22:42:25.737 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.737 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.737 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.738 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202988 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.738 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:25.738 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9433789629999998, 2024-09-17T22:42:25.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024808883, 2024-09-17T22:42:25.738 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.738 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.738 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.738 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.738 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.738 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.738 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202988 2024-09-17T22:42:25.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.739 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:25.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:25.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:25.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5499999999999999e-07 2024-09-17T22:42:25.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726258+0000", 2024-09-17T22:42:25.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8020000000000001e-06 2024-09-17T22:42:25.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.740 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.740 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726272+0000", 2024-09-17T22:42:25.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3828e-05 2024-09-17T22:42:25.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726277+0000", 2024-09-17T22:42:25.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2379999999999997e-06 2024-09-17T22:42:25.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750895+0000", 2024-09-17T22:42:25.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024618543 2024-09-17T22:42:25.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750978+0000", 2024-09-17T22:42:25.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3030999999999998e-05 2024-09-17T22:42:25.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751065+0000", 2024-09-17T22:42:25.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6086000000000002e-05 2024-09-17T22:42:25.742 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.742 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.743 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.743 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202989 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.743 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:42:25.743 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.94335009, 2024-09-17T22:42:25.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024976788, 2024-09-17T22:42:25.743 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.743 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.743 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.743 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.743 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.743 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.744 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202989 2024-09-17T22:42:25.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.744 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:42:25.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:42:25.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726285+0000", 2024-09-17T22:42:25.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T22:42:25.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726287+0000", 2024-09-17T22:42:25.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9860000000000001e-06 2024-09-17T22:42:25.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.745 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.745 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726294+0000", 2024-09-17T22:42:25.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2849999999999997e-06 2024-09-17T22:42:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726299+0000", 2024-09-17T22:42:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0150000000000003e-06 2024-09-17T22:42:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751109+0000", 2024-09-17T22:42:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024810175 2024-09-17T22:42:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751184+0000", 2024-09-17T22:42:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4640000000000004e-05 2024-09-17T22:42:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751261+0000", 2024-09-17T22:42:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7317000000000005e-05 2024-09-17T22:42:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.747 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202990 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9433298749999999, 2024-09-17T22:42:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025156767999999999, 2024-09-17T22:42:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202990 2024-09-17T22:42:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.748 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:25.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:25.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:25.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:42:25.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726307+0000", 2024-09-17T22:42:25.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7090000000000001e-06 2024-09-17T22:42:25.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726312+0000", 2024-09-17T22:42:25.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4909999999999996e-06 2024-09-17T22:42:25.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726317+0000", 2024-09-17T22:42:25.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3800000000000004e-06 2024-09-17T22:42:25.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751308+0000", 2024-09-17T22:42:25.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024991143 2024-09-17T22:42:25.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751385+0000", 2024-09-17T22:42:25.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6793999999999997e-05 2024-09-17T22:42:25.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751461+0000", 2024-09-17T22:42:25.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6891999999999995e-05 2024-09-17T22:42:25.752 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.752 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.752 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.752 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202991 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.752 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:25.752 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9433118409999999, 2024-09-17T22:42:25.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025289477000000001, 2024-09-17T22:42:25.752 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.753 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.753 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.753 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.753 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.753 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.753 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202991 2024-09-17T22:42:25.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.753 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:25.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:25.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.754 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:25.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.27e-07 2024-09-17T22:42:25.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.754 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726325+0000", 2024-09-17T22:42:25.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1459999999999999e-06 2024-09-17T22:42:25.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726333+0000", 2024-09-17T22:42:25.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1289999999999996e-06 2024-09-17T22:42:25.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726339+0000", 2024-09-17T22:42:25.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2530000000000004e-06 2024-09-17T22:42:25.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751499+0000", 2024-09-17T22:42:25.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025160109 2024-09-17T22:42:25.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751543+0000", 2024-09-17T22:42:25.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4317000000000003e-05 2024-09-17T22:42:25.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751612+0000", 2024-09-17T22:42:25.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9096000000000004e-05 2024-09-17T22:42:25.757 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.757 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.757 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.757 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202992 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.757 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:25.757 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9432897770000004, 2024-09-17T22:42:25.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025420585999999998, 2024-09-17T22:42:25.757 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.758 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.758 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.758 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.758 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.758 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.758 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202992 2024-09-17T22:42:25.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.758 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:25.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:25.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:25.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5199999999999998e-07 2024-09-17T22:42:25.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726347+0000", 2024-09-17T22:42:25.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9939999999999999e-06 2024-09-17T22:42:25.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726358+0000", 2024-09-17T22:42:25.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0421999999999999e-05 2024-09-17T22:42:25.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726363+0000", 2024-09-17T22:42:25.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:42:25.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751644+0000", 2024-09-17T22:42:25.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025280727999999999 2024-09-17T22:42:25.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751712+0000", 2024-09-17T22:42:25.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8207999999999995e-05 2024-09-17T22:42:25.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751765+0000", 2024-09-17T22:42:25.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3533999999999999e-05 2024-09-17T22:42:25.762 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.762 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.762 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.762 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202993 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:25.763 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:25.763 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 7.9432657859999996, 2024-09-17T22:42:25.763 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025516259999999999, 2024-09-17T22:42:25.763 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:25.763 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:25.763 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:25.763 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:25.763 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:25.763 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:25.763 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202993 2024-09-17T22:42:25.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.764 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:25.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:25.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:25.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:25.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:25.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:25.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:25.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:25.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:42:25.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:25.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726371+0000", 2024-09-17T22:42:25.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T22:42:25.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:25.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726380+0000", 2024-09-17T22:42:25.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9109999999999999e-06 2024-09-17T22:42:25.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:25.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726385+0000", 2024-09-17T22:42:25.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7380000000000002e-06 2024-09-17T22:42:25.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:25.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751785+0000", 2024-09-17T22:42:25.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025400526 2024-09-17T22:42:25.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:25.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751824+0000", 2024-09-17T22:42:25.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8989999999999998e-05 2024-09-17T22:42:25.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:25.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:25.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:25.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751885+0000", 2024-09-17T22:42:25.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0937000000000003e-05 2024-09-17T22:42:25.767 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.767 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.767 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.767 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:25.767 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:25.767 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:25.770 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:42:25.772 INFO:teuthology.orchestra.run.smithi115.stdout:4646 read 0x722a099 thru 0x722f702 (0x566a bytes) 2024-09-17T22:42:25.772 INFO:teuthology.orchestra.run.smithi115.stdout:4647 read 0xbd0096d thru 0xbd0c8d9 (0xbf6d bytes) 2024-09-17T22:42:25.772 INFO:teuthology.orchestra.run.smithi115.stdout:4649 trunc from 0xbf9f710 to 0x5ea660a 2024-09-17T22:42:25.772 INFO:teuthology.orchestra.run.smithi115.stdout:4651 read 0x7c1f24 thru 0x7c69c7 (0x4aa4 bytes) 2024-09-17T22:42:25.772 INFO:teuthology.orchestra.run.smithi115.stdout:4653 write 0x5ff3941 thru 0x6000878 (0xcf38 bytes) 2024-09-17T22:42:25.772 INFO:teuthology.orchestra.run.smithi115.stdout:4654 write 0xe4ce272 thru 0xe4d4321 (0x60b0 bytes) 2024-09-17T22:42:25.772 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.772 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.772 INFO:teuthology.orchestra.run.smithi115.stdout:4660 read 0x440ac93 thru 0x4414f8d (0xa2fb bytes) 2024-09-17T22:42:25.772 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.772 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.772 INFO:teuthology.orchestra.run.smithi115.stdout:4663 read 0xa4a18f7 thru 0xa4a817b (0x6885 bytes) 2024-09-17T22:42:25.773 INFO:teuthology.orchestra.run.smithi115.stdout:4665 write 0x8f0009a thru 0x8f012f6 (0x125d bytes) 2024-09-17T22:42:25.773 INFO:teuthology.orchestra.run.smithi115.stdout:4667 write 0xb83b5d0 thru 0xb840b62 (0x5593 bytes) 2024-09-17T22:42:25.773 INFO:teuthology.orchestra.run.smithi115.stdout:4669 read 0x24dbd7b thru 0x24e7f9e (0xc224 bytes) 2024-09-17T22:42:25.773 INFO:teuthology.orchestra.run.smithi115.stdout:4670 writesame 0x97a78c4 thru 0x97b6f2d data_size 0x1(0xf66a bytes) 2024-09-17T22:42:25.773 INFO:teuthology.orchestra.run.smithi115.stdout:4671 read 0x37a03cd thru 0x37af8ab (0xf4df bytes) 2024-09-17T22:42:25.773 INFO:teuthology.orchestra.run.smithi115.stdout:4672 write 0x72ffee3 thru 0x730f49e (0xf5bc bytes) 2024-09-17T22:42:25.773 INFO:teuthology.orchestra.run.smithi115.stdout:4675 read 0xd1b7f10 thru 0xd1ba9dd (0x2ace bytes) 2024-09-17T22:42:25.773 INFO:teuthology.orchestra.run.smithi115.stdout:4677 write 0x2aaf48 thru 0x2b63bb (0xb474 bytes) 2024-09-17T22:42:25.773 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.773 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.773 INFO:teuthology.orchestra.run.smithi115.stdout:4680 read 0x94e9762 thru 0x94f165f (0x7efe bytes) 2024-09-17T22:42:25.773 INFO:teuthology.orchestra.run.smithi115.stdout:4681 read 0x73395bf thru 0x733e3ca (0x4e0c bytes) 2024-09-17T22:42:25.773 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.774 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.774 INFO:teuthology.orchestra.run.smithi115.stdout:4684 write 0x87c0e88 thru 0x87cacdf (0x9e58 bytes) 2024-09-17T22:42:25.774 INFO:teuthology.orchestra.run.smithi115.stdout:4685 read 0xbbeedab thru 0xbbf0576 (0x17cc bytes) 2024-09-17T22:42:25.774 INFO:teuthology.orchestra.run.smithi115.stdout:4686 writesame 0x5a70173 thru 0x5a7a99a data_size 0x4(0xa828 bytes) 2024-09-17T22:42:25.774 INFO:teuthology.orchestra.run.smithi115.stdout:4688 writesame 0xc8158b4 thru 0xc819d5e data_size 0x1(0x44ab bytes) 2024-09-17T22:42:25.774 INFO:teuthology.orchestra.run.smithi115.stdout:4689 write 0x1f4e432 thru 0x1f50832 (0x2401 bytes) 2024-09-17T22:42:25.775 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:25.771+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:25.777 INFO:teuthology.orchestra.run.smithi115.stdout:4690 read 0xcc7b02b thru 0xcc81556 (0x652c bytes) 2024-09-17T22:42:25.778 INFO:teuthology.orchestra.run.smithi115.stdout:4691 writesame 0x1cb53e3 thru 0x1cba9e7 data_size 0x1(0x5605 bytes) 2024-09-17T22:42:25.786 INFO:teuthology.orchestra.run.smithi115.stdout:4692 write 0x301314c thru 0x3017964 (0x4819 bytes) 2024-09-17T22:42:25.793 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:25.794 INFO:teuthology.orchestra.run.smithi115.stdout:4696 write 0xa1497aa thru 0xa14fc88 (0x64df bytes) 2024-09-17T22:42:25.800 INFO:teuthology.orchestra.run.smithi115.stdout:4700 read 0x53d767e thru 0x53deb88 (0x750b bytes) 2024-09-17T22:42:25.800 INFO:teuthology.orchestra.run.smithi115.stdout:4702 clone 34 order 24 su 8388608 sc 8 2024-09-17T22:42:25.876 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:25.873+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:25.878 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:25.878 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:25.878 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:25.878 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:25.887 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:42:25.976 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:25.974+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:26.029 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:26.029 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:26.029 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:26.029 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:26.029 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:26.037 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:42:26.077 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:26.075+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:26.141 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:26.141 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:26.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:26.141 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:26.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.142 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:42:26.142 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:26.142 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 82.779505826999994, 2024-09-17T22:42:26.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.249158453, 2024-09-17T22:42:26.142 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.142 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.142 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:26.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:42:26.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:42:26.142 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-17T22:42:26.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.143 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:26.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:26.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358680+0000", 2024-09-17T22:42:26.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.592e-06 2024-09-17T22:42:26.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358688+0000", 2024-09-17T22:42:26.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1610000000000003e-06 2024-09-17T22:42:26.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358697+0000", 2024-09-17T22:42:26.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2580000000000001e-06 2024-09-17T22:42:26.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358706+0000", 2024-09-17T22:42:26.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1640000000000007e-06 2024-09-17T22:42:26.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358766+0000", 2024-09-17T22:42:26.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9893999999999999e-05 2024-09-17T22:42:26.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:42:26.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358778+0000", 2024-09-17T22:42:26.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1148999999999999e-05 2024-09-17T22:42:26.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606347+0000", 2024-09-17T22:42:26.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.247569971 2024-09-17T22:42:26.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606533+0000", 2024-09-17T22:42:26.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018599400000000001 2024-09-17T22:42:26.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:42:26.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606752+0000", 2024-09-17T22:42:26.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000218373 2024-09-17T22:42:26.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:42:26.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607455+0000", 2024-09-17T22:42:26.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070311799999999997 2024-09-17T22:42:26.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:26.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607735+0000", 2024-09-17T22:42:26.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027972300000000001 2024-09-17T22:42:26.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:42:26.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607793+0000", 2024-09-17T22:42:26.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8529999999999997e-05 2024-09-17T22:42:26.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607837+0000", 2024-09-17T22:42:26.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3526000000000003e-05 2024-09-17T22:42:26.148 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.148 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.148 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.149 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186749 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 11960320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:26.149 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:26.149 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.714864505000001, 2024-09-17T22:42:26.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029226166000000001, 2024-09-17T22:42:26.149 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.149 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.149 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.149 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.149 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.149 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.150 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186749 2024-09-17T22:42:26.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.150 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:26.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:26.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:26.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-09-17T22:42:26.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423328+0000", 2024-09-17T22:42:26.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6790000000000005e-06 2024-09-17T22:42:26.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423334+0000", 2024-09-17T22:42:26.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0009999999999998e-06 2024-09-17T22:42:26.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423338+0000", 2024-09-17T22:42:26.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6419999999999998e-06 2024-09-17T22:42:26.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452464+0000", 2024-09-17T22:42:26.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029125389000000002 2024-09-17T22:42:26.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452501+0000", 2024-09-17T22:42:26.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7656e-05 2024-09-17T22:42:26.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452546+0000", 2024-09-17T22:42:26.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4468999999999997e-05 2024-09-17T22:42:26.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.153 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.153 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186750 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12050432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:26.154 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:26.154 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.714838714999999, 2024-09-17T22:42:26.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029307863999999999, 2024-09-17T22:42:26.154 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.154 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.154 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.154 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186750 2024-09-17T22:42:26.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.154 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:26.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:26.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423346+0000", 2024-09-17T22:42:26.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.84e-07 2024-09-17T22:42:26.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423348+0000", 2024-09-17T22:42:26.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7540000000000001e-06 2024-09-17T22:42:26.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423353+0000", 2024-09-17T22:42:26.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:42:26.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423362+0000", 2024-09-17T22:42:26.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9220000000000003e-06 2024-09-17T22:42:26.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452571+0000", 2024-09-17T22:42:26.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029209174000000001 2024-09-17T22:42:26.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452612+0000", 2024-09-17T22:42:26.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1363999999999999e-05 2024-09-17T22:42:26.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452653+0000", 2024-09-17T22:42:26.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0893999999999998e-05 2024-09-17T22:42:26.158 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.158 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.158 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.158 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186751 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12140544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:26.159 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:26.159 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.714816225, 2024-09-17T22:42:26.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029387488, 2024-09-17T22:42:26.159 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.159 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.159 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.159 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.159 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.159 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.159 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186751 2024-09-17T22:42:26.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.159 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:26.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:26.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:26.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1399999999999997e-07 2024-09-17T22:42:26.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423375+0000", 2024-09-17T22:42:26.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8889999999999996e-06 2024-09-17T22:42:26.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423381+0000", 2024-09-17T22:42:26.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5690000000000004e-06 2024-09-17T22:42:26.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423386+0000", 2024-09-17T22:42:26.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9559999999999996e-06 2024-09-17T22:42:26.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452679+0000", 2024-09-17T22:42:26.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029292820000000001 2024-09-17T22:42:26.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452718+0000", 2024-09-17T22:42:26.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8961000000000002e-05 2024-09-17T22:42:26.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452755+0000", 2024-09-17T22:42:26.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7879000000000001e-05 2024-09-17T22:42:26.163 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.163 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.163 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.163 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186752 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12230656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:26.163 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.714784974000001, 2024-09-17T22:42:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029463269, 2024-09-17T22:42:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186752 2024-09-17T22:42:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:26.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:26.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423400+0000", 2024-09-17T22:42:26.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:42:26.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423401+0000", 2024-09-17T22:42:26.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3599999999999999e-06 2024-09-17T22:42:26.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423404+0000", 2024-09-17T22:42:26.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5999999999999998e-06 2024-09-17T22:42:26.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423407+0000", 2024-09-17T22:42:26.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6869999999999999e-06 2024-09-17T22:42:26.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452782+0000", 2024-09-17T22:42:26.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029375107000000001 2024-09-17T22:42:26.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452821+0000", 2024-09-17T22:42:26.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9122999999999998e-05 2024-09-17T22:42:26.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452862+0000", 2024-09-17T22:42:26.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1019000000000001e-05 2024-09-17T22:42:26.168 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.168 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.168 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.168 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186753 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:26.169 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:26.169 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.714772952000001, 2024-09-17T22:42:26.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029553297999999999, 2024-09-17T22:42:26.169 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.169 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.169 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.169 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.169 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.169 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.169 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186753 2024-09-17T22:42:26.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.169 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:26.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:26.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:26.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.98e-07 2024-09-17T22:42:26.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423412+0000", 2024-09-17T22:42:26.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8700000000000004e-07 2024-09-17T22:42:26.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423415+0000", 2024-09-17T22:42:26.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:42:26.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423418+0000", 2024-09-17T22:42:26.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.283e-06 2024-09-17T22:42:26.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452893+0000", 2024-09-17T22:42:26.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029475881999999998 2024-09-17T22:42:26.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452928+0000", 2024-09-17T22:42:26.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4109000000000002e-05 2024-09-17T22:42:26.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452964+0000", 2024-09-17T22:42:26.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6937000000000002e-05 2024-09-17T22:42:26.173 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.173 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.173 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.173 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186754 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12410880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:26.174 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:26.174 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.714763815000001, 2024-09-17T22:42:26.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029656021000000001, 2024-09-17T22:42:26.174 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.174 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.174 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.174 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.174 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.174 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.174 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186754 2024-09-17T22:42:26.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.174 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:26.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:26.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423421+0000", 2024-09-17T22:42:26.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9999999999999999e-07 2024-09-17T22:42:26.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423425+0000", 2024-09-17T22:42:26.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6519999999999997e-06 2024-09-17T22:42:26.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423428+0000", 2024-09-17T22:42:26.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1370000000000002e-06 2024-09-17T22:42:26.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423431+0000", 2024-09-17T22:42:26.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4090000000000001e-06 2024-09-17T22:42:26.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452992+0000", 2024-09-17T22:42:26.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029561545000000002 2024-09-17T22:42:26.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453031+0000", 2024-09-17T22:42:26.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8834000000000001e-05 2024-09-17T22:42:26.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453076+0000", 2024-09-17T22:42:26.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5244000000000003e-05 2024-09-17T22:42:26.178 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.178 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.178 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.179 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186755 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12500992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:26.179 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:26.179 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.714750534, 2024-09-17T22:42:26.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029745649999999998, 2024-09-17T22:42:26.179 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.179 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.179 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.179 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186755 2024-09-17T22:42:26.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.180 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:26.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:26.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:26.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.17e-07 2024-09-17T22:42:26.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423435+0000", 2024-09-17T22:42:26.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:42:26.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423438+0000", 2024-09-17T22:42:26.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8210000000000002e-06 2024-09-17T22:42:26.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423443+0000", 2024-09-17T22:42:26.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1070000000000004e-06 2024-09-17T22:42:26.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453103+0000", 2024-09-17T22:42:26.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029660399 2024-09-17T22:42:26.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453141+0000", 2024-09-17T22:42:26.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.748e-05 2024-09-17T22:42:26.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453179+0000", 2024-09-17T22:42:26.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8671000000000003e-05 2024-09-17T22:42:26.183 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.183 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.183 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.183 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187439 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8437760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:26.184 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:26.184 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.597313437, 2024-09-17T22:42:26.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029278048000000001, 2024-09-17T22:42:26.184 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.184 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.184 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.184 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187439 2024-09-17T22:42:26.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.185 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:26.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:26.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:26.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6199999999999999e-07 2024-09-17T22:42:26.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540872+0000", 2024-09-17T22:42:26.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.09e-07 2024-09-17T22:42:26.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540875+0000", 2024-09-17T22:42:26.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8990000000000001e-06 2024-09-17T22:42:26.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540877+0000", 2024-09-17T22:42:26.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1789999999999998e-06 2024-09-17T22:42:26.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570074+0000", 2024-09-17T22:42:26.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029197358 2024-09-17T22:42:26.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570111+0000", 2024-09-17T22:42:26.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6170000000000001e-05 2024-09-17T22:42:26.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570149+0000", 2024-09-17T22:42:26.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8170999999999998e-05 2024-09-17T22:42:26.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.188 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.188 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187440 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8527872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:26.188 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:26.189 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.597297966999999, 2024-09-17T22:42:26.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029359758999999999, 2024-09-17T22:42:26.189 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.189 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.189 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.189 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.189 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.189 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.190 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187440 2024-09-17T22:42:26.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.190 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:26.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:26.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:26.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2700000000000001e-07 2024-09-17T22:42:26.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540888+0000", 2024-09-17T22:42:26.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-17T22:42:26.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540893+0000", 2024-09-17T22:42:26.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9110000000000001e-06 2024-09-17T22:42:26.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540895+0000", 2024-09-17T22:42:26.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.712e-06 2024-09-17T22:42:26.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570168+0000", 2024-09-17T22:42:26.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029273114999999999 2024-09-17T22:42:26.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570208+0000", 2024-09-17T22:42:26.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9227999999999998e-05 2024-09-17T22:42:26.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570246+0000", 2024-09-17T22:42:26.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8253000000000001e-05 2024-09-17T22:42:26.194 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.194 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.194 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.194 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187441 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8617984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:26.194 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:26.194 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.597285003, 2024-09-17T22:42:26.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029442211999999999, 2024-09-17T22:42:26.194 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.194 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.194 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.194 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.195 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.195 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.195 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187441 2024-09-17T22:42:26.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.195 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:26.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:26.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:26.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:42:26.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540900+0000", 2024-09-17T22:42:26.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7999999999999993e-07 2024-09-17T22:42:26.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540903+0000", 2024-09-17T22:42:26.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9129999999999999e-06 2024-09-17T22:42:26.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540909+0000", 2024-09-17T22:42:26.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4269999999999999e-06 2024-09-17T22:42:26.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570267+0000", 2024-09-17T22:42:26.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029358308 2024-09-17T22:42:26.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570304+0000", 2024-09-17T22:42:26.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6860999999999998e-05 2024-09-17T22:42:26.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570341+0000", 2024-09-17T22:42:26.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7509000000000003e-05 2024-09-17T22:42:26.199 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.199 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.199 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.199 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187442 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8708096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:26.199 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:26.199 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.597272296, 2024-09-17T22:42:26.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029522606999999999, 2024-09-17T22:42:26.199 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.199 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.199 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.199 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.199 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.200 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.200 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187442 2024-09-17T22:42:26.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.200 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:26.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:26.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:26.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.22e-07 2024-09-17T22:42:26.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540917+0000", 2024-09-17T22:42:26.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999998e-06 2024-09-17T22:42:26.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540920+0000", 2024-09-17T22:42:26.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4989999999999999e-06 2024-09-17T22:42:26.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540923+0000", 2024-09-17T22:42:26.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3870000000000002e-06 2024-09-17T22:42:26.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570361+0000", 2024-09-17T22:42:26.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029438345000000001 2024-09-17T22:42:26.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570397+0000", 2024-09-17T22:42:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5429000000000002e-05 2024-09-17T22:42:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570434+0000", 2024-09-17T22:42:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7824999999999998e-05 2024-09-17T22:42:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187443 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8798208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.597258368999999, 2024-09-17T22:42:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029603640000000001, 2024-09-17T22:42:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187443 2024-09-17T22:42:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.05e-07 2024-09-17T22:42:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540927+0000", 2024-09-17T22:42:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:42:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540930+0000", 2024-09-17T22:42:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9229999999999998e-06 2024-09-17T22:42:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540932+0000", 2024-09-17T22:42:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.294e-06 2024-09-17T22:42:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570454+0000", 2024-09-17T22:42:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029522230999999999 2024-09-17T22:42:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570491+0000", 2024-09-17T22:42:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6835999999999998e-05 2024-09-17T22:42:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570529+0000", 2024-09-17T22:42:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8167999999999998e-05 2024-09-17T22:42:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187444 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8888320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.597248895, 2024-09-17T22:42:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029687584999999999, 2024-09-17T22:42:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187444 2024-09-17T22:42:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-09-17T22:42:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540936+0000", 2024-09-17T22:42:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3900000000000003e-07 2024-09-17T22:42:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540939+0000", 2024-09-17T22:42:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.994e-06 2024-09-17T22:42:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540942+0000", 2024-09-17T22:42:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1739999999999999e-06 2024-09-17T22:42:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570549+0000", 2024-09-17T22:42:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029607652000000002 2024-09-17T22:42:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570585+0000", 2024-09-17T22:42:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5775000000000002e-05 2024-09-17T22:42:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570623+0000", 2024-09-17T22:42:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7839999999999997e-05 2024-09-17T22:42:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187445 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8978432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.597240007, 2024-09-17T22:42:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029772837999999999, 2024-09-17T22:42:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187445 2024-09-17T22:42:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:42:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540945+0000", 2024-09-17T22:42:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3699999999999999e-07 2024-09-17T22:42:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540948+0000", 2024-09-17T22:42:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.898e-06 2024-09-17T22:42:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540950+0000", 2024-09-17T22:42:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2249999999999999e-06 2024-09-17T22:42:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570643+0000", 2024-09-17T22:42:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029692463999999998 2024-09-17T22:42:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570680+0000", 2024-09-17T22:42:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6772e-05 2024-09-17T22:42:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570717+0000", 2024-09-17T22:42:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7438999999999998e-05 2024-09-17T22:42:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187446 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9068544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.597231269000002, 2024-09-17T22:42:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029856321000000002, 2024-09-17T22:42:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187446 2024-09-17T22:42:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:42:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540958+0000", 2024-09-17T22:42:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3449999999999996e-06 2024-09-17T22:42:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540962+0000", 2024-09-17T22:42:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3529999999999999e-06 2024-09-17T22:42:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540964+0000", 2024-09-17T22:42:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3810000000000002e-06 2024-09-17T22:42:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570737+0000", 2024-09-17T22:42:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029772824 2024-09-17T22:42:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570770+0000", 2024-09-17T22:42:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2835e-05 2024-09-17T22:42:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570809+0000", 2024-09-17T22:42:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9379999999999999e-05 2024-09-17T22:42:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187447 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9158656~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.597217301000001, 2024-09-17T22:42:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02993262, 2024-09-17T22:42:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.223 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.223 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.223 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187447 2024-09-17T22:42:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-09-17T22:42:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540968+0000", 2024-09-17T22:42:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6500000000000008e-07 2024-09-17T22:42:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540971+0000", 2024-09-17T22:42:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9869999999999999e-06 2024-09-17T22:42:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540973+0000", 2024-09-17T22:42:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3120000000000001e-06 2024-09-17T22:42:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570829+0000", 2024-09-17T22:42:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029855742000000001 2024-09-17T22:42:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570861+0000", 2024-09-17T22:42:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2152e-05 2024-09-17T22:42:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570899+0000", 2024-09-17T22:42:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8241999999999997e-05 2024-09-17T22:42:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187448 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9248768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.597208271, 2024-09-17T22:42:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030014865000000002, 2024-09-17T22:42:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187448 2024-09-17T22:42:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9600000000000001e-07 2024-09-17T22:42:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540977+0000", 2024-09-17T22:42:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.16e-07 2024-09-17T22:42:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540980+0000", 2024-09-17T22:42:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.892e-06 2024-09-17T22:42:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540982+0000", 2024-09-17T22:42:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3410000000000001e-06 2024-09-17T22:42:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570919+0000", 2024-09-17T22:42:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029936800999999999 2024-09-17T22:42:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570953+0000", 2024-09-17T22:42:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3939000000000002e-05 2024-09-17T22:42:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570991+0000", 2024-09-17T22:42:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7780000000000001e-05 2024-09-17T22:42:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187449 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9338880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.597199092, 2024-09-17T22:42:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030110238000000001, 2024-09-17T22:42:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187449 2024-09-17T22:42:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1500000000000001e-07 2024-09-17T22:42:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540986+0000", 2024-09-17T22:42:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9500000000000001e-07 2024-09-17T22:42:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540989+0000", 2024-09-17T22:42:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7949999999999999e-06 2024-09-17T22:42:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540994+0000", 2024-09-17T22:42:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0359999999999997e-06 2024-09-17T22:42:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571011+0000", 2024-09-17T22:42:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030016915000000002 2024-09-17T22:42:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571056+0000", 2024-09-17T22:42:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5457999999999997e-05 2024-09-17T22:42:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571095+0000", 2024-09-17T22:42:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8924000000000002e-05 2024-09-17T22:42:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187450 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9428992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.597187444999999, 2024-09-17T22:42:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030188204999999999, 2024-09-17T22:42:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187450 2024-09-17T22:42:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.16e-07 2024-09-17T22:42:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540998+0000", 2024-09-17T22:42:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9000000000000005e-07 2024-09-17T22:42:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.541001+0000", 2024-09-17T22:42:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0680000000000001e-06 2024-09-17T22:42:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.541006+0000", 2024-09-17T22:42:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4720000000000003e-06 2024-09-17T22:42:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571110+0000", 2024-09-17T22:42:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030103431 2024-09-17T22:42:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571147+0000", 2024-09-17T22:42:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6748999999999997e-05 2024-09-17T22:42:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571185+0000", 2024-09-17T22:42:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8278999999999997e-05 2024-09-17T22:42:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.243 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:26.244 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:42:26.245 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:26.242+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:26.346 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:26.343+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:26.356 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:26.356 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:26.356 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:26.356 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:26.366 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:42:26.446 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:26.444+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:26.473 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-17T22:42:26.475 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:26.476 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:26.476 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:26.476 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:26.476 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:26.488 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:42:26.547 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:26.545+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:26.603 INFO:teuthology.orchestra.run.smithi115.stdout:4703 read 0x71d07c8 thru 0x71d0dbb (0x5f4 bytes) 2024-09-17T22:42:26.605 INFO:teuthology.orchestra.run.smithi115.stdout:4704 writesame 0xbfe778a thru 0xbff5940 data_size 0x33(0xe1b7 bytes) 2024-09-17T22:42:26.616 INFO:teuthology.orchestra.run.smithi115.stdout:4705 trunc from 0xe4d4322 to 0xb8a9f03 2024-09-17T22:42:26.631 INFO:teuthology.orchestra.run.smithi115.stdout:4706 write 0x781b57f thru 0x7822aaa (0x752c bytes) 2024-09-17T22:42:26.635 INFO:teuthology.orchestra.run.smithi115.stdout:4707 punch from 0xaadb5e to 0xab6bbc, (0x905e bytes) 2024-09-17T22:42:26.635 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:26.641 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:26.641 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:26.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:26.641 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:26.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.641 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145246 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3850240~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.641 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:26.641 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.307588420000002, 2024-09-17T22:42:26.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052644991000000002, 2024-09-17T22:42:26.641 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.641 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.641 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.642 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145246 2024-09-17T22:42:26.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.642 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:26.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:26.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329899+0000", 2024-09-17T22:42:26.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-09-17T22:42:26.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329901+0000", 2024-09-17T22:42:26.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1619999999999998e-06 2024-09-17T22:42:26.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329907+0000", 2024-09-17T22:42:26.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9059999999999999e-06 2024-09-17T22:42:26.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329911+0000", 2024-09-17T22:42:26.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7419999999999997e-06 2024-09-17T22:42:26.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382383+0000", 2024-09-17T22:42:26.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052471583000000002 2024-09-17T22:42:26.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382457+0000", 2024-09-17T22:42:26.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4309999999999998e-05 2024-09-17T22:42:26.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382543+0000", 2024-09-17T22:42:26.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5918e-05 2024-09-17T22:42:26.645 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.645 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.645 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.646 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145247 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3946496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.646 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:26.646 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.307569770000001, 2024-09-17T22:42:26.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052805863000000001, 2024-09-17T22:42:26.646 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.646 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.646 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.646 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145247 2024-09-17T22:42:26.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.647 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:26.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:26.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:26.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3200000000000001e-07 2024-09-17T22:42:26.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329919+0000", 2024-09-17T22:42:26.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.888e-06 2024-09-17T22:42:26.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329925+0000", 2024-09-17T22:42:26.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5369999999999997e-06 2024-09-17T22:42:26.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329929+0000", 2024-09-17T22:42:26.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.639e-06 2024-09-17T22:42:26.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382570+0000", 2024-09-17T22:42:26.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052641213999999999 2024-09-17T22:42:26.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382636+0000", 2024-09-17T22:42:26.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6008999999999998e-05 2024-09-17T22:42:26.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382723+0000", 2024-09-17T22:42:26.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6243999999999996e-05 2024-09-17T22:42:26.650 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.650 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.650 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.650 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145248 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3856384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.651 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:26.651 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.307551721999999, 2024-09-17T22:42:26.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053002858, 2024-09-17T22:42:26.651 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.651 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.651 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.651 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.651 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.651 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.651 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145248 2024-09-17T22:42:26.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.651 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:26.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:26.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:26.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.13e-07 2024-09-17T22:42:26.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329937+0000", 2024-09-17T22:42:26.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9300000000000002e-06 2024-09-17T22:42:26.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329943+0000", 2024-09-17T22:42:26.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-09-17T22:42:26.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329948+0000", 2024-09-17T22:42:26.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8579999999999999e-06 2024-09-17T22:42:26.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382763+0000", 2024-09-17T22:42:26.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052814537000000002 2024-09-17T22:42:26.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382840+0000", 2024-09-17T22:42:26.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6879999999999996e-05 2024-09-17T22:42:26.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382938+0000", 2024-09-17T22:42:26.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8159999999999995e-05 2024-09-17T22:42:26.655 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.655 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.655 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.655 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145249 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3952640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.655 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:26.655 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.307532821000002, 2024-09-17T22:42:26.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053188602000000001, 2024-09-17T22:42:26.656 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.656 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.656 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.656 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.656 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.656 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.656 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145249 2024-09-17T22:42:26.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.656 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:26.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:26.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:26.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3700000000000001e-07 2024-09-17T22:42:26.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329956+0000", 2024-09-17T22:42:26.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1430000000000001e-06 2024-09-17T22:42:26.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329962+0000", 2024-09-17T22:42:26.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8270000000000003e-06 2024-09-17T22:42:26.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329967+0000", 2024-09-17T22:42:26.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7249999999999997e-06 2024-09-17T22:42:26.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382969+0000", 2024-09-17T22:42:26.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053001952999999997 2024-09-17T22:42:26.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383040+0000", 2024-09-17T22:42:26.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1272999999999993e-05 2024-09-17T22:42:26.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383142+0000", 2024-09-17T22:42:26.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010234399999999999 2024-09-17T22:42:26.660 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.660 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.660 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.660 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145250 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3862528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.660 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:26.660 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.307514021999999, 2024-09-17T22:42:26.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053359537999999998, 2024-09-17T22:42:26.660 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.661 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.661 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.661 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145250 2024-09-17T22:42:26.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.661 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:26.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:26.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:26.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:42:26.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329975+0000", 2024-09-17T22:42:26.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2199999999999999e-06 2024-09-17T22:42:26.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329981+0000", 2024-09-17T22:42:26.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1020000000000002e-06 2024-09-17T22:42:26.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329991+0000", 2024-09-17T22:42:26.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6800000000000005e-06 2024-09-17T22:42:26.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383171+0000", 2024-09-17T22:42:26.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053180142 2024-09-17T22:42:26.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383242+0000", 2024-09-17T22:42:26.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1310000000000007e-05 2024-09-17T22:42:26.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383332+0000", 2024-09-17T22:42:26.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9703999999999994e-05 2024-09-17T22:42:26.665 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.665 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.665 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.665 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145251 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3958784~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.665 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:26.665 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.307488321999998, 2024-09-17T22:42:26.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053438537000000001, 2024-09-17T22:42:26.665 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.665 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.665 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.666 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145251 2024-09-17T22:42:26.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.666 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:26.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:26.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329999+0000", 2024-09-17T22:42:26.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-07 2024-09-17T22:42:26.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330001+0000", 2024-09-17T22:42:26.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:42:26.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330006+0000", 2024-09-17T22:42:26.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000004e-06 2024-09-17T22:42:26.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330011+0000", 2024-09-17T22:42:26.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4039999999999997e-06 2024-09-17T22:42:26.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383350+0000", 2024-09-17T22:42:26.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053339011999999998 2024-09-17T22:42:26.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383387+0000", 2024-09-17T22:42:26.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7085999999999997e-05 2024-09-17T22:42:26.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383437+0000", 2024-09-17T22:42:26.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9988000000000003e-05 2024-09-17T22:42:26.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.669 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.670 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145252 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3868672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.670 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:26.670 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.307469875000002, 2024-09-17T22:42:26.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053526724999999997, 2024-09-17T22:42:26.670 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.670 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.670 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.670 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145252 2024-09-17T22:42:26.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.671 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:26.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:26.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331054+0000", 2024-09-17T22:42:26.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010371670000000001 2024-09-17T22:42:26.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331059+0000", 2024-09-17T22:42:26.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1680000000000003e-06 2024-09-17T22:42:26.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331075+0000", 2024-09-17T22:42:26.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6028e-05 2024-09-17T22:42:26.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331084+0000", 2024-09-17T22:42:26.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3980000000000005e-06 2024-09-17T22:42:26.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383464+0000", 2024-09-17T22:42:26.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052379851999999998 2024-09-17T22:42:26.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383499+0000", 2024-09-17T22:42:26.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4610000000000002e-05 2024-09-17T22:42:26.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383543+0000", 2024-09-17T22:42:26.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4502000000000002e-05 2024-09-17T22:42:26.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.674 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.674 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145255 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3971072~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.675 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:26.675 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.306339782999999, 2024-09-17T22:42:26.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052677527000000002, 2024-09-17T22:42:26.675 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.675 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.675 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.675 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145255 2024-09-17T22:42:26.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.675 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:26.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:26.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:26.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8800000000000002e-07 2024-09-17T22:42:26.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331150+0000", 2024-09-17T22:42:26.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1160000000000002e-06 2024-09-17T22:42:26.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331156+0000", 2024-09-17T22:42:26.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7469999999999999e-06 2024-09-17T22:42:26.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331162+0000", 2024-09-17T22:42:26.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-17T22:42:26.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383750+0000", 2024-09-17T22:42:26.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052588441999999999 2024-09-17T22:42:26.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383782+0000", 2024-09-17T22:42:26.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1610999999999999e-05 2024-09-17T22:42:26.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383824+0000", 2024-09-17T22:42:26.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2512000000000002e-05 2024-09-17T22:42:26.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.679 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.679 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145256 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3880960~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.679 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:26.679 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.306318361999999, 2024-09-17T22:42:26.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052744775000000001, 2024-09-17T22:42:26.679 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.679 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.679 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.680 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145256 2024-09-17T22:42:26.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.680 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:26.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:26.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331169+0000", 2024-09-17T22:42:26.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:42:26.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331175+0000", 2024-09-17T22:42:26.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5660000000000003e-06 2024-09-17T22:42:26.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331182+0000", 2024-09-17T22:42:26.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0169999999999999e-06 2024-09-17T22:42:26.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331188+0000", 2024-09-17T22:42:26.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2750000000000003e-06 2024-09-17T22:42:26.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383839+0000", 2024-09-17T22:42:26.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052651728000000002 2024-09-17T22:42:26.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383872+0000", 2024-09-17T22:42:26.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2347e-05 2024-09-17T22:42:26.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383913+0000", 2024-09-17T22:42:26.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1362000000000001e-05 2024-09-17T22:42:26.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.683 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.684 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145257 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3977216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.684 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:26.684 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.306292313999997, 2024-09-17T22:42:26.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052809598999999999, 2024-09-17T22:42:26.684 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.684 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.684 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.684 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145257 2024-09-17T22:42:26.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.685 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:26.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:26.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333051+0000", 2024-09-17T22:42:26.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018567189999999999 2024-09-17T22:42:26.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333056+0000", 2024-09-17T22:42:26.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-17T22:42:26.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333067+0000", 2024-09-17T22:42:26.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1704e-05 2024-09-17T22:42:26.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333074+0000", 2024-09-17T22:42:26.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1280000000000002e-06 2024-09-17T22:42:26.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383927+0000", 2024-09-17T22:42:26.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050853094000000001 2024-09-17T22:42:26.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383960+0000", 2024-09-17T22:42:26.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2891000000000001e-05 2024-09-17T22:42:26.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.384004+0000", 2024-09-17T22:42:26.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3565e-05 2024-09-17T22:42:26.688 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.688 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.688 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.688 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146168 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2467840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.688 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:26.689 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.242803692999999, 2024-09-17T22:42:26.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052652501999999997, 2024-09-17T22:42:26.689 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.689 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.689 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.689 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.689 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.689 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.689 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146168 2024-09-17T22:42:26.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.689 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:26.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:26.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394989+0000", 2024-09-17T22:42:26.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030618300000000002 2024-09-17T22:42:26.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394994+0000", 2024-09-17T22:42:26.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.566e-06 2024-09-17T22:42:26.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395003+0000", 2024-09-17T22:42:26.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3619999999999995e-06 2024-09-17T22:42:26.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395009+0000", 2024-09-17T22:42:26.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3589999999999996e-06 2024-09-17T22:42:26.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447180+0000", 2024-09-17T22:42:26.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052170643000000003 2024-09-17T22:42:26.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447251+0000", 2024-09-17T22:42:26.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1382000000000002e-05 2024-09-17T22:42:26.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447335+0000", 2024-09-17T22:42:26.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4006999999999997e-05 2024-09-17T22:42:26.693 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.693 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.693 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.693 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146172 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2480128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.693 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:26.693 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.242448807999999, 2024-09-17T22:42:26.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052714629999999998, 2024-09-17T22:42:26.693 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.693 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146172 2024-09-17T22:42:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395486+0000", 2024-09-17T22:42:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044787000000000001 2024-09-17T22:42:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395491+0000", 2024-09-17T22:42:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8550000000000001e-06 2024-09-17T22:42:26.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395503+0000", 2024-09-17T22:42:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2271e-05 2024-09-17T22:42:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395510+0000", 2024-09-17T22:42:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6320000000000008e-06 2024-09-17T22:42:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447565+0000", 2024-09-17T22:42:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052055016000000003 2024-09-17T22:42:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447644+0000", 2024-09-17T22:42:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8306000000000005e-05 2024-09-17T22:42:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447752+0000", 2024-09-17T22:42:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010868 2024-09-17T22:42:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.697 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146181 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2504704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.241581418000003, 2024-09-17T22:42:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052850394000000002, 2024-09-17T22:42:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.698 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146181 2024-09-17T22:42:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396210+0000", 2024-09-17T22:42:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000305214 2024-09-17T22:42:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396214+0000", 2024-09-17T22:42:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9389999999999999e-06 2024-09-17T22:42:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396228+0000", 2024-09-17T22:42:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3365e-05 2024-09-17T22:42:26.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396235+0000", 2024-09-17T22:42:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2559999999999996e-06 2024-09-17T22:42:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448568+0000", 2024-09-17T22:42:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052333084000000002 2024-09-17T22:42:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448646+0000", 2024-09-17T22:42:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8287000000000004e-05 2024-09-17T22:42:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448756+0000", 2024-09-17T22:42:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109249 2024-09-17T22:42:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146183 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2510848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.241240871000002, 2024-09-17T22:42:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052725453999999998, 2024-09-17T22:42:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146183 2024-09-17T22:42:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396508+0000", 2024-09-17T22:42:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026271299999999998 2024-09-17T22:42:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396512+0000", 2024-09-17T22:42:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8039999999999999e-06 2024-09-17T22:42:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396522+0000", 2024-09-17T22:42:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9289999999999993e-06 2024-09-17T22:42:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396528+0000", 2024-09-17T22:42:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2990000000000002e-06 2024-09-17T22:42:26.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448798+0000", 2024-09-17T22:42:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052269985999999997 2024-09-17T22:42:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448871+0000", 2024-09-17T22:42:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2581000000000002e-05 2024-09-17T22:42:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448971+0000", 2024-09-17T22:42:26.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100142 2024-09-17T22:42:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146391 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.233513379000001, 2024-09-17T22:42:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052666719000000001, 2024-09-17T22:42:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146391 2024-09-17T22:42:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404098+0000", 2024-09-17T22:42:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000124509 2024-09-17T22:42:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404107+0000", 2024-09-17T22:42:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4220000000000006e-06 2024-09-17T22:42:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404117+0000", 2024-09-17T22:42:26.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011e-05 2024-09-17T22:42:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404123+0000", 2024-09-17T22:42:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1530000000000002e-06 2024-09-17T22:42:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456481+0000", 2024-09-17T22:42:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052357923000000001 2024-09-17T22:42:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456551+0000", 2024-09-17T22:42:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9869000000000005e-05 2024-09-17T22:42:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456640+0000", 2024-09-17T22:42:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8733000000000006e-05 2024-09-17T22:42:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146393 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2748416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:26.711 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.233353315999999, 2024-09-17T22:42:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052739239, 2024-09-17T22:42:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146393 2024-09-17T22:42:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:26.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404443+0000", 2024-09-17T22:42:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030977199999999999 2024-09-17T22:42:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404447+0000", 2024-09-17T22:42:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6370000000000001e-06 2024-09-17T22:42:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404469+0000", 2024-09-17T22:42:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2739999999999999e-05 2024-09-17T22:42:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404477+0000", 2024-09-17T22:42:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6559999999999992e-06 2024-09-17T22:42:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456734+0000", 2024-09-17T22:42:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052256889000000001 2024-09-17T22:42:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456796+0000", 2024-09-17T22:42:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2485999999999998e-05 2024-09-17T22:42:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456873+0000", 2024-09-17T22:42:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6058999999999998e-05 2024-09-17T22:42:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.715 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146397 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2760704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.232977552000001, 2024-09-17T22:42:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052788928999999998, 2024-09-17T22:42:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146397 2024-09-17T22:42:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404763+0000", 2024-09-17T22:42:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025351399999999999 2024-09-17T22:42:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404767+0000", 2024-09-17T22:42:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:42:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404777+0000", 2024-09-17T22:42:26.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9310000000000003e-06 2024-09-17T22:42:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404783+0000", 2024-09-17T22:42:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.336e-06 2024-09-17T22:42:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457126+0000", 2024-09-17T22:42:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052342975999999999 2024-09-17T22:42:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457200+0000", 2024-09-17T22:42:26.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4146000000000005e-05 2024-09-17T22:42:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457298+0000", 2024-09-17T22:42:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7807999999999996e-05 2024-09-17T22:42:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146399 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2766848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:26.720 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.232687411000001, 2024-09-17T22:42:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052829779, 2024-09-17T22:42:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146399 2024-09-17T22:42:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:26.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404800+0000", 2024-09-17T22:42:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8500000000000001e-07 2024-09-17T22:42:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404802+0000", 2024-09-17T22:42:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5349999999999999e-06 2024-09-17T22:42:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404811+0000", 2024-09-17T22:42:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9390000000000003e-06 2024-09-17T22:42:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404817+0000", 2024-09-17T22:42:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.524e-06 2024-09-17T22:42:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457461+0000", 2024-09-17T22:42:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052643784999999998 2024-09-17T22:42:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457539+0000", 2024-09-17T22:42:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8589000000000003e-05 2024-09-17T22:42:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457629+0000", 2024-09-17T22:42:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9821999999999995e-05 2024-09-17T22:42:26.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146401 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2772992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.232662163999997, 2024-09-17T22:42:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052910077, 2024-09-17T22:42:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.725 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146401 2024-09-17T22:42:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405106+0000", 2024-09-17T22:42:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000281286 2024-09-17T22:42:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405110+0000", 2024-09-17T22:42:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1459999999999998e-06 2024-09-17T22:42:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405120+0000", 2024-09-17T22:42:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0546e-05 2024-09-17T22:42:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405127+0000", 2024-09-17T22:42:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4300000000000003e-06 2024-09-17T22:42:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457647+0000", 2024-09-17T22:42:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052520229000000002 2024-09-17T22:42:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457684+0000", 2024-09-17T22:42:26.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6609999999999997e-05 2024-09-17T22:42:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457735+0000", 2024-09-17T22:42:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.083e-05 2024-09-17T22:42:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146402 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2779136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:26.729 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.232348752, 2024-09-17T22:42:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052689812000000003, 2024-09-17T22:42:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146402 2024-09-17T22:42:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:26.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405365+0000", 2024-09-17T22:42:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022744599999999999 2024-09-17T22:42:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405370+0000", 2024-09-17T22:42:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2769999999999999e-06 2024-09-17T22:42:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405380+0000", 2024-09-17T22:42:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0349e-05 2024-09-17T22:42:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405386+0000", 2024-09-17T22:42:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5169999999999996e-06 2024-09-17T22:42:26.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457750+0000", 2024-09-17T22:42:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052363448999999999 2024-09-17T22:42:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457784+0000", 2024-09-17T22:42:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4363999999999998e-05 2024-09-17T22:42:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457828+0000", 2024-09-17T22:42:26.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999999e-05 2024-09-17T22:42:26.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.734 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:26.734 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:26.734 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:26.735 INFO:teuthology.orchestra.run.smithi115.stdout:4711 write 0xd4ab9fa thru 0xd4b6a26 (0xb02d bytes) 2024-09-17T22:42:26.736 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:42:26.737 INFO:teuthology.orchestra.run.smithi115.stdout:4712 write 0xe068be6 thru 0xe06c04e (0x3469 bytes) 2024-09-17T22:42:26.737 INFO:teuthology.orchestra.run.smithi115.stdout:4714 write 0x536e9d7 thru 0x5370b0c (0x2136 bytes) 2024-09-17T22:42:26.737 INFO:teuthology.orchestra.run.smithi115.stdout:4715 writesame 0x804ef5 thru 0x809d63 data_size 0x45(0x4e6f bytes) 2024-09-17T22:42:26.737 INFO:teuthology.orchestra.run.smithi115.stdout:4716 read 0x344828 thru 0x347f44 (0x371d bytes) 2024-09-17T22:42:26.737 INFO:teuthology.orchestra.run.smithi115.stdout:4717 trunc from 0xe06c04f to 0xe50160f 2024-09-17T22:42:26.738 INFO:teuthology.orchestra.run.smithi115.stdout:4719 punch from 0x94385c2 to 0x943ea4f, (0x648d bytes) 2024-09-17T22:42:26.738 INFO:teuthology.orchestra.run.smithi115.stdout:4720 trunc from 0xe50160f to 0x96f9688 2024-09-17T22:42:26.738 INFO:teuthology.orchestra.run.smithi115.stdout:4722 punch from 0x544b81b to 0x544df76, (0x275b bytes) 2024-09-17T22:42:26.738 INFO:teuthology.orchestra.run.smithi115.stdout:4724 trunc from 0x96f9688 to 0x8505c87 2024-09-17T22:42:26.738 INFO:teuthology.orchestra.run.smithi115.stdout:4727 read 0x21f0362 thru 0x21f4fa5 (0x4c44 bytes) 2024-09-17T22:42:26.738 INFO:teuthology.orchestra.run.smithi115.stdout:4728 writesame 0x5421e0b thru 0x542e242 data_size 0xb6(0xc438 bytes) 2024-09-17T22:42:26.738 INFO:teuthology.orchestra.run.smithi115.stdout:4729 writesame 0x3068eb thru 0x313596 data_size 0x2(0xccac bytes) 2024-09-17T22:42:26.738 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:26.733+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:26.761 INFO:teuthology.orchestra.run.smithi115.stdout:4730 write 0xb4b54bc thru 0xb4b93c5 (0x3f0a bytes) 2024-09-17T22:42:26.764 INFO:teuthology.orchestra.run.smithi115.stdout:4731 read 0xa019a38 thru 0xa026e4e (0xd417 bytes) 2024-09-17T22:42:26.764 INFO:teuthology.orchestra.run.smithi115.stdout:4732 punch from 0xaf92290 to 0xaf9a1ac, (0x7f1c bytes) 2024-09-17T22:42:26.775 INFO:teuthology.orchestra.run.smithi115.stdout:4734 trunc from 0xb4b93c6 to 0x22548af 2024-09-17T22:42:26.792 INFO:teuthology.orchestra.run.smithi115.stdout:4736 punch from 0x5f5d7d to 0x5fbb0b, (0x5d8e bytes) 2024-09-17T22:42:26.803 INFO:teuthology.orchestra.run.smithi115.stdout:4737 write 0x5ff526a thru 0x5ffd979 (0x8710 bytes) 2024-09-17T22:42:26.804 INFO:teuthology.orchestra.run.smithi115.stdout:4739 writesame 0x1e4f21e thru 0x1e5a3b4 data_size 0x1(0xb197 bytes) 2024-09-17T22:42:26.819 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:26.819 INFO:teuthology.orchestra.run.smithi115.stdout:4741 write 0x4ac9bdd thru 0x4ad6e1a (0xd23e bytes) 2024-09-17T22:42:26.820 INFO:teuthology.orchestra.run.smithi115.stdout:4742 read 0x1afedd4 thru 0x1b0c2fe (0xd52b bytes) 2024-09-17T22:42:26.820 INFO:teuthology.orchestra.run.smithi115.stdout:4743 read 0x40ca5fd thru 0x40d2bfb (0x85ff bytes) 2024-09-17T22:42:26.821 INFO:teuthology.orchestra.run.smithi115.stdout:4744 read 0x226185d thru 0x2268328 (0x6acc bytes) 2024-09-17T22:42:26.821 INFO:teuthology.orchestra.run.smithi115.stdout:4745 punch from 0x443fd9e to 0x444b6ba, (0xb91c bytes) 2024-09-17T22:42:26.821 INFO:teuthology.orchestra.run.smithi115.stdout:4746 write 0x5c8d773 thru 0x5c9c587 (0xee15 bytes) 2024-09-17T22:42:26.823 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:26.823 INFO:teuthology.orchestra.run.smithi115.stdout:4749 read 0x31aba4a thru 0x31ad24a (0x1801 bytes) 2024-09-17T22:42:26.825 INFO:teuthology.orchestra.run.smithi115.stdout:4750 trunc from 0x5ffd97a to 0x3ce1b9a 2024-09-17T22:42:26.836 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:26.833+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:26.848 INFO:teuthology.orchestra.run.smithi115.stdout:4751 write 0xc8163f1 thru 0xc81b3f1 (0x5001 bytes) 2024-09-17T22:42:26.852 INFO:teuthology.orchestra.run.smithi115.stdout:4752 read 0x1d26a30 thru 0x1d2ba37 (0x5008 bytes) 2024-09-17T22:42:26.852 INFO:teuthology.orchestra.run.smithi115.stdout:4753 read 0x3a0e28f thru 0x3a1e0cf (0xfe41 bytes) 2024-09-17T22:42:26.852 INFO:teuthology.orchestra.run.smithi115.stdout:4755 writesame 0x669e272 thru 0x66ac527 data_size 0x22(0xe2b6 bytes) 2024-09-17T22:42:26.854 INFO:teuthology.orchestra.run.smithi115.stdout:4756 write 0x4b4c991 thru 0x4b4dcb5 (0x1325 bytes) 2024-09-17T22:42:26.855 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:26.855 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:26.855 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:26.855 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:26.855 INFO:teuthology.orchestra.run.smithi115.stdout:4759 write 0x89b3d95 thru 0x89c0f1b (0xd187 bytes) 2024-09-17T22:42:26.856 INFO:teuthology.orchestra.run.smithi115.stdout:4760 write 0x1cff65b thru 0x1d00380 (0xd26 bytes) 2024-09-17T22:42:26.857 INFO:teuthology.orchestra.run.smithi115.stdout:4761 writesame 0x1e4a013 thru 0x1e50183 data_size 0x5(0x6171 bytes) 2024-09-17T22:42:26.860 INFO:teuthology.orchestra.run.smithi115.stdout:4762 writesame 0xc1099c4 thru 0xc115402 data_size 0x3(0xba3f bytes) 2024-09-17T22:42:26.864 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:42:26.877 INFO:teuthology.orchestra.run.smithi115.stdout:4763 trunc from 0xc81b3f2 to 0x6803467 2024-09-17T22:42:26.887 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:26.888 INFO:teuthology.orchestra.run.smithi115.stdout:4765 write 0x6787ee3 thru 0x6789bb9 (0x1cd7 bytes) 2024-09-17T22:42:26.896 INFO:teuthology.orchestra.run.smithi115.stdout:4766 write 0x8a86475 thru 0x8a91dea (0xb976 bytes) 2024-09-17T22:42:26.899 INFO:teuthology.orchestra.run.smithi115.stdout:4768 read 0x18b2ed7 thru 0x18c1c21 (0xed4b bytes) 2024-09-17T22:42:26.900 INFO:teuthology.orchestra.run.smithi115.stdout:4769 writesame 0x2a1b65 thru 0x2a6a56 data_size 0x2f(0x4ef2 bytes) 2024-09-17T22:42:26.901 INFO:teuthology.orchestra.run.smithi115.stdout:4770 punch from 0x7552d80 to 0x7561773, (0xe9f3 bytes) 2024-09-17T22:42:26.901 INFO:teuthology.orchestra.run.smithi115.stdout:4771 writesame 0x17487ff thru 0x174eec4 data_size 0xf(0x66c6 bytes) 2024-09-17T22:42:26.910 INFO:teuthology.orchestra.run.smithi115.stdout:4773 write 0x9f60668 thru 0x9f6a052 (0x99eb bytes) 2024-09-17T22:42:26.913 INFO:teuthology.orchestra.run.smithi115.stdout:4774 write 0x2e24306 thru 0x2e2bad5 (0x77d0 bytes) 2024-09-17T22:42:26.916 INFO:teuthology.orchestra.run.smithi115.stdout:4775 read 0x7641fd4 thru 0x764405b (0x2088 bytes) 2024-09-17T22:42:26.916 INFO:teuthology.orchestra.run.smithi115.stdout:4776 writesame 0x4715c4f thru 0x471e88a data_size 0x32(0x8c3c bytes) 2024-09-17T22:42:26.918 INFO:teuthology.orchestra.run.smithi115.stdout:4777 write 0x90eaa68 thru 0x90f791d (0xceb6 bytes) 2024-09-17T22:42:26.930 INFO:teuthology.orchestra.run.smithi115.stdout:4779 write 0xb6aa06b thru 0xb6b7ebd (0xde53 bytes) 2024-09-17T22:42:26.933 INFO:teuthology.orchestra.run.smithi115.stdout:4782 writesame 0x107ecbf thru 0x108b692 data_size 0x2(0xc9d4 bytes) 2024-09-17T22:42:26.937 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:26.934+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:26.945 INFO:teuthology.orchestra.run.smithi115.stdout:4783 write 0x11cd2bc thru 0x11d50df (0x7e24 bytes) 2024-09-17T22:42:26.947 INFO:teuthology.orchestra.run.smithi115.stdout:4784 punch from 0x99b627c to 0x99bfd8f, (0x9b13 bytes) 2024-09-17T22:42:26.947 INFO:teuthology.orchestra.run.smithi115.stdout:4786 write 0xc66b62 thru 0xc6cb06 (0x5fa5 bytes) 2024-09-17T22:42:26.948 INFO:teuthology.orchestra.run.smithi115.stdout:4787 punch from 0x981df6b to 0x9829d8b, (0xbe20 bytes) 2024-09-17T22:42:26.948 INFO:teuthology.orchestra.run.smithi115.stdout:4788 read 0x3e0affe thru 0x3e13a0d (0x8a10 bytes) 2024-09-17T22:42:26.949 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:26.951 INFO:teuthology.orchestra.run.smithi115.stdout:4790 trunc from 0xb6b7ebe to 0x8eb8de2 2024-09-17T22:42:26.966 INFO:teuthology.orchestra.run.smithi115.stdout:4792 writesame 0x3880aef thru 0x388c7d8 data_size 0x1(0xbcea bytes) 2024-09-17T22:42:26.989 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:26.989 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:26.989 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:26.989 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:26.989 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:26.991 INFO:teuthology.orchestra.run.smithi115.stdout:4795 trunc from 0x8eb8de2 to 0x9e7b9b2 2024-09-17T22:42:26.997 INFO:teuthology.orchestra.run.smithi115.stdout:4796 punch from 0x2823577 to 0x282c01c, (0x8aa5 bytes) 2024-09-17T22:42:26.997 INFO:teuthology.orchestra.run.smithi115.stdout:4797 writesame 0x3651755 thru 0x365e3d0 data_size 0x17(0xcc7c bytes) 2024-09-17T22:42:27.000 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:42:27.001 INFO:teuthology.orchestra.run.smithi115.stdout:4800 read 0x3bb05fc thru 0x3bb7330 (0x6d35 bytes) 2024-09-17T22:42:27.005 INFO:teuthology.orchestra.run.smithi115.stdout:4801 trunc from 0x9e7b9b2 to 0xd2af777 2024-09-17T22:42:27.011 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:27.011 INFO:teuthology.orchestra.run.smithi115.stdout:4805 flatten 2024-09-17T22:42:27.016 INFO:teuthology.orchestra.run.smithi115.stdout:4806 write 0x40dc832 thru 0x40dd64b (0xe1a bytes) 2024-09-17T22:42:27.018 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:27.018 INFO:teuthology.orchestra.run.smithi115.stdout:4808 writesame 0x8205faf thru 0x82152ca data_size 0x2(0xf31c bytes) 2024-09-17T22:42:27.033 INFO:teuthology.orchestra.run.smithi115.stdout:4810 writesame 0x44e8433 thru 0x44f799a data_size 0x4(0xf568 bytes) 2024-09-17T22:42:27.038 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:27.035+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:27.048 INFO:teuthology.orchestra.run.smithi115.stdout:4811 write 0xe640c2f thru 0xe64b2d4 (0xa6a6 bytes) 2024-09-17T22:42:27.052 INFO:teuthology.orchestra.run.smithi115.stdout:4812 write 0xbf5f2a3 thru 0xbf6779b (0x84f9 bytes) 2024-09-17T22:42:27.055 INFO:teuthology.orchestra.run.smithi115.stdout:4813 writesame 0x8c0ad71 thru 0x8c13485 data_size 0x1(0x8715 bytes) 2024-09-17T22:42:27.086 INFO:teuthology.orchestra.run.smithi115.stdout:4815 trunc from 0xe64b2d5 to 0x27e8175 2024-09-17T22:42:27.114 INFO:teuthology.orchestra.run.smithi115.stdout:4816 trunc from 0x27e8175 to 0x6e75c 2024-09-17T22:42:27.117 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:27.117 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:27.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:27.117 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:27.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.117 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4229.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-17T22:42:27.117 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:27.117 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.718553251000003, 2024-09-17T22:42:27.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25708877200000002, 2024-09-17T22:42:27.117 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.117 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.118 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.118 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.118 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4229", 2024-09-17T22:42:27.118 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/3080427038", 2024-09-17T22:42:27.118 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2 2024-09-17T22:42:27.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.118 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:27.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:27.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394611+0000", 2024-09-17T22:42:27.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3149999999999999e-06 2024-09-17T22:42:27.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394619+0000", 2024-09-17T22:42:27.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240000000000003e-06 2024-09-17T22:42:27.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394632+0000", 2024-09-17T22:42:27.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2921e-05 2024-09-17T22:42:27.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394639+0000", 2024-09-17T22:42:27.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-09-17T22:42:27.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394703+0000", 2024-09-17T22:42:27.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4407999999999997e-05 2024-09-17T22:42:27.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:42:27.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394711+0000", 2024-09-17T22:42:27.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5939999999999996e-06 2024-09-17T22:42:27.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651545+0000", 2024-09-17T22:42:27.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25683409200000001 2024-09-17T22:42:27.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651698+0000", 2024-09-17T22:42:27.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000153678 2024-09-17T22:42:27.122 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.122 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.122 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.122 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38736 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:27.122 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:27.123 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.335957246, 2024-09-17T22:42:27.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051395418999999998, 2024-09-17T22:42:27.123 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.123 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.123 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:27.123 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.123 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:27.123 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.123 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38736 2024-09-17T22:42:27.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.123 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:27.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:27.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:27.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6899999999999998e-07 2024-09-17T22:42:27.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777212+0000", 2024-09-17T22:42:27.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1240000000000002e-06 2024-09-17T22:42:27.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777219+0000", 2024-09-17T22:42:27.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5409999999999997e-06 2024-09-17T22:42:27.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777224+0000", 2024-09-17T22:42:27.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:42:27.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.826131+0000", 2024-09-17T22:42:27.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048907369999999999 2024-09-17T22:42:27.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:27.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828457+0000", 2024-09-17T22:42:27.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023256209999999999 2024-09-17T22:42:27.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828601+0000", 2024-09-17T22:42:27.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014438799999999999 2024-09-17T22:42:27.127 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.127 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.127 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.127 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38752 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 673792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:27.127 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:27.127 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.335933175000001, 2024-09-17T22:42:27.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051582301999999997, 2024-09-17T22:42:27.128 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.128 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.128 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:27.128 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.128 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:27.128 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.128 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38752 2024-09-17T22:42:27.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.128 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:27.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:27.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:27.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3000000000000001e-07 2024-09-17T22:42:27.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777232+0000", 2024-09-17T22:42:27.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9709999999999998e-06 2024-09-17T22:42:27.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777238+0000", 2024-09-17T22:42:27.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7209999999999999e-06 2024-09-17T22:42:27.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777243+0000", 2024-09-17T22:42:27.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5210000000000004e-06 2024-09-17T22:42:27.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828633+0000", 2024-09-17T22:42:27.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051390915000000002 2024-09-17T22:42:27.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:27.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828711+0000", 2024-09-17T22:42:27.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7474999999999999e-05 2024-09-17T22:42:27.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828812+0000", 2024-09-17T22:42:27.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010116900000000001 2024-09-17T22:42:27.132 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.132 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.132 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.132 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38753 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:27.132 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:27.132 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.335913902000001, 2024-09-17T22:42:27.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051770444999999998, 2024-09-17T22:42:27.132 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.133 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.133 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:27.133 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.133 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:27.133 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.133 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38753 2024-09-17T22:42:27.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.133 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:27.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:27.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:27.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:42:27.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777251+0000", 2024-09-17T22:42:27.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8059999999999999e-06 2024-09-17T22:42:27.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777257+0000", 2024-09-17T22:42:27.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2480000000000001e-06 2024-09-17T22:42:27.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777262+0000", 2024-09-17T22:42:27.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000004e-06 2024-09-17T22:42:27.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828842+0000", 2024-09-17T22:42:27.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051580203999999998 2024-09-17T22:42:27.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:27.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828917+0000", 2024-09-17T22:42:27.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4980000000000004e-05 2024-09-17T22:42:27.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829019+0000", 2024-09-17T22:42:27.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102248 2024-09-17T22:42:27.136 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.137 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.137 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.137 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38766 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:27.137 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:27.137 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.335894916999997, 2024-09-17T22:42:27.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051958839999999999, 2024-09-17T22:42:27.137 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.137 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.137 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:27.137 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.137 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:27.137 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.138 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38766 2024-09-17T22:42:27.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.138 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:27.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:27.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777269+0000", 2024-09-17T22:42:27.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:42:27.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777271+0000", 2024-09-17T22:42:27.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-17T22:42:27.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777276+0000", 2024-09-17T22:42:27.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:42:27.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777281+0000", 2024-09-17T22:42:27.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:42:27.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829055+0000", 2024-09-17T22:42:27.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051773570999999997 2024-09-17T22:42:27.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:27.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829128+0000", 2024-09-17T22:42:27.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3131000000000002e-05 2024-09-17T22:42:27.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829227+0000", 2024-09-17T22:42:27.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8800000000000003e-05 2024-09-17T22:42:27.141 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.141 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.141 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.142 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38767 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:27.142 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:27.142 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.335874934000003, 2024-09-17T22:42:27.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137018, 2024-09-17T22:42:27.142 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.142 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.142 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:27.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:27.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.142 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38767 2024-09-17T22:42:27.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.142 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:27.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:27.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:27.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:42:27.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777290+0000", 2024-09-17T22:42:27.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.88e-06 2024-09-17T22:42:27.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777296+0000", 2024-09-17T22:42:27.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-09-17T22:42:27.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777300+0000", 2024-09-17T22:42:27.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4510000000000002e-06 2024-09-17T22:42:27.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829257+0000", 2024-09-17T22:42:27.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051956975000000002 2024-09-17T22:42:27.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:27.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829328+0000", 2024-09-17T22:42:27.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1026999999999996e-05 2024-09-17T22:42:27.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829425+0000", 2024-09-17T22:42:27.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.679e-05 2024-09-17T22:42:27.146 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.146 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.146 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.146 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38784 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 675840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:27.146 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:27.146 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.335856479999997, 2024-09-17T22:42:27.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052344522999999997, 2024-09-17T22:42:27.147 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.147 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.147 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:27.147 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.147 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:27.147 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.147 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38784 2024-09-17T22:42:27.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.147 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:27.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:27.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777307+0000", 2024-09-17T22:42:27.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:42:27.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777309+0000", 2024-09-17T22:42:27.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.063e-06 2024-09-17T22:42:27.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777314+0000", 2024-09-17T22:42:27.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-17T22:42:27.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777319+0000", 2024-09-17T22:42:27.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999996e-06 2024-09-17T22:42:27.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829456+0000", 2024-09-17T22:42:27.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137151 2024-09-17T22:42:27.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:27.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829527+0000", 2024-09-17T22:42:27.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1588000000000006e-05 2024-09-17T22:42:27.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829651+0000", 2024-09-17T22:42:27.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012365 2024-09-17T22:42:27.151 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.151 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.151 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.151 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38785 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 749568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:27.151 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:27.151 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.335838365000001, 2024-09-17T22:42:27.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052546572999999999, 2024-09-17T22:42:27.151 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.151 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.151 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:27.151 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.152 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:27.152 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.152 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38785 2024-09-17T22:42:27.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.152 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:27.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:27.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:27.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:42:27.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777327+0000", 2024-09-17T22:42:27.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:42:27.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777333+0000", 2024-09-17T22:42:27.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7309999999999998e-06 2024-09-17T22:42:27.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777337+0000", 2024-09-17T22:42:27.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:42:27.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829686+0000", 2024-09-17T22:42:27.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052349159999999999 2024-09-17T22:42:27.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:27.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829766+0000", 2024-09-17T22:42:27.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9326999999999995e-05 2024-09-17T22:42:27.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829871+0000", 2024-09-17T22:42:27.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000105466 2024-09-17T22:42:27.155 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.155 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.155 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.156 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38798 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:27.156 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:27.156 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.335204757, 2024-09-17T22:42:27.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052117564999999998, 2024-09-17T22:42:27.156 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.156 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.156 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:27.156 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.156 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:27.156 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.156 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38798 2024-09-17T22:42:27.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.157 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:27.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:27.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777959+0000", 2024-09-17T22:42:27.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9599999999999999e-07 2024-09-17T22:42:27.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777963+0000", 2024-09-17T22:42:27.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2660000000000003e-06 2024-09-17T22:42:27.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777971+0000", 2024-09-17T22:42:27.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3489999999999992e-06 2024-09-17T22:42:27.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777978+0000", 2024-09-17T22:42:27.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:42:27.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829906+0000", 2024-09-17T22:42:27.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051927635999999999 2024-09-17T22:42:27.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:27.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829986+0000", 2024-09-17T22:42:27.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0444000000000004e-05 2024-09-17T22:42:27.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830076+0000", 2024-09-17T22:42:27.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9473000000000002e-05 2024-09-17T22:42:27.160 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.160 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.160 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.160 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38799 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 882688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:27.160 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:27.161 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.335173318000002, 2024-09-17T22:42:27.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052359606000000003, 2024-09-17T22:42:27.161 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.161 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.161 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:27.161 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.161 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:27.161 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.161 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38799 2024-09-17T22:42:27.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.161 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:27.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:27.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:27.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8299999999999997e-07 2024-09-17T22:42:27.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777992+0000", 2024-09-17T22:42:27.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.176e-06 2024-09-17T22:42:27.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777998+0000", 2024-09-17T22:42:27.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.173e-06 2024-09-17T22:42:27.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778003+0000", 2024-09-17T22:42:27.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6349999999999997e-06 2024-09-17T22:42:27.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830112+0000", 2024-09-17T22:42:27.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052109366999999997 2024-09-17T22:42:27.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:27.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830192+0000", 2024-09-17T22:42:27.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9814000000000005e-05 2024-09-17T22:42:27.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830349+0000", 2024-09-17T22:42:27.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015695800000000001 2024-09-17T22:42:27.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.165 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.165 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38814 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:27.165 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:27.165 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.334889914000001, 2024-09-17T22:42:27.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052274339000000003, 2024-09-17T22:42:27.166 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.166 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.166 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:27.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:27.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.166 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38814 2024-09-17T22:42:27.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.166 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:27.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:27.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:27.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:42:27.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778276+0000", 2024-09-17T22:42:27.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2029999999999999e-06 2024-09-17T22:42:27.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778282+0000", 2024-09-17T22:42:27.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0059999999999998e-06 2024-09-17T22:42:27.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778287+0000", 2024-09-17T22:42:27.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0039999999999999e-06 2024-09-17T22:42:27.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830382+0000", 2024-09-17T22:42:27.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052095379999999997 2024-09-17T22:42:27.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:27.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830455+0000", 2024-09-17T22:42:27.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3127000000000006e-05 2024-09-17T22:42:27.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830547+0000", 2024-09-17T22:42:27.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2207999999999995e-05 2024-09-17T22:42:27.170 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.170 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.170 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.170 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38828 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 163840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:27.170 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:27.170 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.334707506000001, 2024-09-17T22:42:27.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052466039999999999, 2024-09-17T22:42:27.170 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.170 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.170 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:27.170 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.171 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:27.171 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.171 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38828 2024-09-17T22:42:27.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.171 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:27.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:27.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778456+0000", 2024-09-17T22:42:27.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7599999999999996e-07 2024-09-17T22:42:27.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778460+0000", 2024-09-17T22:42:27.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8229999999999997e-06 2024-09-17T22:42:27.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778470+0000", 2024-09-17T22:42:27.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8199999999999992e-06 2024-09-17T22:42:27.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778476+0000", 2024-09-17T22:42:27.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2589999999999997e-06 2024-09-17T22:42:27.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830578+0000", 2024-09-17T22:42:27.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052102273999999997 2024-09-17T22:42:27.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:27.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830667+0000", 2024-09-17T22:42:27.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8701000000000003e-05 2024-09-17T22:42:27.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830921+0000", 2024-09-17T22:42:27.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025438700000000002 2024-09-17T22:42:27.174 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.174 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.175 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.175 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38829 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:27.175 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:27.175 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.334676109, 2024-09-17T22:42:27.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052648882000000001, 2024-09-17T22:42:27.175 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.175 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.175 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:27.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:27.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.175 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38829 2024-09-17T22:42:27.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.176 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:27.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:27.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:27.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7e-07 2024-09-17T22:42:27.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778490+0000", 2024-09-17T22:42:27.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:42:27.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778495+0000", 2024-09-17T22:42:27.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9329999999999998e-06 2024-09-17T22:42:27.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778500+0000", 2024-09-17T22:42:27.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9509999999999997e-06 2024-09-17T22:42:27.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830958+0000", 2024-09-17T22:42:27.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05245785 2024-09-17T22:42:27.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:27.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831051+0000", 2024-09-17T22:42:27.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2989e-05 2024-09-17T22:42:27.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831136+0000", 2024-09-17T22:42:27.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4448000000000002e-05 2024-09-17T22:42:27.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.179 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.180 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38891 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:27.180 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:27.180 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.331364051000001, 2024-09-17T22:42:27.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050847934999999997, 2024-09-17T22:42:27.180 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.180 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.180 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:27.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:27.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.180 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38891 2024-09-17T22:42:27.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.181 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:27.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:27.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:27.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-09-17T22:42:27.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781802+0000", 2024-09-17T22:42:27.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:42:27.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781810+0000", 2024-09-17T22:42:27.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8019999999999992e-06 2024-09-17T22:42:27.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781816+0000", 2024-09-17T22:42:27.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.285e-06 2024-09-17T22:42:27.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832485+0000", 2024-09-17T22:42:27.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050668462999999997 2024-09-17T22:42:27.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:27.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832565+0000", 2024-09-17T22:42:27.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9692999999999997e-05 2024-09-17T22:42:27.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832647+0000", 2024-09-17T22:42:27.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2224000000000004e-05 2024-09-17T22:42:27.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.184 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.184 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38892 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:27.185 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:27.185 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.331335801999998, 2024-09-17T22:42:27.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050946120999999997, 2024-09-17T22:42:27.185 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.185 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.185 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:27.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:27.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.185 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38892 2024-09-17T22:42:27.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.185 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:27.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:27.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781828+0000", 2024-09-17T22:42:27.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5400000000000002e-07 2024-09-17T22:42:27.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781830+0000", 2024-09-17T22:42:27.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1959999999999998e-06 2024-09-17T22:42:27.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781837+0000", 2024-09-17T22:42:27.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8310000000000003e-06 2024-09-17T22:42:27.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781843+0000", 2024-09-17T22:42:27.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1e-06 2024-09-17T22:42:27.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832674+0000", 2024-09-17T22:42:27.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050831182000000003 2024-09-17T22:42:27.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:27.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832725+0000", 2024-09-17T22:42:27.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1588000000000001e-05 2024-09-17T22:42:27.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832773+0000", 2024-09-17T22:42:27.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7769999999999998e-05 2024-09-17T22:42:27.189 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.189 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.189 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.189 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38909 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 36864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:27.189 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:27.190 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.331313031999997, 2024-09-17T22:42:27.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051046518999999999, 2024-09-17T22:42:27.190 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.190 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.190 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:27.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:27.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.190 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38909 2024-09-17T22:42:27.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.190 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:27.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:27.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:27.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:42:27.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781852+0000", 2024-09-17T22:42:27.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0590000000000001e-06 2024-09-17T22:42:27.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781859+0000", 2024-09-17T22:42:27.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3380000000000001e-06 2024-09-17T22:42:27.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781864+0000", 2024-09-17T22:42:27.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:42:27.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832804+0000", 2024-09-17T22:42:27.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050939614000000001 2024-09-17T22:42:27.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:27.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832845+0000", 2024-09-17T22:42:27.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1014999999999999e-05 2024-09-17T22:42:27.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832896+0000", 2024-09-17T22:42:27.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1836999999999999e-05 2024-09-17T22:42:27.194 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.194 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.194 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.194 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38910 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:27.194 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:27.194 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.331292722000001, 2024-09-17T22:42:27.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051216818999999997, 2024-09-17T22:42:27.194 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.194 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.194 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:27.194 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.195 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:27.195 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.195 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38910 2024-09-17T22:42:27.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.195 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:27.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:27.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781871+0000", 2024-09-17T22:42:27.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:42:27.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781873+0000", 2024-09-17T22:42:27.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:42:27.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781879+0000", 2024-09-17T22:42:27.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0700000000000003e-06 2024-09-17T22:42:27.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781884+0000", 2024-09-17T22:42:27.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-09-17T22:42:27.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832932+0000", 2024-09-17T22:42:27.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051048219999999998 2024-09-17T22:42:27.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:27.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833001+0000", 2024-09-17T22:42:27.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8504999999999995e-05 2024-09-17T22:42:27.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833087+0000", 2024-09-17T22:42:27.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6464000000000004e-05 2024-09-17T22:42:27.198 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.198 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.198 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.199 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38923 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:27.199 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:27.199 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.331272402000003, 2024-09-17T22:42:27.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051379573999999997, 2024-09-17T22:42:27.199 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.199 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.199 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:27.199 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.199 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:27.199 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.199 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38923 2024-09-17T22:42:27.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.199 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:27.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:27.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:27.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:42:27.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781893+0000", 2024-09-17T22:42:27.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-09-17T22:42:27.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781900+0000", 2024-09-17T22:42:27.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3860000000000004e-06 2024-09-17T22:42:27.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781904+0000", 2024-09-17T22:42:27.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9300000000000002e-06 2024-09-17T22:42:27.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833115+0000", 2024-09-17T22:42:27.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051210845999999997 2024-09-17T22:42:27.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:27.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833188+0000", 2024-09-17T22:42:27.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2276999999999999e-05 2024-09-17T22:42:27.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833270+0000", 2024-09-17T22:42:27.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2501000000000002e-05 2024-09-17T22:42:27.203 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.203 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.203 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.203 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38941 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:27.203 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:27.203 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 57.330479867000001, 2024-09-17T22:42:27.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050969459000000002, 2024-09-17T22:42:27.203 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.204 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.204 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:27.204 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.204 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:27.204 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.204 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38941 2024-09-17T22:42:27.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.204 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:27.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:27.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782684+0000", 2024-09-17T22:42:27.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:42:27.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782686+0000", 2024-09-17T22:42:27.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.396e-06 2024-09-17T22:42:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782693+0000", 2024-09-17T22:42:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1999999999999997e-06 2024-09-17T22:42:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782699+0000", 2024-09-17T22:42:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7910000000000001e-06 2024-09-17T22:42:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833471+0000", 2024-09-17T22:42:27.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050772333000000003 2024-09-17T22:42:27.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:27.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833536+0000", 2024-09-17T22:42:27.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4380000000000004e-05 2024-09-17T22:42:27.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833653+0000", 2024-09-17T22:42:27.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011691799999999999 2024-09-17T22:42:27.207 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.207 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.208 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.208 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:151189 3.14 3:2b696b13:::rbd_data.115f14f2d1d5.0000000000000009:e [read 3127296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.208 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:27.208 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.400259122000001, 2024-09-17T22:42:27.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.054254700000000003, 2024-09-17T22:42:27.208 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:27.208 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:27.208 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:27.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:27.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:27.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.209 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 151189 2024-09-17T22:42:27.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.209 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:27.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:27.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:27.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:27.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:27.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:27.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:27.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713051+0000", 2024-09-17T22:42:27.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014722100000000001 2024-09-17T22:42:27.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:27.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713055+0000", 2024-09-17T22:42:27.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.16e-06 2024-09-17T22:42:27.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:27.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713065+0000", 2024-09-17T22:42:27.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8390000000000002e-06 2024-09-17T22:42:27.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713073+0000", 2024-09-17T22:42:27.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0390000000000005e-06 2024-09-17T22:42:27.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:27.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762644+0000", 2024-09-17T22:42:27.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049571160000000003 2024-09-17T22:42:27.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:27.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762739+0000", 2024-09-17T22:42:27.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4565e-05 2024-09-17T22:42:27.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:27.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:27.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:27.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.767159+0000", 2024-09-17T22:42:27.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044197159999999997 2024-09-17T22:42:27.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.212 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:27.212 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:27.212 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:27.214 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:42:27.215 INFO:teuthology.orchestra.run.smithi115.stdout:4817 read 0xf332 thru 0x17abc (0x878b bytes) 2024-09-17T22:42:27.215 INFO:teuthology.orchestra.run.smithi115.stdout:4818 write 0x25c517d thru 0x25ce026 (0x8eaa bytes) 2024-09-17T22:42:27.215 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:27.215 INFO:teuthology.orchestra.run.smithi115.stdout:4823 punch from 0x73ad21 to 0x74142a, (0x6709 bytes) 2024-09-17T22:42:27.216 INFO:teuthology.orchestra.run.smithi115.stdout:4824 read 0x1265d2e thru 0x126c530 (0x6803 bytes) 2024-09-17T22:42:27.216 INFO:teuthology.orchestra.run.smithi115.stdout:4825 writesame 0xb25a7fc thru 0xb25cb42 data_size 0x1(0x2347 bytes) 2024-09-17T22:42:27.216 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:27.216 INFO:teuthology.orchestra.run.smithi115.stdout:4827 writesame 0x16980e1 thru 0x16a597d data_size 0x17(0xd89d bytes) 2024-09-17T22:42:27.216 INFO:teuthology.orchestra.run.smithi115.stdout:4828 punch from 0x8734751 to 0x873d023, (0x88d2 bytes) 2024-09-17T22:42:27.216 INFO:teuthology.orchestra.run.smithi115.stdout:4830 read 0x1dee7c2 thru 0x1df1280 (0x2abf bytes) 2024-09-17T22:42:27.216 INFO:teuthology.orchestra.run.smithi115.stdout:4831 trunc from 0xb25cb43 to 0x27680fd 2024-09-17T22:42:27.216 INFO:teuthology.orchestra.run.smithi115.stdout:4832 write 0x26cc9f2 thru 0x26da109 (0xd718 bytes) 2024-09-17T22:42:27.216 INFO:teuthology.orchestra.run.smithi115.stdout:4833 read 0x19c33e6 thru 0x19c5ff3 (0x2c0e bytes) 2024-09-17T22:42:27.216 INFO:teuthology.orchestra.run.smithi115.stdout:4834 trunc from 0x27680fd to 0x9dee97 2024-09-17T22:42:27.216 INFO:teuthology.orchestra.run.smithi115.stdout:4835 write 0xc919fcc thru 0xc91fc7f (0x5cb4 bytes) 2024-09-17T22:42:27.217 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:27.211+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:27.217 INFO:teuthology.orchestra.run.smithi115.stdout:4836 read 0x99898e8 thru 0x9994c87 (0xb3a0 bytes) 2024-09-17T22:42:27.217 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:27.217 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:27.217 INFO:teuthology.orchestra.run.smithi115.stdout:4839 writesame 0x1ca818c thru 0x1cb2f58 data_size 0x3(0xadcd bytes) 2024-09-17T22:42:27.222 INFO:teuthology.orchestra.run.smithi115.stdout:4840 read 0x4803d40 thru 0x480402f (0x2f0 bytes) 2024-09-17T22:42:27.222 INFO:teuthology.orchestra.run.smithi115.stdout:4841 write 0x8bb1400 thru 0x8bbea9d (0xd69e bytes) 2024-09-17T22:42:27.225 INFO:teuthology.orchestra.run.smithi115.stdout:4843 trunc from 0xc91fc80 to 0xd6ac93a 2024-09-17T22:42:27.231 INFO:teuthology.orchestra.run.smithi115.stdout:4844 read 0xc1570bc thru 0xc166b78 (0xfabd bytes) 2024-09-17T22:42:27.232 INFO:teuthology.orchestra.run.smithi115.stdout:4845 trunc from 0xd6ac93a to 0xc152a4f 2024-09-17T22:42:27.239 INFO:teuthology.orchestra.run.smithi115.stdout:4847 writesame 0x4a86a53 thru 0x4a93d81 data_size 0x3(0xd32f bytes) 2024-09-17T22:42:27.248 INFO:teuthology.orchestra.run.smithi115.stdout:4848 punch from 0x7e80e96 to 0x7e82a59, (0x1bc3 bytes) 2024-09-17T22:42:27.248 INFO:teuthology.orchestra.run.smithi115.stdout:4849 punch from 0x320c8d4 to 0x3215496, (0x8bc2 bytes) 2024-09-17T22:42:27.248 INFO:teuthology.orchestra.run.smithi115.stdout:4851 write 0xfb2238 thru 0xfb22af (0x78 bytes) 2024-09-17T22:42:27.249 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:27.249 INFO:teuthology.orchestra.run.smithi115.stdout:4855 write 0xb49f9d2 thru 0xb4aa115 (0xa744 bytes) 2024-09-17T22:42:27.252 INFO:teuthology.orchestra.run.smithi115.stdout:4856 write 0x1085409 thru 0x108763e (0x2236 bytes) 2024-09-17T22:42:27.255 INFO:teuthology.orchestra.run.smithi115.stdout:4859 read 0xad8e458 thru 0xad90875 (0x241e bytes) 2024-09-17T22:42:27.268 INFO:teuthology.orchestra.run.smithi115.stdout:4860 trunc from 0xc152a4f to 0x9cb768 2024-09-17T22:42:27.275 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:42:27.283 INFO:teuthology.orchestra.run.smithi115.stdout:4863 punch from 0x633774 to 0x641ad5, (0xe361 bytes) 2024-09-17T22:42:27.300 INFO:teuthology.orchestra.run.smithi115.stdout:4864 write 0xaa517e9 thru 0xaa603e9 (0xec01 bytes) 2024-09-17T22:42:27.311 INFO:teuthology.orchestra.run.smithi115.stdout:4865 trunc from 0xaa603ea to 0x4835e92 2024-09-17T22:42:27.314 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:27.312+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:27.328 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:27.328 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:27.328 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:27.328 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:27.334 INFO:teuthology.orchestra.run.smithi115.stdout:4866 writesame 0xaa02fce thru 0xaa059a7 data_size 0xb(0x29da bytes) 2024-09-17T22:42:27.334 INFO:teuthology.orchestra.run.smithi115.stdout:4867 write 0x6d05244 thru 0x6d10ed6 (0xbc93 bytes) 2024-09-17T22:42:27.337 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:42:27.339 INFO:teuthology.orchestra.run.smithi115.stdout:4868 read 0x5954da8 thru 0x595acfb (0x5f54 bytes) 2024-09-17T22:42:27.339 INFO:teuthology.orchestra.run.smithi115.stdout:4869 write 0x7c33ed3 thru 0x7c3f1f7 (0xb325 bytes) 2024-09-17T22:42:27.342 INFO:teuthology.orchestra.run.smithi115.stdout:4870 read 0x372887 thru 0x38055f (0xdcd9 bytes) 2024-09-17T22:42:27.342 INFO:teuthology.orchestra.run.smithi115.stdout:4871 read 0x8bd8d61 thru 0x8be1189 (0x8429 bytes) 2024-09-17T22:42:27.342 INFO:teuthology.orchestra.run.smithi115.stdout:4872 writesame 0xa6df88c thru 0xa6e7aa0 data_size 0x1(0x8215 bytes) 2024-09-17T22:42:27.361 INFO:teuthology.orchestra.run.smithi115.stdout:4873 read 0x5c8de9e thru 0x5c9793a (0x9a9d bytes) 2024-09-17T22:42:27.364 INFO:teuthology.orchestra.run.smithi115.stdout:4874 trunc from 0xaa059a8 to 0xdcfb0e3 2024-09-17T22:42:27.370 INFO:teuthology.orchestra.run.smithi115.stdout:4875 write 0xac94284 thru 0xac9d9fc (0x9779 bytes) 2024-09-17T22:42:27.374 INFO:teuthology.orchestra.run.smithi115.stdout:4876 trunc from 0xdcfb0e3 to 0xdf9a6c9 2024-09-17T22:42:27.382 INFO:teuthology.orchestra.run.smithi115.stdout:4877 trunc from 0xdf9a6c9 to 0xd1d309a 2024-09-17T22:42:27.388 INFO:teuthology.orchestra.run.smithi115.stdout:4878 read 0x5568ff6 thru 0x5573214 (0xa21f bytes) 2024-09-17T22:42:27.389 INFO:teuthology.orchestra.run.smithi115.stdout:4879 trunc from 0xd1d309a to 0xdf10afe 2024-09-17T22:42:27.395 INFO:teuthology.orchestra.run.smithi115.stdout:4880 writesame 0x759007b thru 0x75996d7 data_size 0x9(0x965d bytes) 2024-09-17T22:42:27.398 INFO:teuthology.orchestra.run.smithi115.stdout:4881 writesame 0x20965ee thru 0x20a07e9 data_size 0xe(0xa1fc bytes) 2024-09-17T22:42:27.403 INFO:teuthology.orchestra.run.smithi115.stdout:4882 writesame 0xb7b36d4 thru 0xb7b6750 data_size 0x1(0x307d bytes) 2024-09-17T22:42:27.411 INFO:teuthology.orchestra.run.smithi115.stdout:4883 punch from 0xa6f7797 to 0xa704e49, (0xd6b2 bytes) 2024-09-17T22:42:27.411 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:27.411 INFO:teuthology.orchestra.run.smithi115.stdout:4889 punch from 0x49a1a66 to 0x49a83a6, (0x6940 bytes) 2024-09-17T22:42:27.411 INFO:teuthology.orchestra.run.smithi115.stdout:4890 write 0x2f362de thru 0x2f3bd4a (0x5a6d bytes) 2024-09-17T22:42:27.413 INFO:teuthology.orchestra.run.smithi115.stdout:4891 write 0x229c24 thru 0x22c302 (0x26df bytes) 2024-09-17T22:42:27.414 INFO:teuthology.orchestra.run.smithi115.stdout:4892 punch from 0xcc8e942 to 0xcc9413b, (0x57f9 bytes) 2024-09-17T22:42:27.414 INFO:teuthology.orchestra.run.smithi115.stdout:4893 write 0x81b30c7 thru 0x81b8850 (0x578a bytes) 2024-09-17T22:42:27.414 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:27.412+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:27.417 INFO:teuthology.orchestra.run.smithi115.stdout:4894 write 0x4b6b73d thru 0x4b6e8e4 (0x31a8 bytes) 2024-09-17T22:42:27.418 INFO:teuthology.orchestra.run.smithi115.stdout:4895 read 0x8ac02a7 thru 0x8acc6ca (0xc424 bytes) 2024-09-17T22:42:27.426 INFO:teuthology.orchestra.run.smithi115.stdout:4899 trunc from 0xdf10afe to 0x74e2fe8 2024-09-17T22:42:27.442 INFO:teuthology.orchestra.run.smithi115.stdout:4900 writesame 0x4572b98 thru 0x45769a6 data_size 0x1(0x3e0f bytes) 2024-09-17T22:42:27.447 INFO:teuthology.orchestra.run.smithi115.stdout:4901 writesame 0x48fb263 thru 0x48fc306 data_size 0x1e(0x10a4 bytes) 2024-09-17T22:42:27.449 INFO:teuthology.orchestra.run.smithi115.stdout:4902 read 0x4258295 thru 0x4262474 (0xa1e0 bytes) 2024-09-17T22:42:27.449 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:27.449 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:27.459 INFO:teuthology.orchestra.run.smithi115.stdout:4907 write 0x8567545 thru 0x85675a7 (0x63 bytes) 2024-09-17T22:42:27.462 INFO:teuthology.orchestra.run.smithi115.stdout:4908 read 0x4d4d15d thru 0x4d56bed (0x9a91 bytes) 2024-09-17T22:42:27.465 INFO:teuthology.orchestra.run.smithi115.stdout:4909 trunc from 0x85675a8 to 0xaad2df1 2024-09-17T22:42:27.470 INFO:teuthology.orchestra.run.smithi115.stdout:4910 trunc from 0xaad2df1 to 0xa5d1300 2024-09-17T22:42:27.482 INFO:teuthology.orchestra.run.smithi115.stdout:4911 trunc from 0xa5d1300 to 0x4c31c06 2024-09-17T22:42:27.493 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:27.494 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:27.494 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:27.494 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:27.494 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:27.494 INFO:teuthology.orchestra.run.smithi115.stdout:4912 read 0x3bdfbe6 thru 0x3be8475 (0x8890 bytes) 2024-09-17T22:42:27.494 INFO:teuthology.orchestra.run.smithi115.stdout:4913 read 0x7e6a66 thru 0x7e7182 (0x71d bytes) 2024-09-17T22:42:27.495 INFO:teuthology.orchestra.run.smithi115.stdout:4914 write 0x3fae1bd thru 0x3fb6df6 (0x8c3a bytes) 2024-09-17T22:42:27.496 INFO:teuthology.orchestra.run.smithi115.stdout:4915 punch from 0x331c066 to 0x332bf14, (0xfeae bytes) 2024-09-17T22:42:27.496 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:27.503 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:42:27.513 INFO:teuthology.orchestra.run.smithi115.stdout:4917 write 0xe7469b4 thru 0xe74d8aa (0x6ef7 bytes) 2024-09-17T22:42:27.517 INFO:teuthology.orchestra.run.smithi115.stdout:4918 read 0x19692ec thru 0x196bc7e (0x2993 bytes) 2024-09-17T22:42:27.517 INFO:teuthology.orchestra.run.smithi115.stdout:4919 punch from 0x9b14ffd to 0x9b226f6, (0xd6f9 bytes) 2024-09-17T22:42:27.517 INFO:teuthology.orchestra.run.smithi115.stdout:4920 punch from 0x5736a90 to 0x573a3f0, (0x3960 bytes) 2024-09-17T22:42:27.517 INFO:teuthology.orchestra.run.smithi115.stdout:4921 read 0xbdbc7cd thru 0xbdc00d8 (0x390c bytes) 2024-09-17T22:42:27.517 INFO:teuthology.orchestra.run.smithi115.stdout:4922 write 0x6e6a9a thru 0x6edf6a (0x74d1 bytes) 2024-09-17T22:42:27.518 INFO:teuthology.orchestra.run.smithi115.stdout:4924 read 0x831879b thru 0x83198e3 (0x1149 bytes) 2024-09-17T22:42:27.518 INFO:teuthology.orchestra.run.smithi115.stdout:4926 writesame 0x60083df thru 0x60091a7 data_size 0x1(0xdc9 bytes) 2024-09-17T22:42:27.520 INFO:teuthology.orchestra.run.smithi115.stdout:4928 read 0x154240d thru 0x1547764 (0x5358 bytes) 2024-09-17T22:42:27.520 INFO:teuthology.orchestra.run.smithi115.stdout:4929 write 0xb420df2 thru 0xb42f10e (0xe31d bytes) 2024-09-17T22:42:27.523 INFO:teuthology.orchestra.run.smithi115.stdout:4932 read 0x3f7780f thru 0x3f838f7 (0xc0e9 bytes) 2024-09-17T22:42:27.523 INFO:teuthology.orchestra.run.smithi115.stdout:4933 read 0x25f5bc1 thru 0x2603a22 (0xde62 bytes) 2024-09-17T22:42:27.524 INFO:teuthology.orchestra.run.smithi115.stdout:4934 write 0x2ae9baf thru 0x2af1b2e (0x7f80 bytes) 2024-09-17T22:42:27.525 INFO:teuthology.orchestra.run.smithi115.stdout:4937 punch from 0x102b220 to 0x1033b01, (0x88e1 bytes) 2024-09-17T22:42:27.526 INFO:teuthology.orchestra.run.smithi115.stdout:4938 trunc from 0xe74d8ab to 0xe1ab8b3 2024-09-17T22:42:27.533 INFO:teuthology.orchestra.run.smithi115.stdout:4939 punch from 0x925f142 to 0x9267427, (0x82e5 bytes) 2024-09-17T22:42:27.533 INFO:teuthology.orchestra.run.smithi115.stdout:4940 write 0xdd1680b thru 0xdd23cc9 (0xd4bf bytes) 2024-09-17T22:42:27.536 INFO:teuthology.orchestra.run.smithi115.stdout:4941 write 0x12d7693 thru 0x12dc5c3 (0x4f31 bytes) 2024-09-17T22:42:27.539 INFO:teuthology.orchestra.run.smithi115.stdout:4942 read 0xb83317f thru 0xb83f86f (0xc6f1 bytes) 2024-09-17T22:42:27.539 INFO:teuthology.orchestra.run.smithi115.stdout:4943 read 0xd0cc870 thru 0xd0d547d (0x8c0e bytes) 2024-09-17T22:42:27.540 INFO:teuthology.orchestra.run.smithi115.stdout:4944 trunc from 0xe1ab8b3 to 0xe889193 2024-09-17T22:42:27.546 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:27.544+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:27.560 INFO:teuthology.orchestra.run.smithi115.stdout:4945 trunc from 0xe889193 to 0x20e08fc 2024-09-17T22:42:27.583 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:42:27.595 INFO:teuthology.orchestra.run.smithi115.stdout:4946 write 0x8c0ff03 thru 0x8c195d8 (0x96d6 bytes) 2024-09-17T22:42:27.605 INFO:teuthology.orchestra.run.smithi115.stdout:4950 trunc from 0x8c195d9 to 0x37bb7d4 2024-09-17T22:42:27.614 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:27.614 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:27.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:27.614 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:27.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.614 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158673 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.614 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:27.614 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369565326, 2024-09-17T22:42:27.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039400614, 2024-09-17T22:42:27.615 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.615 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.615 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.615 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.615 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.615 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.615 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158673 2024-09-17T22:42:27.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.615 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:27.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:27.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:27.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.04e-07 2024-09-17T22:42:27.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241563+0000", 2024-09-17T22:42:27.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5799999999999998e-07 2024-09-17T22:42:27.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241566+0000", 2024-09-17T22:42:27.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8279999999999999e-06 2024-09-17T22:42:27.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241568+0000", 2024-09-17T22:42:27.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2709999999999999e-06 2024-09-17T22:42:27.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280954+0000", 2024-09-17T22:42:27.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039386051999999998 2024-09-17T22:42:27.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280962+0000", 2024-09-17T22:42:27.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3010000000000007e-06 2024-09-17T22:42:27.618 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.618 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.618 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.619 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158675 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.619 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:27.619 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369556035000002, 2024-09-17T22:42:27.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039408033000000002, 2024-09-17T22:42:27.619 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.619 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.619 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.619 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.619 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158675 2024-09-17T22:42:27.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.619 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:27.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:27.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:27.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.36e-07 2024-09-17T22:42:27.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241572+0000", 2024-09-17T22:42:27.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6599999999999994e-07 2024-09-17T22:42:27.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241575+0000", 2024-09-17T22:42:27.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8389999999999998e-06 2024-09-17T22:42:27.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241577+0000", 2024-09-17T22:42:27.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4119999999999999e-06 2024-09-17T22:42:27.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280975+0000", 2024-09-17T22:42:27.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039397205999999997 2024-09-17T22:42:27.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280979+0000", 2024-09-17T22:42:27.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.374e-06 2024-09-17T22:42:27.623 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.623 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.623 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.623 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158677 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.624 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:27.624 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369533592000003, 2024-09-17T22:42:27.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039409307999999997, 2024-09-17T22:42:27.624 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.624 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.624 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.624 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.624 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.624 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.624 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158677 2024-09-17T22:42:27.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.624 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:27.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:27.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241594+0000", 2024-09-17T22:42:27.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:42:27.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241595+0000", 2024-09-17T22:42:27.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2729999999999999e-06 2024-09-17T22:42:27.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241602+0000", 2024-09-17T22:42:27.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:42:27.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241606+0000", 2024-09-17T22:42:27.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3019999999999999e-06 2024-09-17T22:42:27.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280994+0000", 2024-09-17T22:42:27.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039388484000000001 2024-09-17T22:42:27.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281003+0000", 2024-09-17T22:42:27.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5459999999999992e-06 2024-09-17T22:42:27.627 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.627 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.628 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.628 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158679 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.628 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:27.628 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369517913999999, 2024-09-17T22:42:27.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039413403999999999, 2024-09-17T22:42:27.628 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.628 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.628 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.628 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.628 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.628 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.629 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158679 2024-09-17T22:42:27.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.629 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:27.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:27.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:27.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4999999999999999e-07 2024-09-17T22:42:27.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241613+0000", 2024-09-17T22:42:27.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.207e-06 2024-09-17T22:42:27.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241616+0000", 2024-09-17T22:42:27.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.275e-06 2024-09-17T22:42:27.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241618+0000", 2024-09-17T22:42:27.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:42:27.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281017+0000", 2024-09-17T22:42:27.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039398385000000001 2024-09-17T22:42:27.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281022+0000", 2024-09-17T22:42:27.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.896e-06 2024-09-17T22:42:27.632 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.632 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.632 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.633 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158681 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.633 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:27.633 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369505736000001, 2024-09-17T22:42:27.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039425292000000001, 2024-09-17T22:42:27.633 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.633 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.633 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.633 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.633 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.633 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.633 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158681 2024-09-17T22:42:27.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.633 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:27.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:27.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:27.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:42:27.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241622+0000", 2024-09-17T22:42:27.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:42:27.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241625+0000", 2024-09-17T22:42:27.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9629999999999998e-06 2024-09-17T22:42:27.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241628+0000", 2024-09-17T22:42:27.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4339999999999999e-06 2024-09-17T22:42:27.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281039+0000", 2024-09-17T22:42:27.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039410672000000001 2024-09-17T22:42:27.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.636 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.636 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281047+0000", 2024-09-17T22:42:27.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9969999999999998e-06 2024-09-17T22:42:27.636 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.636 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.636 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.637 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158682 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.637 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:27.637 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369496452, 2024-09-17T22:42:27.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039436455000000002, 2024-09-17T22:42:27.637 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.637 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.637 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.637 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.637 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.637 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.637 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158682 2024-09-17T22:42:27.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.638 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:27.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:27.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:27.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:42:27.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241632+0000", 2024-09-17T22:42:27.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:42:27.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241635+0000", 2024-09-17T22:42:27.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8940000000000002e-06 2024-09-17T22:42:27.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241637+0000", 2024-09-17T22:42:27.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3659999999999999e-06 2024-09-17T22:42:27.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281061+0000", 2024-09-17T22:42:27.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039423664999999997 2024-09-17T22:42:27.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281067+0000", 2024-09-17T22:42:27.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.285e-06 2024-09-17T22:42:27.641 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.641 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.641 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.641 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158684 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.641 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:27.641 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369487106000001, 2024-09-17T22:42:27.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462980000000002, 2024-09-17T22:42:27.641 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.641 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.641 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.641 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.642 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.642 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.642 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158684 2024-09-17T22:42:27.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.642 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:27.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:27.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:27.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:27.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241641+0000", 2024-09-17T22:42:27.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:42:27.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241644+0000", 2024-09-17T22:42:27.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8870000000000001e-06 2024-09-17T22:42:27.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241646+0000", 2024-09-17T22:42:27.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3530000000000002e-06 2024-09-17T22:42:27.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281093+0000", 2024-09-17T22:42:27.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039446947000000003 2024-09-17T22:42:27.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281103+0000", 2024-09-17T22:42:27.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6299999999999993e-06 2024-09-17T22:42:27.645 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.645 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.646 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.646 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158686 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.646 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:27.646 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369477936000003, 2024-09-17T22:42:27.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039474214000000001, 2024-09-17T22:42:27.646 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.646 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.646 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.646 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.646 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.646 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.647 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158686 2024-09-17T22:42:27.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.647 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:27.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:27.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:27.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:42:27.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241650+0000", 2024-09-17T22:42:27.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2999999999999999e-07 2024-09-17T22:42:27.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241653+0000", 2024-09-17T22:42:27.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9689999999999999e-06 2024-09-17T22:42:27.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241655+0000", 2024-09-17T22:42:27.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3460000000000001e-06 2024-09-17T22:42:27.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281118+0000", 2024-09-17T22:42:27.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462876000000001 2024-09-17T22:42:27.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281123+0000", 2024-09-17T22:42:27.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8840000000000002e-06 2024-09-17T22:42:27.651 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.651 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.651 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.651 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158688 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.651 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:27.651 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369468705000003, 2024-09-17T22:42:27.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039490458999999999, 2024-09-17T22:42:27.651 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.651 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.651 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.652 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.652 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.652 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.652 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158688 2024-09-17T22:42:27.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.652 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:27.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:27.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:27.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:42:27.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241659+0000", 2024-09-17T22:42:27.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3699999999999999e-07 2024-09-17T22:42:27.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241662+0000", 2024-09-17T22:42:27.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8499999999999998e-06 2024-09-17T22:42:27.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241665+0000", 2024-09-17T22:42:27.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.356e-06 2024-09-17T22:42:27.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281141+0000", 2024-09-17T22:42:27.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039476031000000002 2024-09-17T22:42:27.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281149+0000", 2024-09-17T22:42:27.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0709999999999995e-06 2024-09-17T22:42:27.655 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.655 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.655 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.655 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158690 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.655 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:27.655 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369459646000003, 2024-09-17T22:42:27.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039502378999999997, 2024-09-17T22:42:27.656 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.656 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.656 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.656 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.656 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.656 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.656 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158690 2024-09-17T22:42:27.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.656 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:27.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:27.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241668+0000", 2024-09-17T22:42:27.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:27.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241669+0000", 2024-09-17T22:42:27.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1009999999999999e-06 2024-09-17T22:42:27.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241672+0000", 2024-09-17T22:42:27.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9059999999999998e-06 2024-09-17T22:42:27.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241674+0000", 2024-09-17T22:42:27.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4439999999999998e-06 2024-09-17T22:42:27.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281162+0000", 2024-09-17T22:42:27.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039488189 2024-09-17T22:42:27.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281170+0000", 2024-09-17T22:42:27.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5310000000000004e-06 2024-09-17T22:42:27.660 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.661 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.661 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.661 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.661 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158692 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.661 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:27.661 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369450213, 2024-09-17T22:42:27.661 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039531064999999997, 2024-09-17T22:42:27.661 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.661 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.661 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.661 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.662 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158692 2024-09-17T22:42:27.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.662 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:27.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.662 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.662 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.662 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:27.662 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:27.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:27.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241678+0000", 2024-09-17T22:42:27.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0800000000000003e-07 2024-09-17T22:42:27.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.663 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.663 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.663 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241681+0000", 2024-09-17T22:42:27.663 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8729999999999999e-06 2024-09-17T22:42:27.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241683+0000", 2024-09-17T22:42:27.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.447e-06 2024-09-17T22:42:27.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.664 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281199+0000", 2024-09-17T22:42:27.664 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039515388999999998 2024-09-17T22:42:27.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.664 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.664 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.665 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281208+0000", 2024-09-17T22:42:27.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2399999999999996e-06 2024-09-17T22:42:27.665 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.665 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.665 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.665 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.665 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158694 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.665 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:27.665 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369441041999998, 2024-09-17T22:42:27.665 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039543986000000003, 2024-09-17T22:42:27.665 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.665 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.665 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.666 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.666 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.666 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.666 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158694 2024-09-17T22:42:27.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.666 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:27.666 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.666 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.666 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.666 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:27.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:27.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:42:27.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.667 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.667 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241687+0000", 2024-09-17T22:42:27.667 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:42:27.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241691+0000", 2024-09-17T22:42:27.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.811e-06 2024-09-17T22:42:27.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241693+0000", 2024-09-17T22:42:27.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.385e-06 2024-09-17T22:42:27.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.668 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.668 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.668 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281223+0000", 2024-09-17T22:42:27.668 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039529332 2024-09-17T22:42:27.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.669 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.669 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281230+0000", 2024-09-17T22:42:27.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2139999999999999e-06 2024-09-17T22:42:27.669 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.669 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.669 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.669 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.669 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158696 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.669 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:27.669 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369430653999999, 2024-09-17T22:42:27.669 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039570458000000003, 2024-09-17T22:42:27.669 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.670 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.670 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.670 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.670 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.670 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.670 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158696 2024-09-17T22:42:27.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.670 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.670 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.670 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.670 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:27.670 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:27.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:42:27.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:42:27.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.671 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.671 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.671 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:42:27.671 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.33e-07 2024-09-17T22:42:27.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241700+0000", 2024-09-17T22:42:27.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8949999999999998e-06 2024-09-17T22:42:27.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.672 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241703+0000", 2024-09-17T22:42:27.672 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3159999999999999e-06 2024-09-17T22:42:27.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.672 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.672 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281257+0000", 2024-09-17T22:42:27.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039553862000000002 2024-09-17T22:42:27.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.673 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.673 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.673 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281267+0000", 2024-09-17T22:42:27.673 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.024e-05 2024-09-17T22:42:27.673 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.673 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.673 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.673 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.674 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.674 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158698 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.674 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:27.674 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369420576000003, 2024-09-17T22:42:27.674 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039576493999999997, 2024-09-17T22:42:27.674 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.674 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.674 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.674 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.674 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.674 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.674 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158698 2024-09-17T22:42:27.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.675 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:27.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:27.675 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.675 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.675 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241707+0000", 2024-09-17T22:42:27.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:42:27.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241708+0000", 2024-09-17T22:42:27.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5000000000000001e-07 2024-09-17T22:42:27.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.676 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.676 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241711+0000", 2024-09-17T22:42:27.676 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9670000000000001e-06 2024-09-17T22:42:27.676 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.676 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241713+0000", 2024-09-17T22:42:27.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2689999999999998e-06 2024-09-17T22:42:27.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281278+0000", 2024-09-17T22:42:27.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039565206999999998 2024-09-17T22:42:27.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.677 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.677 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.677 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281283+0000", 2024-09-17T22:42:27.677 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:42:27.678 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.678 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.678 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.678 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.678 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158700 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.678 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:27.678 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369411364999998, 2024-09-17T22:42:27.678 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039604292999999999, 2024-09-17T22:42:27.678 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.678 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.678 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.678 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.678 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.679 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.679 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158700 2024-09-17T22:42:27.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.679 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.679 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.679 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.679 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:27.679 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:27.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:27.680 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:27.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.680 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.680 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241717+0000", 2024-09-17T22:42:27.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.78e-07 2024-09-17T22:42:27.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241720+0000", 2024-09-17T22:42:27.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8119999999999999e-06 2024-09-17T22:42:27.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.681 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.681 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241722+0000", 2024-09-17T22:42:27.681 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-17T22:42:27.681 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.681 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281311+0000", 2024-09-17T22:42:27.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039588894999999999 2024-09-17T22:42:27.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.682 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.682 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.682 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281320+0000", 2024-09-17T22:42:27.682 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9989999999999997e-06 2024-09-17T22:42:27.682 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.682 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.682 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.683 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.683 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158702 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.683 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:27.683 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369400042999999, 2024-09-17T22:42:27.683 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039616142, 2024-09-17T22:42:27.683 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.683 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.683 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.683 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.683 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.683 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.683 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158702 2024-09-17T22:42:27.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.684 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:27.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.684 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:27.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.684 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.684 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:27.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4400000000000001e-07 2024-09-17T22:42:27.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241728+0000", 2024-09-17T22:42:27.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0839999999999999e-06 2024-09-17T22:42:27.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241731+0000", 2024-09-17T22:42:27.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0690000000000002e-06 2024-09-17T22:42:27.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241734+0000", 2024-09-17T22:42:27.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4159999999999998e-06 2024-09-17T22:42:27.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281334+0000", 2024-09-17T22:42:27.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039600110000000001 2024-09-17T22:42:27.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281343+0000", 2024-09-17T22:42:27.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2189999999999993e-06 2024-09-17T22:42:27.687 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.687 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.687 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.687 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158704 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.687 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:27.687 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369390179, 2024-09-17T22:42:27.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039641485999999997, 2024-09-17T22:42:27.687 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.687 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.687 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.687 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.688 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.688 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.688 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158704 2024-09-17T22:42:27.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.688 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:27.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:27.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:27.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:42:27.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.689 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241738+0000", 2024-09-17T22:42:27.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0589999999999999e-06 2024-09-17T22:42:27.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241741+0000", 2024-09-17T22:42:27.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.943e-06 2024-09-17T22:42:27.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241744+0000", 2024-09-17T22:42:27.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4819999999999999e-06 2024-09-17T22:42:27.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281370+0000", 2024-09-17T22:42:27.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039625661 2024-09-17T22:42:27.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281378+0000", 2024-09-17T22:42:27.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1310000000000006e-06 2024-09-17T22:42:27.691 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.691 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.691 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.691 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158705 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.691 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:27.691 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369379629000001, 2024-09-17T22:42:27.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039652637999999997, 2024-09-17T22:42:27.691 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.692 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.692 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.692 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.692 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.692 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.692 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158705 2024-09-17T22:42:27.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.692 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:27.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:27.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:42:27.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-07 2024-09-17T22:42:27.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:42:27.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0599999999999999e-07 2024-09-17T22:42:27.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241751+0000", 2024-09-17T22:42:27.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.926e-06 2024-09-17T22:42:27.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241754+0000", 2024-09-17T22:42:27.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4360000000000001e-06 2024-09-17T22:42:27.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281392+0000", 2024-09-17T22:42:27.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039638566 2024-09-17T22:42:27.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281400+0000", 2024-09-17T22:42:27.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.571e-06 2024-09-17T22:42:27.695 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.695 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.695 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.696 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158707 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.696 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:27.696 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369358527999999, 2024-09-17T22:42:27.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039654809999999999, 2024-09-17T22:42:27.696 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.696 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.696 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.696 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.696 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.696 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.696 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158707 2024-09-17T22:42:27.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.697 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:27.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:27.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241769+0000", 2024-09-17T22:42:27.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4299999999999999e-07 2024-09-17T22:42:27.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241770+0000", 2024-09-17T22:42:27.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3769999999999999e-06 2024-09-17T22:42:27.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.698 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241773+0000", 2024-09-17T22:42:27.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2109999999999998e-06 2024-09-17T22:42:27.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241776+0000", 2024-09-17T22:42:27.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5069999999999999e-06 2024-09-17T22:42:27.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281415+0000", 2024-09-17T22:42:27.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039639499000000002 2024-09-17T22:42:27.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281423+0000", 2024-09-17T22:42:27.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8730000000000007e-06 2024-09-17T22:42:27.699 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.700 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.700 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.700 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158709 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:27.700 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:27.700 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.369347918999999, 2024-09-17T22:42:27.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039661937000000001, 2024-09-17T22:42:27.700 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:27.700 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:27.700 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:27.700 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:27.700 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:27.701 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:27.701 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158709 2024-09-17T22:42:27.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.701 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:27.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:27.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:27.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:27.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:27.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:27.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:27.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242843+0000", 2024-09-17T22:42:27.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010634780000000001 2024-09-17T22:42:27.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:27.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242845+0000", 2024-09-17T22:42:27.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7209999999999999e-06 2024-09-17T22:42:27.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.702 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:27.702 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242851+0000", 2024-09-17T22:42:27.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6949999999999997e-06 2024-09-17T22:42:27.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:27.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242855+0000", 2024-09-17T22:42:27.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7960000000000002e-06 2024-09-17T22:42:27.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:27.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281434+0000", 2024-09-17T22:42:27.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038579519 2024-09-17T22:42:27.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:27.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:27.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:27.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281441+0000", 2024-09-17T22:42:27.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7279999999999998e-06 2024-09-17T22:42:27.704 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.704 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.704 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.704 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:27.704 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:27.704 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:27.706 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:42:27.707 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:27.703+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:27.708 INFO:teuthology.orchestra.run.smithi115.stdout:4952 write 0x35f3bc3 thru 0x35ff70c (0xbb4a bytes) 2024-09-17T22:42:27.708 INFO:teuthology.orchestra.run.smithi115.stdout:4953 trunc from 0x37bb7d4 to 0xed8bbd1 2024-09-17T22:42:27.709 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:27.709 INFO:teuthology.orchestra.run.smithi115.stdout:4955 write 0x6bd26e2 thru 0x6be1988 (0xf2a7 bytes) 2024-09-17T22:42:27.709 INFO:teuthology.orchestra.run.smithi115.stdout:4956 write 0x996e772 thru 0x997dc9c (0xf52b bytes) 2024-09-17T22:42:27.709 INFO:teuthology.orchestra.run.smithi115.stdout:4957 write 0xdeada6a thru 0xdebb285 (0xd81c bytes) 2024-09-17T22:42:27.709 INFO:teuthology.orchestra.run.smithi115.stdout:4959 punch from 0x2919313 to 0x2920c94, (0x7981 bytes) 2024-09-17T22:42:27.709 INFO:teuthology.orchestra.run.smithi115.stdout:4960 punch from 0xa71cac7 to 0xa722f11, (0x644a bytes) 2024-09-17T22:42:27.709 INFO:teuthology.orchestra.run.smithi115.stdout:4961 trunc from 0xed8bbd1 to 0xd4ba005 2024-09-17T22:42:27.709 INFO:teuthology.orchestra.run.smithi115.stdout:4962 read 0x8149157 thru 0x81544c6 (0xb370 bytes) 2024-09-17T22:42:27.709 INFO:teuthology.orchestra.run.smithi115.stdout:4965 write 0x45ae67b thru 0x45b4fc3 (0x6949 bytes) 2024-09-17T22:42:27.709 INFO:teuthology.orchestra.run.smithi115.stdout:4966 write 0xd457e16 thru 0xd4647a0 (0xc98b bytes) 2024-09-17T22:42:27.709 INFO:teuthology.orchestra.run.smithi115.stdout:4968 punch from 0x6f57c62 to 0x6f5e716, (0x6ab4 bytes) 2024-09-17T22:42:27.709 INFO:teuthology.orchestra.run.smithi115.stdout:4969 read 0xcbd4fa5 thru 0xcbdaf7d (0x5fd9 bytes) 2024-09-17T22:42:27.709 INFO:teuthology.orchestra.run.smithi115.stdout:4970 read 0x3603c4b thru 0x36105fc (0xc9b2 bytes) 2024-09-17T22:42:27.709 INFO:teuthology.orchestra.run.smithi115.stdout:4971 read 0xbf57e11 thru 0xbf5eaa7 (0x6c97 bytes) 2024-09-17T22:42:27.710 INFO:teuthology.orchestra.run.smithi115.stdout:4972 writesame 0xab0cf63 thru 0xab141c8 data_size 0x9(0x7266 bytes) 2024-09-17T22:42:27.710 INFO:teuthology.orchestra.run.smithi115.stdout:4973 punch from 0x8804152 to 0x8808ccb, (0x4b79 bytes) 2024-09-17T22:42:27.710 INFO:teuthology.orchestra.run.smithi115.stdout:4974 read 0x7242f03 thru 0x724bff4 (0x90f2 bytes) 2024-09-17T22:42:27.710 INFO:teuthology.orchestra.run.smithi115.stdout:4975 read 0xcfeeabf thru 0xcffa0c1 (0xb603 bytes) 2024-09-17T22:42:27.710 INFO:teuthology.orchestra.run.smithi115.stdout:4976 trunc from 0xd4ba005 to 0x874a77f 2024-09-17T22:42:27.710 INFO:teuthology.orchestra.run.smithi115.stdout:4978 write 0x7ba0d6 thru 0x7c25a7 (0x84d2 bytes) 2024-09-17T22:42:27.710 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:27.710 INFO:teuthology.orchestra.run.smithi115.stdout:4980 read 0x38000b6 thru 0x3804e98 (0x4de3 bytes) 2024-09-17T22:42:27.710 INFO:teuthology.orchestra.run.smithi115.stdout:4981 read 0x69f6760 thru 0x69fb9cc (0x526d bytes) 2024-09-17T22:42:27.710 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:27.711 INFO:teuthology.orchestra.run.smithi115.stdout:4983 write 0xbca0b68 thru 0xbcaaec4 (0xa35d bytes) 2024-09-17T22:42:27.711 INFO:teuthology.orchestra.run.smithi115.stdout:4985 read 0x9654d97 thru 0x965fcc0 (0xaf2a bytes) 2024-09-17T22:42:27.711 INFO:teuthology.orchestra.run.smithi115.stdout:4987 write 0x8219ca4 thru 0x82284d0 (0xe82d bytes) 2024-09-17T22:42:27.711 INFO:teuthology.orchestra.run.smithi115.stdout:4988 write 0x28d6873 thru 0x28df6fe (0x8e8c bytes) 2024-09-17T22:42:27.711 INFO:teuthology.orchestra.run.smithi115.stdout:4989 trunc from 0xbcaaec5 to 0xb94ed30 2024-09-17T22:42:27.717 INFO:teuthology.orchestra.run.smithi115.stdout:4990 write 0x9df9b51 thru 0x9dfec64 (0x5114 bytes) 2024-09-17T22:42:27.719 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:27.720 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:27.720 INFO:teuthology.orchestra.run.smithi115.stdout:4991 trunc from 0xb94ed30 to 0xb26054f 2024-09-17T22:42:27.729 INFO:teuthology.orchestra.run.smithi115.stdout:4993 write 0x965e99b thru 0x96640f7 (0x575d bytes) 2024-09-17T22:42:27.732 INFO:teuthology.orchestra.run.smithi115.stdout:4994 write 0x618352b thru 0x618e8b5 (0xb38b bytes) 2024-09-17T22:42:27.733 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:27.734 INFO:teuthology.orchestra.run.smithi115.stdout:4996 trunc from 0xb26054f to 0xc4ed718 2024-09-17T22:42:27.738 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:27.738 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:27.740 INFO:teuthology.orchestra.run.smithi115.stdout:4998 write 0xd1f7d5 thru 0xd24087 (0x48b3 bytes) 2024-09-17T22:42:27.742 INFO:teuthology.orchestra.run.smithi115.stdout:4999 read 0x7f651bc thru 0x7f6c75a (0x759f bytes) 2024-09-17T22:42:27.742 INFO:teuthology.orchestra.run.smithi115.stdout:5000 write 0x80ef99d thru 0x80fcdbe (0xd422 bytes) 2024-09-17T22:42:27.759 INFO:teuthology.orchestra.run.smithi115.stdout:5001 trunc from 0xc4ed718 to 0x11df7bd 2024-09-17T22:42:27.765 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:27.765 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:27.776 INFO:teuthology.orchestra.run.smithi115.stdout:5006 punch from 0xea03a0 to 0xea1234, (0xe94 bytes) 2024-09-17T22:42:27.788 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:27.789 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:27.791 INFO:teuthology.orchestra.run.smithi115.stdout:5007 write 0x7470abd thru 0x74768ff (0x5e43 bytes) 2024-09-17T22:42:27.794 INFO:teuthology.orchestra.run.smithi115.stdout:5008 read 0x17a55ae thru 0x17b32e4 (0xdd37 bytes) 2024-09-17T22:42:27.794 INFO:teuthology.orchestra.run.smithi115.stdout:5010 read 0x644d48 thru 0x65124a (0xc503 bytes) 2024-09-17T22:42:27.795 INFO:teuthology.orchestra.run.smithi115.stdout:5011 read 0x50dd30b thru 0x50e5599 (0x828f bytes) 2024-09-17T22:42:27.795 INFO:teuthology.orchestra.run.smithi115.stdout:5012 write 0x5254757 thru 0x5260ac2 (0xc36c bytes) 2024-09-17T22:42:27.797 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:27.804 INFO:teuthology.orchestra.run.smithi115.stdout:5014 trunc from 0x7476900 to 0x1a41f1f 2024-09-17T22:42:27.806 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:27.804+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:27.812 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:27.812 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:27.813 INFO:teuthology.orchestra.run.smithi115.stdout:5015 punch from 0x12d7ff8 to 0x12da490, (0x2498 bytes) 2024-09-17T22:42:27.825 INFO:teuthology.orchestra.run.smithi115.stdout:5016 trunc from 0x1a41f1f to 0x9cb69b7 2024-09-17T22:42:27.832 INFO:teuthology.orchestra.run.smithi115.stdout:5018 write 0x6cff7c thru 0x6dec28 (0xecad bytes) 2024-09-17T22:42:27.839 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:27.839 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:27.840 INFO:teuthology.orchestra.run.smithi115.stdout:5019 write 0xa9a2fc9 thru 0xa9a6a2f (0x3a67 bytes) 2024-09-17T22:42:27.843 INFO:teuthology.orchestra.run.smithi115.stdout:5020 writesame 0x40d9328 thru 0x40da3ad data_size 0x3(0x1086 bytes) 2024-09-17T22:42:27.844 INFO:teuthology.orchestra.run.smithi115.stdout:5021 read 0x9b06eea thru 0x9b0bf0d (0x5024 bytes) 2024-09-17T22:42:27.844 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:27.850 INFO:teuthology.orchestra.run.smithi115.stdout:5025 writesame 0xaf213d7 thru 0xaf27fa8 data_size 0x25(0x6bd2 bytes) 2024-09-17T22:42:27.857 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:27.857 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:27.858 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:27.858 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:27.860 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:27.860 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:27.861 INFO:teuthology.orchestra.run.smithi115.stdout:5028 write 0xe337add thru 0xe34603a (0xe55e bytes) 2024-09-17T22:42:27.864 INFO:teuthology.orchestra.run.smithi115.stdout:5031 punch from 0x18f01a4 to 0x18fbbfb, (0xba57 bytes) 2024-09-17T22:42:27.864 INFO:teuthology.orchestra.run.smithi115.stdout:5032 punch from 0xd039eb7 to 0xd0455aa, (0xb6f3 bytes) 2024-09-17T22:42:27.864 INFO:teuthology.orchestra.run.smithi115.stdout:5034 writesame 0x781f9a6 thru 0x78217a1 data_size 0x26(0x1dfc bytes) 2024-09-17T22:42:27.868 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:42:27.870 INFO:teuthology.orchestra.run.smithi115.stdout:5035 trunc from 0xe34603b to 0xbfde933 2024-09-17T22:42:27.878 INFO:teuthology.orchestra.run.smithi115.stdout:5036 read 0x3f7453a thru 0x3f80217 (0xbcde bytes) 2024-09-17T22:42:27.878 INFO:teuthology.orchestra.run.smithi115.stdout:5037 writesame 0x2ade669 thru 0x2aedc60 data_size 0x44(0xf5f8 bytes) 2024-09-17T22:42:27.891 INFO:teuthology.orchestra.run.smithi115.stdout:5038 trunc from 0xbfde933 to 0x3d6f92f 2024-09-17T22:42:27.907 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:27.904+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:27.917 INFO:teuthology.orchestra.run.smithi115.stdout:5039 writesame 0x9e56427 thru 0x9e63dd8 data_size 0x5(0xd9b2 bytes) 2024-09-17T22:42:27.926 INFO:teuthology.orchestra.run.smithi115.stdout:5040 writesame 0xca8b4ee thru 0xca8dfef data_size 0xf(0x2b02 bytes) 2024-09-17T22:42:27.926 INFO:teuthology.orchestra.run.smithi115.stdout:5041 read 0x680dc8c thru 0x681862b (0xa9a0 bytes) 2024-09-17T22:42:27.926 INFO:teuthology.orchestra.run.smithi115.stdout:5042 read 0x870efa7 thru 0x8715142 (0x619c bytes) 2024-09-17T22:42:27.928 INFO:teuthology.orchestra.run.smithi115.stdout:5043 trunc from 0xca8dff0 to 0xdfc90a0 2024-09-17T22:42:27.933 INFO:teuthology.orchestra.run.smithi115.stdout:5044 writesame 0xb0d0099 thru 0xb0d7850 data_size 0xc(0x77b8 bytes) 2024-09-17T22:42:27.937 INFO:teuthology.orchestra.run.smithi115.stdout:5045 write 0x707c196 thru 0x70886ca (0xc535 bytes) 2024-09-17T22:42:27.940 INFO:teuthology.orchestra.run.smithi115.stdout:5047 write 0x82b8e14 thru 0x82bd7e9 (0x49d6 bytes) 2024-09-17T22:42:27.943 INFO:teuthology.orchestra.run.smithi115.stdout:5048 punch from 0xb9e3d0d to 0xb9eaf20, (0x7213 bytes) 2024-09-17T22:42:27.943 INFO:teuthology.orchestra.run.smithi115.stdout:5050 write 0x82de4a9 thru 0x82ee1db (0xfd33 bytes) 2024-09-17T22:42:27.944 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:27.945 INFO:teuthology.orchestra.run.smithi115.stdout:5052 read 0x29b152c thru 0x29b65db (0x50b0 bytes) 2024-09-17T22:42:27.945 INFO:teuthology.orchestra.run.smithi115.stdout:5053 read 0x2317958 thru 0x231a890 (0x2f39 bytes) 2024-09-17T22:42:27.945 INFO:teuthology.orchestra.run.smithi115.stdout:5055 read 0x5692432 thru 0x56a0dbc (0xe98b bytes) 2024-09-17T22:42:27.946 INFO:teuthology.orchestra.run.smithi115.stdout:5056 write 0xdda5f98 thru 0xddaec93 (0x8cfc bytes) 2024-09-17T22:42:27.949 INFO:teuthology.orchestra.run.smithi115.stdout:5057 write 0xb810ff4 thru 0xb81eea7 (0xdeb4 bytes) 2024-09-17T22:42:27.957 INFO:teuthology.orchestra.run.smithi115.stdout:5058 write 0x7880917 thru 0x789079a (0xfe84 bytes) 2024-09-17T22:42:27.958 INFO:teuthology.orchestra.run.smithi115.stdout:5059 writesame 0x8af8ad7 thru 0x8afa26c data_size 0x1(0x1796 bytes) 2024-09-17T22:42:27.964 INFO:teuthology.orchestra.run.smithi115.stdout:5060 punch from 0x558d168 to 0x5590af2, (0x398a bytes) 2024-09-17T22:42:27.964 INFO:teuthology.orchestra.run.smithi115.stdout:5061 read 0x6681a0e thru 0x668404f (0x2642 bytes) 2024-09-17T22:42:27.964 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:27.964 INFO:teuthology.orchestra.run.smithi115.stdout:5063 trunc from 0xdfc90a0 to 0xd6631d4 2024-09-17T22:42:27.972 INFO:teuthology.orchestra.run.smithi115.stdout:5064 writesame 0xb22e876 thru 0xb2338ad data_size 0x22(0x5038 bytes) 2024-09-17T22:42:27.973 INFO:teuthology.orchestra.run.smithi115.stdout:5065 read 0x76ed3a0 thru 0x76f6d40 (0x99a1 bytes) 2024-09-17T22:42:27.973 INFO:teuthology.orchestra.run.smithi115.stdout:5066 punch from 0x707a7d3 to 0x7084321, (0x9b4e bytes) 2024-09-17T22:42:27.974 INFO:teuthology.orchestra.run.smithi115.stdout:5068 write 0x3101540 thru 0x310d2ae (0xbd6f bytes) 2024-09-17T22:42:27.975 INFO:teuthology.orchestra.run.smithi115.stdout:5069 read 0x490b43b thru 0x4915de1 (0xa9a7 bytes) 2024-09-17T22:42:27.975 INFO:teuthology.orchestra.run.smithi115.stdout:5070 read 0x2e07d07 thru 0x2e1400f (0xc309 bytes) 2024-09-17T22:42:27.976 INFO:teuthology.orchestra.run.smithi115.stdout:5071 write 0x2ae41fb thru 0x2aeff72 (0xbd78 bytes) 2024-09-17T22:42:27.978 INFO:teuthology.orchestra.run.smithi115.stdout:5073 read 0x2ff37e2 thru 0x2ff5cd7 (0x24f6 bytes) 2024-09-17T22:42:27.979 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:27.979 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:27.988 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:42:27.989 INFO:teuthology.orchestra.run.smithi115.stdout:5074 trunc from 0xd6631d4 to 0x3bd3610 2024-09-17T22:42:28.005 INFO:teuthology.orchestra.run.smithi115.stdout:5075 read 0x2e9db9f thru 0x2e9f5b5 (0x1a17 bytes) 2024-09-17T22:42:28.005 INFO:teuthology.orchestra.run.smithi115.stdout:5076 write 0x11b8414 thru 0x11c0765 (0x8352 bytes) 2024-09-17T22:42:28.021 INFO:teuthology.orchestra.run.smithi115.stdout:5077 write 0x9f93d10 thru 0x9f9e1fc (0xa4ed bytes) 2024-09-17T22:42:28.024 INFO:teuthology.orchestra.run.smithi115.stdout:5078 read 0x10cc2d1 thru 0x10d1fdf (0x5d0f bytes) 2024-09-17T22:42:28.025 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.029 INFO:teuthology.orchestra.run.smithi115.stdout:5080 trunc from 0x9f9e1fd to 0x6d13467 2024-09-17T22:42:28.031 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:28.029+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:28.040 INFO:teuthology.orchestra.run.smithi115.stdout:5081 read 0x372de06 thru 0x373d783 (0xf97e bytes) 2024-09-17T22:42:28.048 INFO:teuthology.orchestra.run.smithi115.stdout:5083 trunc from 0x6d13467 to 0xd3b5c34 2024-09-17T22:42:28.056 INFO:teuthology.orchestra.run.smithi115.stdout:5084 write 0x5d60e93 thru 0x5d67c40 (0x6dae bytes) 2024-09-17T22:42:28.059 INFO:teuthology.orchestra.run.smithi115.stdout:5089 writesame 0x92d7a85 thru 0x92e176f data_size 0x11(0x9ceb bytes) 2024-09-17T22:42:28.064 INFO:teuthology.orchestra.run.smithi115.stdout:5090 trunc from 0xd3b5c34 to 0xc474ead 2024-09-17T22:42:28.072 INFO:teuthology.orchestra.run.smithi115.stdout:5091 write 0xbb042ef thru 0xbb0d9ec (0x96fe bytes) 2024-09-17T22:42:28.077 INFO:teuthology.orchestra.run.smithi115.stdout:5092 read 0x66e80d9 thru 0x66ea52a (0x2452 bytes) 2024-09-17T22:42:28.077 INFO:teuthology.orchestra.run.smithi115.stdout:5093 punch from 0xea4b05 to 0xeac0ef, (0x75ea bytes) 2024-09-17T22:42:28.077 INFO:teuthology.orchestra.run.smithi115.stdout:5094 punch from 0xad254c5 to 0xad2e37a, (0x8eb5 bytes) 2024-09-17T22:42:28.077 INFO:teuthology.orchestra.run.smithi115.stdout:5095 write 0x6846e88 thru 0x684b31e (0x4497 bytes) 2024-09-17T22:42:28.078 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.082 INFO:teuthology.orchestra.run.smithi115.stdout:5097 trunc from 0xc474ead to 0x981ed19 2024-09-17T22:42:28.101 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:28.101 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:28.104 INFO:teuthology.orchestra.run.smithi115.stdout:5098 write 0xc3b81ce thru 0xc3c4fb2 (0xcde5 bytes) 2024-09-17T22:42:28.108 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.108 INFO:teuthology.orchestra.run.smithi115.stdout:5101 punch from 0xba4ed73 to 0xba5c1fc, (0xd489 bytes) 2024-09-17T22:42:28.108 INFO:teuthology.orchestra.run.smithi115.stdout:5104 write 0x72fbe6 thru 0x7333fa (0x3815 bytes) 2024-09-17T22:42:28.108 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:28.105+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:28.109 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.109 INFO:teuthology.orchestra.run.smithi115.stdout:5106 read 0x300e36a thru 0x301b0ad (0xcd44 bytes) 2024-09-17T22:42:28.109 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.111 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:42:28.111 INFO:teuthology.orchestra.run.smithi115.stdout:5108 trunc from 0xc3c4fb3 to 0xae2db3f 2024-09-17T22:42:28.129 INFO:teuthology.orchestra.run.smithi115.stdout:5109 write 0xd526d54 thru 0xd528bbe (0x1e6b bytes) 2024-09-17T22:42:28.131 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.131 INFO:teuthology.orchestra.run.smithi115.stdout:5112 write 0x9c80298 thru 0x9c8da52 (0xd7bb bytes) 2024-09-17T22:42:28.134 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.134 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.134 INFO:teuthology.orchestra.run.smithi115.stdout:5115 write 0xadeb120 thru 0xadec9fd (0x18de bytes) 2024-09-17T22:42:28.137 INFO:teuthology.orchestra.run.smithi115.stdout:5116 writesame 0x5d853bf thru 0x5d9108d data_size 0x7(0xbccf bytes) 2024-09-17T22:42:28.140 INFO:teuthology.orchestra.run.smithi115.stdout:5117 read 0xcfbeee8 thru 0xcfcc60f (0xd728 bytes) 2024-09-17T22:42:28.140 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.140 INFO:teuthology.orchestra.run.smithi115.stdout:5119 read 0x21bd598 thru 0x21c9a7c (0xc4e5 bytes) 2024-09-17T22:42:28.140 INFO:teuthology.orchestra.run.smithi115.stdout:5121 read 0xb7dd303 thru 0xb7deab9 (0x17b7 bytes) 2024-09-17T22:42:28.140 INFO:teuthology.orchestra.run.smithi115.stdout:5122 write 0xc07c6a0 thru 0xc082950 (0x62b1 bytes) 2024-09-17T22:42:28.141 INFO:teuthology.orchestra.run.smithi115.stdout:5123 punch from 0x434ecc7 to 0x434f81c, (0xb55 bytes) 2024-09-17T22:42:28.142 INFO:teuthology.orchestra.run.smithi115.stdout:5124 write 0x52175f6 thru 0x522275e (0xb169 bytes) 2024-09-17T22:42:28.143 INFO:teuthology.orchestra.run.smithi115.stdout:5126 write 0x62ded0b thru 0x62e5f8a (0x7280 bytes) 2024-09-17T22:42:28.146 INFO:teuthology.orchestra.run.smithi115.stdout:5127 read 0xd51e3f6 thru 0xd52436b (0x5f76 bytes) 2024-09-17T22:42:28.146 INFO:teuthology.orchestra.run.smithi115.stdout:5128 read 0x49c23e3 thru 0x49c412a (0x1d48 bytes) 2024-09-17T22:42:28.147 INFO:teuthology.orchestra.run.smithi115.stdout:5129 read 0x96c026 thru 0x96d898 (0x1873 bytes) 2024-09-17T22:42:28.147 INFO:teuthology.orchestra.run.smithi115.stdout:5130 read 0xae93ecb thru 0xae9da82 (0x9bb8 bytes) 2024-09-17T22:42:28.147 INFO:teuthology.orchestra.run.smithi115.stdout:5132 punch from 0x4626241 to 0x4626a79, (0x838 bytes) 2024-09-17T22:42:28.147 INFO:teuthology.orchestra.run.smithi115.stdout:5133 read 0x52e20bd thru 0x52e646c (0x43b0 bytes) 2024-09-17T22:42:28.155 INFO:teuthology.orchestra.run.smithi115.stdout:5134 trunc from 0xd528bbf to 0x679ff0d 2024-09-17T22:42:28.168 INFO:teuthology.orchestra.run.smithi115.stdout:5136 read 0x1f81210 thru 0x1f8afe4 (0x9dd5 bytes) 2024-09-17T22:42:28.184 INFO:teuthology.orchestra.run.smithi115.stdout:5137 writesame 0xe9d6705 thru 0xe9da332 data_size 0x1(0x3c2e bytes) 2024-09-17T22:42:28.184 INFO:teuthology.orchestra.run.smithi115.stdout:5140 read 0x4999131 thru 0x49a0caf (0x7b7f bytes) 2024-09-17T22:42:28.185 INFO:teuthology.orchestra.run.smithi115.stdout:5143 punch from 0x77c3c5c to 0x77d0634, (0xc9d8 bytes) 2024-09-17T22:42:28.185 INFO:teuthology.orchestra.run.smithi115.stdout:5146 read 0x26f12b6 thru 0x26f9b1f (0x886a bytes) 2024-09-17T22:42:28.185 INFO:teuthology.orchestra.run.smithi115.stdout:5147 write 0x2d86cb5 thru 0x2d8af8c (0x42d8 bytes) 2024-09-17T22:42:28.187 INFO:teuthology.orchestra.run.smithi115.stdout:5149 write 0xcae3989 thru 0xcae3af1 (0x169 bytes) 2024-09-17T22:42:28.201 INFO:teuthology.orchestra.run.smithi115.stdout:5150 trunc from 0xe9da333 to 0x5267b8a 2024-09-17T22:42:28.209 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:28.206+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:28.213 INFO:teuthology.orchestra.run.smithi115.stdout:5152 read 0x1408de0 thru 0x140dd75 (0x4f96 bytes) 2024-09-17T22:42:28.213 INFO:teuthology.orchestra.run.smithi115.stdout:5154 punch from 0x43aa06a to 0x43b24bd, (0x8453 bytes) 2024-09-17T22:42:28.216 INFO:teuthology.orchestra.run.smithi115.stdout:5155 trunc from 0x5267b8a to 0x3031828 2024-09-17T22:42:28.226 INFO:teuthology.orchestra.run.smithi115.stdout:5157 read 0x2ce33c8 thru 0x2cf001f (0xcc58 bytes) 2024-09-17T22:42:28.232 INFO:teuthology.orchestra.run.smithi115.stdout:5158 writesame 0x3181595 thru 0x318e4bf data_size 0x1(0xcf2b bytes) 2024-09-17T22:42:28.247 INFO:teuthology.orchestra.run.smithi115.stdout:5160 write 0xa128fe6 thru 0xa137c6e (0xec89 bytes) 2024-09-17T22:42:28.250 INFO:teuthology.orchestra.run.smithi115.stdout:5161 writesame 0x147a2d2 thru 0x148677e data_size 0xd(0xc4ad bytes) 2024-09-17T22:42:28.252 INFO:teuthology.orchestra.run.smithi115.stdout:5163 writesame 0x49b1797 thru 0x49b8429 data_size 0x55(0x6c93 bytes) 2024-09-17T22:42:28.254 INFO:teuthology.orchestra.run.smithi115.stdout:5164 read 0x745b062 thru 0x745e1d1 (0x3170 bytes) 2024-09-17T22:42:28.261 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:28.261 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:28.263 INFO:teuthology.orchestra.run.smithi115.stdout:5165 writesame 0xc652211 thru 0xc65238d data_size 0x1(0x17d bytes) 2024-09-17T22:42:28.263 INFO:teuthology.orchestra.run.smithi115.stdout:5166 writesame 0x5ef46c1 thru 0x5f03f50 data_size 0x29(0xf890 bytes) 2024-09-17T22:42:28.264 INFO:teuthology.orchestra.run.smithi115.stdout:5167 read 0x2206409 thru 0x2211533 (0xb12b bytes) 2024-09-17T22:42:28.266 INFO:teuthology.orchestra.run.smithi115.stdout:5168 trunc from 0xc65238e to 0xb83f4e2 2024-09-17T22:42:28.271 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:42:28.272 INFO:teuthology.orchestra.run.smithi115.stdout:5169 read 0x7a4e790 thru 0x7a578c2 (0x9133 bytes) 2024-09-17T22:42:28.272 INFO:teuthology.orchestra.run.smithi115.stdout:5170 write 0x5465081 thru 0x546d629 (0x85a9 bytes) 2024-09-17T22:42:28.274 INFO:teuthology.orchestra.run.smithi115.stdout:5173 punch from 0x1791933 to 0x1799fc2, (0x868f bytes) 2024-09-17T22:42:28.274 INFO:teuthology.orchestra.run.smithi115.stdout:5174 read 0x2a8900f thru 0x2a8e499 (0x548b bytes) 2024-09-17T22:42:28.274 INFO:teuthology.orchestra.run.smithi115.stdout:5176 punch from 0x81cdd9a to 0x81da409, (0xc66f bytes) 2024-09-17T22:42:28.274 INFO:teuthology.orchestra.run.smithi115.stdout:5177 writesame 0x1a2d13f thru 0x1a3653b data_size 0x1(0x93fd bytes) 2024-09-17T22:42:28.281 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is unset 2024-09-17T22:42:28.288 INFO:teuthology.orchestra.run.smithi115.stdout:5178 punch from 0x9d58d5b to 0x9d60b4b, (0x7df0 bytes) 2024-09-17T22:42:28.289 INFO:teuthology.orchestra.run.smithi115.stdout:5179 read 0x4df75a5 thru 0x4e00cbb (0x9717 bytes) 2024-09-17T22:42:28.290 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.299 INFO:teuthology.orchestra.run.smithi115.stdout:5182 trunc from 0xb83f4e2 to 0x32a9e09 2024-09-17T22:42:28.312 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:28.310+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:28.314 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.326 INFO:teuthology.orchestra.run.smithi115.stdout:5185 trunc from 0x32a9e09 to 0xd13d2d4 2024-09-17T22:42:28.338 INFO:teuthology.orchestra.run.smithi115.stdout:5187 write 0xdb43569 thru 0xdb49da2 (0x683a bytes) 2024-09-17T22:42:28.342 INFO:teuthology.orchestra.run.smithi115.stdout:5188 read 0xafe43e5 thru 0xafe60fd (0x1d19 bytes) 2024-09-17T22:42:28.346 INFO:teuthology.orchestra.run.smithi115.stdout:5189 trunc from 0xdb49da3 to 0xa80a8a5 2024-09-17T22:42:28.354 INFO:teuthology.orchestra.run.smithi115.stdout:5193 write 0x56d5b46 thru 0x56dadb2 (0x526d bytes) 2024-09-17T22:42:28.356 INFO:teuthology.orchestra.run.smithi115.stdout:5194 read 0x4cfa503 thru 0x4cfe4f0 (0x3fee bytes) 2024-09-17T22:42:28.356 INFO:teuthology.orchestra.run.smithi115.stdout:5195 read 0x2250b59 thru 0x226024e (0xf6f6 bytes) 2024-09-17T22:42:28.356 INFO:teuthology.orchestra.run.smithi115.stdout:5196 writesame 0x197e19a thru 0x198a943 data_size 0x15(0xc7aa bytes) 2024-09-17T22:42:28.358 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.358 INFO:teuthology.orchestra.run.smithi115.stdout:5199 write 0x33b23ca thru 0x33bc21a (0x9e51 bytes) 2024-09-17T22:42:28.359 INFO:teuthology.orchestra.run.smithi115.stdout:5200 write 0x233dcaa thru 0x234dbee (0xff45 bytes) 2024-09-17T22:42:28.361 INFO:teuthology.orchestra.run.smithi115.stdout:5201 punch from 0x45647e4 to 0x4573d2d, (0xf549 bytes) 2024-09-17T22:42:28.369 INFO:teuthology.orchestra.run.smithi115.stdout:5203 trunc from 0xa80a8a5 to 0x3e37e0b 2024-09-17T22:42:28.374 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:42:28.374 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:42:28.375 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T22:42:28.375 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T22:42:28.394 INFO:teuthology.orchestra.run.smithi115.stdout:5204 write 0x9364f6b thru 0x93711ef (0xc285 bytes) 2024-09-17T22:42:28.397 INFO:teuthology.orchestra.run.smithi115.stdout:5206 write 0x8f9e95d thru 0x8fa1b67 (0x320b bytes) 2024-09-17T22:42:28.400 INFO:teuthology.orchestra.run.smithi115.stdout:5207 punch from 0x6531404 to 0x6535414, (0x4010 bytes) 2024-09-17T22:42:28.405 INFO:teuthology.orchestra.run.smithi115.stdout:5208 trunc from 0x93711f0 to 0xd1b6fed 2024-09-17T22:42:28.411 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:28.408+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:28.417 INFO:teuthology.orchestra.run.smithi115.stdout:5209 writesame 0xe07eb84 thru 0xe082a55 data_size 0x2b(0x3ed2 bytes) 2024-09-17T22:42:28.417 INFO:teuthology.orchestra.run.smithi115.stdout:5210 write 0x365428e thru 0x3659725 (0x5498 bytes) 2024-09-17T22:42:28.419 INFO:teuthology.orchestra.run.smithi115.stdout:5211 write 0x7bbc63f thru 0x7bca5c4 (0xdf86 bytes) 2024-09-17T22:42:28.419 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:28.420 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:28.422 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.422 INFO:teuthology.orchestra.run.smithi115.stdout:5213 read 0x4aadf8 thru 0x4b56c0 (0xa8c9 bytes) 2024-09-17T22:42:28.429 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:42:28.436 INFO:teuthology.orchestra.run.smithi115.stdout:5214 trunc from 0xe082a56 to 0x29a6ce4 2024-09-17T22:42:28.469 INFO:teuthology.orchestra.run.smithi115.stdout:5215 write 0x811cacf thru 0x8127b8b (0xb0bd bytes) 2024-09-17T22:42:28.480 INFO:teuthology.orchestra.run.smithi115.stdout:5216 trunc from 0x8127b8c to 0xeadb86a 2024-09-17T22:42:28.486 INFO:teuthology.orchestra.run.smithi115.stdout:5217 read 0x6941d15 thru 0x6948df2 (0x70de bytes) 2024-09-17T22:42:28.486 INFO:teuthology.orchestra.run.smithi115.stdout:5218 read 0xc2bca12 thru 0xc2c5f31 (0x9520 bytes) 2024-09-17T22:42:28.487 INFO:teuthology.orchestra.run.smithi115.stdout:5219 trunc from 0xeadb86a to 0xe52930f 2024-09-17T22:42:28.493 INFO:teuthology.orchestra.run.smithi115.stdout:5220 write 0xb887e3d thru 0xb88f2e6 (0x74aa bytes) 2024-09-17T22:42:28.497 INFO:teuthology.orchestra.run.smithi115.stdout:5221 read 0xa450afb thru 0xa457119 (0x661f bytes) 2024-09-17T22:42:28.497 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.497 INFO:teuthology.orchestra.run.smithi115.stdout:5224 write 0x3ce9bee thru 0x3cf110d (0x7520 bytes) 2024-09-17T22:42:28.500 INFO:teuthology.orchestra.run.smithi115.stdout:5225 read 0x80e05a4 thru 0x80e4369 (0x3dc6 bytes) 2024-09-17T22:42:28.500 INFO:teuthology.orchestra.run.smithi115.stdout:5226 write 0x9450836 thru 0x9459151 (0x891c bytes) 2024-09-17T22:42:28.504 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.504 INFO:teuthology.orchestra.run.smithi115.stdout:5228 writesame 0x19b3c6e thru 0x19bc7e4 data_size 0x3(0x8b77 bytes) 2024-09-17T22:42:28.508 INFO:teuthology.orchestra.run.smithi115.stdout:5229 write 0xb358385 thru 0xb35ccc3 (0x493f bytes) 2024-09-17T22:42:28.511 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:28.509+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:28.524 INFO:teuthology.orchestra.run.smithi115.stdout:5233 trunc from 0xe52930f to 0x3b03270 2024-09-17T22:42:28.542 INFO:teuthology.orchestra.run.smithi115.stdout:5234 trunc from 0x3b03270 to 0x832505b 2024-09-17T22:42:28.548 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.549 INFO:teuthology.orchestra.run.smithi115.stdout:5237 write 0x2b67210 thru 0x2b6a77e (0x356f bytes) 2024-09-17T22:42:28.561 INFO:teuthology.orchestra.run.smithi115.stdout:5238 write 0xc490637 thru 0xc49b693 (0xb05d bytes) 2024-09-17T22:42:28.564 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.565 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.565 INFO:teuthology.orchestra.run.smithi115.stdout:5244 write 0x79bc496 thru 0x79cbc6b (0xf7d6 bytes) 2024-09-17T22:42:28.569 INFO:teuthology.orchestra.run.smithi115.stdout:5245 trunc from 0xc49b694 to 0x9a2c90d 2024-09-17T22:42:28.576 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:28.576 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:28.577 INFO:teuthology.orchestra.run.smithi115.stdout:5246 read 0x713aa25 thru 0x714a8f1 (0xfecd bytes) 2024-09-17T22:42:28.577 INFO:teuthology.orchestra.run.smithi115.stdout:5249 read 0xf700b4 thru 0xf7405b (0x3fa8 bytes) 2024-09-17T22:42:28.577 INFO:teuthology.orchestra.run.smithi115.stdout:5250 read 0x72ef669 thru 0x72f234c (0x2ce4 bytes) 2024-09-17T22:42:28.578 INFO:teuthology.orchestra.run.smithi115.stdout:5252 read 0x2c7fdeb thru 0x2c8d588 (0xd79e bytes) 2024-09-17T22:42:28.578 INFO:teuthology.orchestra.run.smithi115.stdout:5253 read 0x8f73481 thru 0x8f7b260 (0x7de0 bytes) 2024-09-17T22:42:28.578 INFO:teuthology.orchestra.run.smithi115.stdout:5255 read 0x22112dd thru 0x221bb92 (0xa8b6 bytes) 2024-09-17T22:42:28.579 INFO:teuthology.orchestra.run.smithi115.stdout:5256 punch from 0x5334f43 to 0x5343b6a, (0xec27 bytes) 2024-09-17T22:42:28.579 INFO:teuthology.orchestra.run.smithi115.stdout:5257 writesame 0x2ae9c52 thru 0x2aee12f data_size 0x52(0x44de bytes) 2024-09-17T22:42:28.580 INFO:teuthology.orchestra.run.smithi115.stdout:5258 writesame 0x7505fbc thru 0x750d3bd data_size 0x1f(0x7402 bytes) 2024-09-17T22:42:28.583 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.583 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.586 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:42:28.594 INFO:teuthology.orchestra.run.smithi115.stdout:5264 writesame 0xe07b077 thru 0xe08ac17 data_size 0x1(0xfba1 bytes) 2024-09-17T22:42:28.594 INFO:teuthology.orchestra.run.smithi115.stdout:5265 write 0xcf39943 thru 0xcf47e6d (0xe52b bytes) 2024-09-17T22:42:28.612 INFO:teuthology.orchestra.run.smithi115.stdout:5266 trunc from 0xe08ac18 to 0x1c56a7e 2024-09-17T22:42:28.629 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:28.627+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:28.635 INFO:teuthology.orchestra.run.smithi115.stdout:5268 read 0x2205f3 thru 0x2250d2 (0x4ae0 bytes) 2024-09-17T22:42:28.642 INFO:teuthology.orchestra.run.smithi115.stdout:5269 write 0x2171ca4 thru 0x2179886 (0x7be3 bytes) 2024-09-17T22:42:28.646 INFO:teuthology.orchestra.run.smithi115.stdout:5270 write 0x78205b thru 0x78d241 (0xb1e7 bytes) 2024-09-17T22:42:28.647 INFO:teuthology.orchestra.run.smithi115.stdout:5272 punch from 0x1923cc1 to 0x1931e74, (0xe1b3 bytes) 2024-09-17T22:42:28.661 INFO:teuthology.orchestra.run.smithi115.stdout:5273 writesame 0x7968e2b thru 0x7973887 data_size 0x1(0xaa5d bytes) 2024-09-17T22:42:28.672 INFO:teuthology.orchestra.run.smithi115.stdout:5274 writesame 0xaafe258 thru 0xab07fd6 data_size 0x1(0x9d7f bytes) 2024-09-17T22:42:28.680 INFO:teuthology.orchestra.run.smithi115.stdout:5276 write 0xc9b043f thru 0xc9b2615 (0x21d7 bytes) 2024-09-17T22:42:28.683 INFO:teuthology.orchestra.run.smithi115.stdout:5277 read 0x434499b thru 0x435134f (0xc9b5 bytes) 2024-09-17T22:42:28.684 INFO:teuthology.orchestra.run.smithi115.stdout:5278 write 0x6a42f88 thru 0x6a47006 (0x407f bytes) 2024-09-17T22:42:28.687 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.691 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:42:28.691 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:28.691 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 81, 2024-09-17T22:42:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "551c5ed8-e790-485a-bc95-a9578a29275c", 2024-09-17T22:42:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-09-17T22:40:50.189230+0000", 2024-09-17T22:42:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-09-17T22:42:28.269557+0000", 2024-09-17T22:42:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: "last_up_change": "2024-09-17T22:41:53.099515+0000", 2024-09-17T22:42:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: "last_in_change": "2024-09-17T22:40:52.260526+0000", 2024-09-17T22:42:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T22:42:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_num": 5799936, 2024-09-17T22:42:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_set": [ 2024-09-17T22:42:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: "pglog_hardlimit", 2024-09-17T22:42:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: "purged_snapdirs", 2024-09-17T22:42:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_deletes", 2024-09-17T22:42:28.692 INFO:teuthology.orchestra.run.smithi115.stdout: "sortbitwise" 2024-09-17T22:42:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_version": 5, 2024-09-17T22:42:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T22:42:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T22:42:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T22:42:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_snapshot": "", 2024-09-17T22:42:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_max": 3, 2024-09-17T22:42:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "max_osd": 8, 2024-09-17T22:42:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "require_min_compat_client": "luminous", 2024-09-17T22:42:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "min_compat_client": "luminous", 2024-09-17T22:42:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "require_osd_release": "squid", 2024-09-17T22:42:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "allow_crimson": false, 2024-09-17T22:42:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: "pools": [ 2024-09-17T22:42:28.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 1, 2024-09-17T22:42:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_name": ".mgr", 2024-09-17T22:42:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "create_time": "2024-09-17T22:41:02.338356+0000", 2024-09-17T22:42:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-09-17T22:42:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_names": "hashpspool", 2024-09-17T22:42:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "type": 1, 2024-09-17T22:42:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 2, 2024-09-17T22:42:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "min_size": 1, 2024-09-17T22:42:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_rule": 0, 2024-09-17T22:42:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:42:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:42:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:42:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:42:28.694 INFO:teuthology.orchestra.run.smithi115.stdout: "is_stretch_pool": false, 2024-09-17T22:42:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "object_hash": 2, 2024-09-17T22:42:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:42:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num": 1, 2024-09-17T22:42:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num": 1, 2024-09-17T22:42:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num_target": 1, 2024-09-17T22:42:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_target": 1, 2024-09-17T22:42:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_pending": 1, 2024-09-17T22:42:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "last_pg_merge_meta": { 2024-09-17T22:42:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "source_pgid": "0.0", 2024-09-17T22:42:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "ready_epoch": 0, 2024-09-17T22:42:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_started": 0, 2024-09-17T22:42:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_clean": 0, 2024-09-17T22:42:28.695 INFO:teuthology.orchestra.run.smithi115.stdout: "source_version": "0'0", 2024-09-17T22:42:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: "target_version": "0'0" 2024-09-17T22:42:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: "last_change": "17", 2024-09-17T22:42:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend": "0", 2024-09-17T22:42:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:42:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:42:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: "auid": 0, 2024-09-17T22:42:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:42:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_seq": 0, 2024-09-17T22:42:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_epoch": 0, 2024-09-17T22:42:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_snaps": [], 2024-09-17T22:42:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps": "[]", 2024-09-17T22:42:28.696 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_bytes": 0, 2024-09-17T22:42:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_objects": 0, 2024-09-17T22:42:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "tiers": [], 2024-09-17T22:42:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "tier_of": -1, 2024-09-17T22:42:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "read_tier": -1, 2024-09-17T22:42:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "write_tier": -1, 2024-09-17T22:42:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_mode": "none", 2024-09-17T22:42:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_bytes": 0, 2024-09-17T22:42:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_objects": 0, 2024-09-17T22:42:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:42:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:42:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:42:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_flush_age": 0, 2024-09-17T22:42:28.697 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_evict_age": 0, 2024-09-17T22:42:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profile": "", 2024-09-17T22:42:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_params": { 2024-09-17T22:42:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "none" 2024-09-17T22:42:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_period": 0, 2024-09-17T22:42:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_count": 0, 2024-09-17T22:42:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: "use_gmt_hitset": true, 2024-09-17T22:42:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:42:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:42:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:42:28.698 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:42:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: "grade_table": [], 2024-09-17T22:42:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: "stripe_width": 0, 2024-09-17T22:42:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: "expected_num_objects": 0, 2024-09-17T22:42:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: "fast_read": false, 2024-09-17T22:42:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: "options": { 2024-09-17T22:42:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_max": 32, 2024-09-17T22:42:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_min": 1 2024-09-17T22:42:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: "application_metadata": { 2024-09-17T22:42:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": {} 2024-09-17T22:42:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.699 INFO:teuthology.orchestra.run.smithi115.stdout: "read_balance": { 2024-09-17T22:42:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: "score_type": "Fair distribution", 2024-09-17T22:42:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: "score_acting": 8, 2024-09-17T22:42:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: "score_stable": 8, 2024-09-17T22:42:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: "optimal_score": 0.25, 2024-09-17T22:42:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_acting": 2, 2024-09-17T22:42:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_stable": 2, 2024-09-17T22:42:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity_weighted": 1, 2024-09-17T22:42:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity": 1, 2024-09-17T22:42:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity_weighted": 1 2024-09-17T22:42:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-17T22:42:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_name": "rbd", 2024-09-17T22:42:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "create_time": "2024-09-17T22:41:08.097861+0000", 2024-09-17T22:42:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 8193, 2024-09-17T22:42:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T22:42:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "type": 1, 2024-09-17T22:42:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 2, 2024-09-17T22:42:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "min_size": 1, 2024-09-17T22:42:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_rule": 0, 2024-09-17T22:42:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:42:28.701 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:42:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:42:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:42:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: "is_stretch_pool": false, 2024-09-17T22:42:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: "object_hash": 2, 2024-09-17T22:42:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:42:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num": 8, 2024-09-17T22:42:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num": 8, 2024-09-17T22:42:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num_target": 8, 2024-09-17T22:42:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_target": 8, 2024-09-17T22:42:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_pending": 8, 2024-09-17T22:42:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: "last_pg_merge_meta": { 2024-09-17T22:42:28.702 INFO:teuthology.orchestra.run.smithi115.stdout: "source_pgid": "0.0", 2024-09-17T22:42:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: "ready_epoch": 0, 2024-09-17T22:42:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_started": 0, 2024-09-17T22:42:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_clean": 0, 2024-09-17T22:42:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: "source_version": "0'0", 2024-09-17T22:42:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: "target_version": "0'0" 2024-09-17T22:42:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: "last_change": "21", 2024-09-17T22:42:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend": "0", 2024-09-17T22:42:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:42:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:42:28.703 INFO:teuthology.orchestra.run.smithi115.stdout: "auid": 0, 2024-09-17T22:42:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:42:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_seq": 2, 2024-09-17T22:42:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_epoch": 21, 2024-09-17T22:42:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_snaps": [], 2024-09-17T22:42:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps": "[]", 2024-09-17T22:42:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_bytes": 0, 2024-09-17T22:42:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_objects": 0, 2024-09-17T22:42:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: "tiers": [], 2024-09-17T22:42:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: "tier_of": -1, 2024-09-17T22:42:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: "read_tier": -1, 2024-09-17T22:42:28.704 INFO:teuthology.orchestra.run.smithi115.stdout: "write_tier": -1, 2024-09-17T22:42:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_mode": "none", 2024-09-17T22:42:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_bytes": 0, 2024-09-17T22:42:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_objects": 0, 2024-09-17T22:42:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:42:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:42:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:42:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_flush_age": 0, 2024-09-17T22:42:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_evict_age": 0, 2024-09-17T22:42:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profile": "", 2024-09-17T22:42:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_params": { 2024-09-17T22:42:28.705 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "none" 2024-09-17T22:42:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_period": 0, 2024-09-17T22:42:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_count": 0, 2024-09-17T22:42:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "use_gmt_hitset": true, 2024-09-17T22:42:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:42:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:42:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:42:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:42:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "grade_table": [], 2024-09-17T22:42:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "stripe_width": 0, 2024-09-17T22:42:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "expected_num_objects": 0, 2024-09-17T22:42:28.706 INFO:teuthology.orchestra.run.smithi115.stdout: "fast_read": false, 2024-09-17T22:42:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: "options": {}, 2024-09-17T22:42:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: "application_metadata": { 2024-09-17T22:42:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd": {} 2024-09-17T22:42:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: "read_balance": { 2024-09-17T22:42:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: "score_type": "Fair distribution", 2024-09-17T22:42:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: "score_acting": 5, 2024-09-17T22:42:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: "score_stable": 5, 2024-09-17T22:42:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: "optimal_score": 0.75, 2024-09-17T22:42:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_acting": 3.75, 2024-09-17T22:42:28.707 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_stable": 3.75, 2024-09-17T22:42:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity_weighted": 1, 2024-09-17T22:42:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity": 1, 2024-09-17T22:42:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity_weighted": 1 2024-09-17T22:42:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 3, 2024-09-17T22:42:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_name": "pool_client.0", 2024-09-17T22:42:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: "create_time": "2024-09-17T22:41:17.501542+0000", 2024-09-17T22:42:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 8193, 2024-09-17T22:42:28.708 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T22:42:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: "type": 1, 2024-09-17T22:42:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 2, 2024-09-17T22:42:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: "min_size": 1, 2024-09-17T22:42:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_rule": 0, 2024-09-17T22:42:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:42:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:42:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:42:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:42:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: "is_stretch_pool": false, 2024-09-17T22:42:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: "object_hash": 2, 2024-09-17T22:42:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:42:28.709 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num": 32, 2024-09-17T22:42:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num": 32, 2024-09-17T22:42:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num_target": 32, 2024-09-17T22:42:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_target": 32, 2024-09-17T22:42:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_pending": 32, 2024-09-17T22:42:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: "last_pg_merge_meta": { 2024-09-17T22:42:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: "source_pgid": "0.0", 2024-09-17T22:42:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: "ready_epoch": 0, 2024-09-17T22:42:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_started": 0, 2024-09-17T22:42:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_clean": 0, 2024-09-17T22:42:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: "source_version": "0'0", 2024-09-17T22:42:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: "target_version": "0'0" 2024-09-17T22:42:28.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "last_change": "80", 2024-09-17T22:42:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend": "0", 2024-09-17T22:42:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:42:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:42:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "auid": 0, 2024-09-17T22:42:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:42:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_seq": 36, 2024-09-17T22:42:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_epoch": 80, 2024-09-17T22:42:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_snaps": [], 2024-09-17T22:42:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps": "[]", 2024-09-17T22:42:28.711 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_bytes": 0, 2024-09-17T22:42:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_objects": 0, 2024-09-17T22:42:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "tiers": [], 2024-09-17T22:42:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "tier_of": -1, 2024-09-17T22:42:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "read_tier": -1, 2024-09-17T22:42:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "write_tier": -1, 2024-09-17T22:42:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_mode": "none", 2024-09-17T22:42:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_bytes": 0, 2024-09-17T22:42:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_objects": 0, 2024-09-17T22:42:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:42:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:42:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:42:28.712 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_flush_age": 0, 2024-09-17T22:42:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_evict_age": 0, 2024-09-17T22:42:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profile": "", 2024-09-17T22:42:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_params": { 2024-09-17T22:42:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "none" 2024-09-17T22:42:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_period": 0, 2024-09-17T22:42:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_count": 0, 2024-09-17T22:42:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "use_gmt_hitset": true, 2024-09-17T22:42:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:42:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:42:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:42:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:42:28.713 INFO:teuthology.orchestra.run.smithi115.stdout: "grade_table": [], 2024-09-17T22:42:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: "stripe_width": 0, 2024-09-17T22:42:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: "expected_num_objects": 0, 2024-09-17T22:42:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: "fast_read": false, 2024-09-17T22:42:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: "options": {}, 2024-09-17T22:42:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: "application_metadata": { 2024-09-17T22:42:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd": {} 2024-09-17T22:42:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: "read_balance": { 2024-09-17T22:42:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: "score_type": "Fair distribution", 2024-09-17T22:42:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: "score_acting": 1.7400000095367432, 2024-09-17T22:42:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: "score_stable": 1.7400000095367432, 2024-09-17T22:42:28.714 INFO:teuthology.orchestra.run.smithi115.stdout: "optimal_score": 0.87999999523162842, 2024-09-17T22:42:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-17T22:42:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-17T22:42:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity_weighted": 1, 2024-09-17T22:42:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity": 1, 2024-09-17T22:42:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity_weighted": 1 2024-09-17T22:42:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:42:28.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 0, 2024-09-17T22:42:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "3bd5c11f-e7c2-4968-839a-2799c00d93a6", 2024-09-17T22:42:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 0, 2024-09-17T22:42:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 23, 2024-09-17T22:42:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 51, 2024-09-17T22:42:28.716 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6824", 2024-09-17T22:42:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6825", 2024-09-17T22:42:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6826", 2024-09-17T22:42:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6827", 2024-09-17T22:42:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6830", 2024-09-17T22:42:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.719 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6831", 2024-09-17T22:42:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6828", 2024-09-17T22:42:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.720 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6829", 2024-09-17T22:42:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6825/2782271783", 2024-09-17T22:42:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6827/2782271783", 2024-09-17T22:42:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6831/2782271783", 2024-09-17T22:42:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6829/2782271783", 2024-09-17T22:42:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "exists" 2024-09-17T22:42:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.721 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 1, 2024-09-17T22:42:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "02918203-cf2e-49fc-b994-743b5d81b4a1", 2024-09-17T22:42:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 51, 2024-09-17T22:42:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6816", 2024-09-17T22:42:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6817", 2024-09-17T22:42:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6818", 2024-09-17T22:42:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6819", 2024-09-17T22:42:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6822", 2024-09-17T22:42:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6823", 2024-09-17T22:42:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:28.725 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6820", 2024-09-17T22:42:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6821", 2024-09-17T22:42:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.726 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6817/1941780390", 2024-09-17T22:42:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6819/1941780390", 2024-09-17T22:42:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6823/1941780390", 2024-09-17T22:42:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6821/1941780390", 2024-09-17T22:42:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 2, 2024-09-17T22:42:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "22ee34ef-522e-4121-85b7-38bbafeca5de", 2024-09-17T22:42:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:28.727 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 13, 2024-09-17T22:42:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 51, 2024-09-17T22:42:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6808", 2024-09-17T22:42:28.728 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6809", 2024-09-17T22:42:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.729 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6810", 2024-09-17T22:42:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6811", 2024-09-17T22:42:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6814", 2024-09-17T22:42:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6815", 2024-09-17T22:42:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.731 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6812", 2024-09-17T22:42:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6813", 2024-09-17T22:42:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.732 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6809/689014698", 2024-09-17T22:42:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6811/689014698", 2024-09-17T22:42:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6815/689014698", 2024-09-17T22:42:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6813/689014698", 2024-09-17T22:42:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 3, 2024-09-17T22:42:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "5d464921-5696-43c5-b89d-430a54ddefb5", 2024-09-17T22:42:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 13, 2024-09-17T22:42:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 54, 2024-09-17T22:42:28.734 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6800", 2024-09-17T22:42:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6801", 2024-09-17T22:42:28.735 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6802", 2024-09-17T22:42:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6803", 2024-09-17T22:42:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.737 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6806", 2024-09-17T22:42:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6807", 2024-09-17T22:42:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:28.738 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6804", 2024-09-17T22:42:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6805", 2024-09-17T22:42:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6801/453192890", 2024-09-17T22:42:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6803/453192890", 2024-09-17T22:42:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6807/453192890", 2024-09-17T22:42:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6805/453192890", 2024-09-17T22:42:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.740 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 4, 2024-09-17T22:42:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "4cb6973a-63ae-4f48-af94-37610aee3c06", 2024-09-17T22:42:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 51, 2024-09-17T22:42:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:28.741 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6800", 2024-09-17T22:42:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6801", 2024-09-17T22:42:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.742 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6802", 2024-09-17T22:42:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6803", 2024-09-17T22:42:28.743 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6806", 2024-09-17T22:42:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6807", 2024-09-17T22:42:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6804", 2024-09-17T22:42:28.745 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6805", 2024-09-17T22:42:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6801/380388860", 2024-09-17T22:42:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6803/380388860", 2024-09-17T22:42:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6807/380388860", 2024-09-17T22:42:28.746 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6805/380388860", 2024-09-17T22:42:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 5, 2024-09-17T22:42:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "ef9fe11b-b719-48f4-9c6c-e1b1b355acdc", 2024-09-17T22:42:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:28.747 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 23, 2024-09-17T22:42:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6808", 2024-09-17T22:42:28.748 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6809", 2024-09-17T22:42:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.749 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6810", 2024-09-17T22:42:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6811", 2024-09-17T22:42:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6814", 2024-09-17T22:42:28.750 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6815", 2024-09-17T22:42:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.751 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6812", 2024-09-17T22:42:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6813", 2024-09-17T22:42:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.752 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6809/943488949", 2024-09-17T22:42:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6811/943488949", 2024-09-17T22:42:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6815/943488949", 2024-09-17T22:42:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6813/943488949", 2024-09-17T22:42:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 6, 2024-09-17T22:42:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "e4045aec-22c1-401b-bfbf-64f115bc32af", 2024-09-17T22:42:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:28.753 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 54, 2024-09-17T22:42:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6816", 2024-09-17T22:42:28.754 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6817", 2024-09-17T22:42:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6818", 2024-09-17T22:42:28.755 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6819", 2024-09-17T22:42:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.756 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6822", 2024-09-17T22:42:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6823", 2024-09-17T22:42:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.757 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6820", 2024-09-17T22:42:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6821", 2024-09-17T22:42:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6817/1873501077", 2024-09-17T22:42:28.758 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6819/1873501077", 2024-09-17T22:42:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6823/1873501077", 2024-09-17T22:42:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6821/1873501077", 2024-09-17T22:42:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 7, 2024-09-17T22:42:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "4a05a5d7-8281-4572-a9d0-6095ee5e168b", 2024-09-17T22:42:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:28.759 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 54, 2024-09-17T22:42:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.760 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6824", 2024-09-17T22:42:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6825", 2024-09-17T22:42:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.761 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6826", 2024-09-17T22:42:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6827", 2024-09-17T22:42:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:28.762 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6830", 2024-09-17T22:42:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.763 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6831", 2024-09-17T22:42:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6828", 2024-09-17T22:42:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:28.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6829", 2024-09-17T22:42:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6825/3609357666", 2024-09-17T22:42:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6827/3609357666", 2024-09-17T22:42:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6831/3609357666", 2024-09-17T22:42:28.765 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6829/3609357666", 2024-09-17T22:42:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_xinfo": [ 2024-09-17T22:42:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 0, 2024-09-17T22:42:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:28.766 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:59.351758+0000", 2024-09-17T22:42:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 51 2024-09-17T22:42:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 1, 2024-09-17T22:42:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:28.767 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:59.058915+0000", 2024-09-17T22:42:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 2, 2024-09-17T22:42:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:28.768 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.922383+0000", 2024-09-17T22:42:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 3, 2024-09-17T22:42:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:28.769 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.802379+0000", 2024-09-17T22:42:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 4, 2024-09-17T22:42:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.881681+0000", 2024-09-17T22:42:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:28.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 5, 2024-09-17T22:42:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.616649+0000", 2024-09-17T22:42:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.772 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 6, 2024-09-17T22:42:28.772 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:28.772 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:28.772 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:28.772 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:28.772 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:28.772 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.589555+0000", 2024-09-17T22:42:28.772 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:28.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.772 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 7, 2024-09-17T22:42:28.773 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:28.773 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:28.773 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:28.773 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:28.773 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:28.773 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:59.047783+0000", 2024-09-17T22:42:28.773 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:28.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.773 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:28.773 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_upmap": [], 2024-09-17T22:42:28.773 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_upmap_items": [ 2024-09-17T22:42:28.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.773 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.12", 2024-09-17T22:42:28.774 INFO:teuthology.orchestra.run.smithi115.stdout: "mappings": [ 2024-09-17T22:42:28.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.774 INFO:teuthology.orchestra.run.smithi115.stdout: "from": 3, 2024-09-17T22:42:28.774 INFO:teuthology.orchestra.run.smithi115.stdout: "to": 0 2024-09-17T22:42:28.774 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.774 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.774 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.16", 2024-09-17T22:42:28.774 INFO:teuthology.orchestra.run.smithi115.stdout: "mappings": [ 2024-09-17T22:42:28.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.775 INFO:teuthology.orchestra.run.smithi115.stdout: "from": 2, 2024-09-17T22:42:28.775 INFO:teuthology.orchestra.run.smithi115.stdout: "to": 0 2024-09-17T22:42:28.775 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.775 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.775 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.1d", 2024-09-17T22:42:28.775 INFO:teuthology.orchestra.run.smithi115.stdout: "mappings": [ 2024-09-17T22:42:28.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.775 INFO:teuthology.orchestra.run.smithi115.stdout: "from": 3, 2024-09-17T22:42:28.775 INFO:teuthology.orchestra.run.smithi115.stdout: "to": 7 2024-09-17T22:42:28.775 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.776 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.776 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:28.776 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_upmap_primaries": [], 2024-09-17T22:42:28.776 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_temp": [ 2024-09-17T22:42:28.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.776 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.12", 2024-09-17T22:42:28.776 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:42:28.776 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-09-17T22:42:28.776 INFO:teuthology.orchestra.run.smithi115.stdout: 3 2024-09-17T22:42:28.776 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.777 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.16", 2024-09-17T22:42:28.777 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:42:28.777 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-09-17T22:42:28.777 INFO:teuthology.orchestra.run.smithi115.stdout: 2 2024-09-17T22:42:28.777 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:28.777 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.1d", 2024-09-17T22:42:28.777 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:42:28.777 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-09-17T22:42:28.777 INFO:teuthology.orchestra.run.smithi115.stdout: 3 2024-09-17T22:42:28.777 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:28.777 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.777 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:28.778 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_temp": [], 2024-09-17T22:42:28.778 INFO:teuthology.orchestra.run.smithi115.stdout: "blocklist": {}, 2024-09-17T22:42:28.778 INFO:teuthology.orchestra.run.smithi115.stdout: "range_blocklist": {}, 2024-09-17T22:42:28.778 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profiles": { 2024-09-17T22:42:28.778 INFO:teuthology.orchestra.run.smithi115.stdout: "default": { 2024-09-17T22:42:28.778 INFO:teuthology.orchestra.run.smithi115.stdout: "crush-failure-domain": "osd", 2024-09-17T22:42:28.778 INFO:teuthology.orchestra.run.smithi115.stdout: "k": "2", 2024-09-17T22:42:28.778 INFO:teuthology.orchestra.run.smithi115.stdout: "m": "1", 2024-09-17T22:42:28.778 INFO:teuthology.orchestra.run.smithi115.stdout: "plugin": "jerasure", 2024-09-17T22:42:28.778 INFO:teuthology.orchestra.run.smithi115.stdout: "technique": "reed_sol_van" 2024-09-17T22:42:28.778 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:28.778 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps_queue": [], 2024-09-17T22:42:28.778 INFO:teuthology.orchestra.run.smithi115.stdout: "new_removed_snaps": [], 2024-09-17T22:42:28.779 INFO:teuthology.orchestra.run.smithi115.stdout: "new_purged_snaps": [], 2024-09-17T22:42:28.779 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_node_flags": {}, 2024-09-17T22:42:28.779 INFO:teuthology.orchestra.run.smithi115.stdout: "device_class_flags": {}, 2024-09-17T22:42:28.779 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": { 2024-09-17T22:42:28.779 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode_enabled": false, 2024-09-17T22:42:28.779 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_bucket_count": 0, 2024-09-17T22:42:28.779 INFO:teuthology.orchestra.run.smithi115.stdout: "degraded_stretch_mode": 0, 2024-09-17T22:42:28.779 INFO:teuthology.orchestra.run.smithi115.stdout: "recovering_stretch_mode": 0, 2024-09-17T22:42:28.779 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode_bucket": 0 2024-09-17T22:42:28.779 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:28.779 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:28.780 INFO:teuthology.orchestra.run.smithi115.stdout:5280 trunc from 0xc9b2616 to 0x4a73a16 2024-09-17T22:42:28.781 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:28.782 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:28.782 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:42:28.783 INFO:tasks.thrashosds.thrasher:j is {'epoch': 81, 'fsid': '551c5ed8-e790-485a-bc95-a9578a29275c', 'created': '2024-09-17T22:40:50.189230+0000', 'modified': '2024-09-17T22:42:28.269557+0000', 'last_up_change': '2024-09-17T22:41:53.099515+0000', 'last_in_change': '2024-09-17T22:40:52.260526+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T22:41:02.338356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T22:41:08.097861+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5, 'score_stable': 5, 'optimal_score': 0.75, 'raw_score_acting': 3.75, 'raw_score_stable': 3.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T22:41:17.501542+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 36, 'snap_epoch': 80, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '3bd5c11f-e7c2-4968-839a-2799c00d93a6', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 51, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6824', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6825', 'nonce': 2782271783}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 2782271783}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 2782271783}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 2782271783}]}, 'public_addr': '172.21.15.115:6825/2782271783', 'cluster_addr': '172.21.15.115:6827/2782271783', 'heartbeat_back_addr': '172.21.15.115:6831/2782271783', 'heartbeat_front_addr': '172.21.15.115:6829/2782271783', 'state': ['exists']}, {'osd': 1, 'uuid': '02918203-cf2e-49fc-b994-743b5d81b4a1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 1941780390}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 1941780390}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 1941780390}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 1941780390}]}, 'public_addr': '172.21.15.115:6817/1941780390', 'cluster_addr': '172.21.15.115:6819/1941780390', 'heartbeat_back_addr': '172.21.15.115:6823/1941780390', 'heartbeat_front_addr': '172.21.15.115:6821/1941780390', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '22ee34ef-522e-4121-85b7-38bbafeca5de', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 689014698}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 689014698}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 689014698}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 689014698}]}, 'public_addr': '172.21.15.115:6809/689014698', 'cluster_addr': '172.21.15.115:6811/689014698', 'heartbeat_back_addr': '172.21.15.115:6815/689014698', 'heartbeat_front_addr': '172.21.15.115:6813/689014698', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5d464921-5696-43c5-b89d-430a54ddefb5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 453192890}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 453192890}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 453192890}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 453192890}]}, 'public_addr': '172.21.15.115:6801/453192890', 'cluster_addr': '172.21.15.115:6803/453192890', 'heartbeat_back_addr': '172.21.15.115:6807/453192890', 'heartbeat_front_addr': '172.21.15.115:6805/453192890', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '4cb6973a-63ae-4f48-af94-37610aee3c06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6800', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6801', 'nonce': 380388860}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6802', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6803', 'nonce': 380388860}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6806', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6807', 'nonce': 380388860}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6804', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6805', 'nonce': 380388860}]}, 'public_addr': '172.21.15.129:6801/380388860', 'cluster_addr': '172.21.15.129:6803/380388860', 'heartbeat_back_addr': '172.21.15.129:6807/380388860', 'heartbeat_front_addr': '172.21.15.129:6805/380388860', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ef9fe11b-b719-48f4-9c6c-e1b1b355acdc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6808', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6809', 'nonce': 943488949}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6810', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6811', 'nonce': 943488949}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6814', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6815', 'nonce': 943488949}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6812', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6813', 'nonce': 943488949}]}, 'public_addr': '172.21.15.129:6809/943488949', 'cluster_addr': '172.21.15.129:6811/943488949', 'heartbeat_back_addr': '172.21.15.129:6815/943488949', 'heartbeat_front_addr': '172.21.15.129:6813/943488949', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e4045aec-22c1-401b-bfbf-64f115bc32af', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6816', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6817', 'nonce': 1873501077}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6818', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6819', 'nonce': 1873501077}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6822', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6823', 'nonce': 1873501077}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6820', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6821', 'nonce': 1873501077}]}, 'public_addr': '172.21.15.129:6817/1873501077', 'cluster_addr': '172.21.15.129:6819/1873501077', 'heartbeat_back_addr': '172.21.15.129:6823/1873501077', 'heartbeat_front_addr': '172.21.15.129:6821/1873501077', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '4a05a5d7-8281-4572-a9d0-6095ee5e168b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6824', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6825', 'nonce': 3609357666}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6826', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6827', 'nonce': 3609357666}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6830', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6831', 'nonce': 3609357666}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6828', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6829', 'nonce': 3609357666}]}, 'public_addr': '172.21.15.129:6825/3609357666', 'cluster_addr': '172.21.15.129:6827/3609357666', 'heartbeat_back_addr': '172.21.15.129:6831/3609357666', 'heartbeat_front_addr': '172.21.15.129:6829/3609357666', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:59.351758+0000', 'dead_epoch': 51}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:59.058915+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.922383+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.802379+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.881681+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.616649+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.589555+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:59.047783+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.12', 'mappings': [{'from': 3, 'to': 0}]}, {'pgid': '3.16', 'mappings': [{'from': 2, 'to': 0}]}, {'pgid': '3.1d', 'mappings': [{'from': 3, 'to': 7}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.12', 'osds': [6, 3]}, {'pgid': '3.16', 'osds': [3, 2]}, {'pgid': '3.1d', 'osds': [7, 3]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T22:42:28.783 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:42:28.785 INFO:teuthology.orchestra.run.smithi115.stdout:5281 read 0x3f9d49c thru 0x3fa5c13 (0x8778 bytes) 2024-09-17T22:42:28.786 INFO:teuthology.orchestra.run.smithi115.stdout:5282 writesame 0xe8b96b thru 0xe972d0 data_size 0x13(0xb966 bytes) 2024-09-17T22:42:28.786 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.786 INFO:teuthology.orchestra.run.smithi115.stdout:5287 write 0x41127e thru 0x419331 (0x80b4 bytes) 2024-09-17T22:42:28.786 INFO:teuthology.orchestra.run.smithi115.stdout:5288 write 0x18ba88d thru 0x18c974a (0xeebe bytes) 2024-09-17T22:42:28.786 INFO:teuthology.orchestra.run.smithi115.stdout:5291 write 0x3c6b65 thru 0x3cbfda (0x5476 bytes) 2024-09-17T22:42:28.786 INFO:teuthology.orchestra.run.smithi115.stdout:5292 write 0xd175447 thru 0xd185410 (0xffca bytes) 2024-09-17T22:42:28.786 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.786 INFO:teuthology.orchestra.run.smithi115.stdout:5295 read 0x2aefb47 thru 0x2afa964 (0xae1e bytes) 2024-09-17T22:42:28.786 INFO:teuthology.orchestra.run.smithi115.stdout:5296 write 0xe0d42c6 thru 0xe0e2e5d (0xeb98 bytes) 2024-09-17T22:42:28.786 INFO:teuthology.orchestra.run.smithi115.stdout:5298 writesame 0xd1d6802 thru 0xd1d7f78 data_size 0x1(0x1777 bytes) 2024-09-17T22:42:28.786 INFO:teuthology.orchestra.run.smithi115.stdout:5299 read 0xb467935 thru 0xb46c95d (0x5029 bytes) 2024-09-17T22:42:28.787 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.787 INFO:teuthology.orchestra.run.smithi115.stdout:5302 writesame 0xe318dea thru 0xe326683 data_size 0x19(0xd89a bytes) 2024-09-17T22:42:28.787 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.787 INFO:teuthology.orchestra.run.smithi115.stdout:5304 read 0xcbaf592 thru 0xcbbe328 (0xed97 bytes) 2024-09-17T22:42:28.787 INFO:teuthology.orchestra.run.smithi115.stdout:5305 read 0x6cff051 thru 0x6d03c9c (0x4c4c bytes) 2024-09-17T22:42:28.787 INFO:teuthology.orchestra.run.smithi115.stdout:5306 write 0x7f1c3f1 thru 0x7f288d5 (0xc4e5 bytes) 2024-09-17T22:42:28.787 INFO:teuthology.orchestra.run.smithi115.stdout:5307 read 0xabffedd thru 0xac05525 (0x5649 bytes) 2024-09-17T22:42:28.787 INFO:teuthology.orchestra.run.smithi115.stdout:5308 read 0x1668984 thru 0x166c361 (0x39de bytes) 2024-09-17T22:42:28.787 INFO:teuthology.orchestra.run.smithi115.stdout:5309 read 0xe05d2cc thru 0xe05dd03 (0xa38 bytes) 2024-09-17T22:42:28.787 INFO:teuthology.orchestra.run.smithi115.stdout:5311 writesame 0xd2c494a thru 0xd2cf142 data_size 0x1(0xa7f9 bytes) 2024-09-17T22:42:28.787 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:28.778+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:28.792 INFO:teuthology.orchestra.run.smithi115.stdout:5312 trunc from 0xe326684 to 0x4b77aaa 2024-09-17T22:42:28.804 INFO:teuthology.orchestra.run.smithi115.stdout:5313 punch from 0x9e3bb7 to 0x9f311c, (0xf565 bytes) 2024-09-17T22:42:28.805 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.808 INFO:teuthology.orchestra.run.smithi115.stdout:5317 trunc from 0x4b77aaa to 0x8401918 2024-09-17T22:42:28.822 INFO:teuthology.orchestra.run.smithi115.stdout:5318 write 0xa28338a thru 0xa28d0b1 (0x9d28 bytes) 2024-09-17T22:42:28.825 INFO:teuthology.orchestra.run.smithi115.stdout:5319 write 0x5ef1b98 thru 0x5ef2ced (0x1156 bytes) 2024-09-17T22:42:28.826 INFO:teuthology.orchestra.run.smithi115.stdout:5320 read 0x630f89d thru 0x631e76b (0xeecf bytes) 2024-09-17T22:42:28.827 INFO:teuthology.orchestra.run.smithi115.stdout:5321 read 0x2a39566 thru 0x2a45e94 (0xc92f bytes) 2024-09-17T22:42:28.827 INFO:teuthology.orchestra.run.smithi115.stdout:5322 read 0x2e3bb9d thru 0x2e48678 (0xcadc bytes) 2024-09-17T22:42:28.827 INFO:teuthology.orchestra.run.smithi115.stdout:5323 read 0x271fb0f thru 0x2723429 (0x391b bytes) 2024-09-17T22:42:28.831 INFO:teuthology.orchestra.run.smithi115.stdout:5324 trunc from 0xa28d0b2 to 0x7e931c9 2024-09-17T22:42:28.844 INFO:teuthology.orchestra.run.smithi115.stdout:5326 punch from 0x556546d to 0x5571b9c, (0xc72f bytes) 2024-09-17T22:42:28.845 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.845 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.850 INFO:teuthology.orchestra.run.smithi115.stdout:5332 trunc from 0x7e931c9 to 0x3a6147c 2024-09-17T22:42:28.857 INFO:teuthology.orchestra.run.smithi115.stdout:5333 read 0x309b29e thru 0x309cb15 (0x1878 bytes) 2024-09-17T22:42:28.857 INFO:teuthology.orchestra.run.smithi115.stdout:5335 punch from 0x1e0d3af to 0x1e19f41, (0xcb92 bytes) 2024-09-17T22:42:28.869 INFO:teuthology.orchestra.run.smithi115.stdout:5336 trunc from 0x3a6147c to 0xd5ad1a8 2024-09-17T22:42:28.875 INFO:teuthology.orchestra.run.smithi115.stdout:5337 punch from 0x5c82e2 to 0x5d04b8, (0x81d6 bytes) 2024-09-17T22:42:28.881 INFO:teuthology.orchestra.run.smithi115.stdout:5339 trunc from 0xd5ad1a8 to 0x84da212 2024-09-17T22:42:28.882 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:28.879+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:28.888 INFO:teuthology.orchestra.run.smithi115.stdout:5340 punch from 0x6eff58a to 0x6eff8fc, (0x372 bytes) 2024-09-17T22:42:28.888 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.897 INFO:teuthology.orchestra.run.smithi115.stdout:5342 write 0xa526ca1 thru 0xa52f34e (0x86ae bytes) 2024-09-17T22:42:28.909 INFO:teuthology.orchestra.run.smithi115.stdout:5344 trunc from 0xa52f34f to 0x2ee176f 2024-09-17T22:42:28.925 INFO:teuthology.orchestra.run.smithi115.stdout:5345 punch from 0x21d5a55 to 0x21e4068, (0xe613 bytes) 2024-09-17T22:42:28.925 INFO:teuthology.orchestra.run.smithi115.stdout:5347 read 0x64abca thru 0x652db2 (0x81e9 bytes) 2024-09-17T22:42:28.930 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:28.930 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:28.934 INFO:teuthology.orchestra.run.smithi115.stdout:5348 write 0x55843cd thru 0x55928ed (0xe521 bytes) 2024-09-17T22:42:28.936 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.936 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.936 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.936 INFO:teuthology.orchestra.run.smithi115.stdout:5352 read 0x323927 thru 0x3244fd (0xbd7 bytes) 2024-09-17T22:42:28.937 INFO:teuthology.orchestra.run.smithi115.stdout:5353 read 0xa4fbc0 thru 0xa53b6b (0x3fac bytes) 2024-09-17T22:42:28.937 INFO:teuthology.orchestra.run.smithi115.stdout:5354 write 0x45d2219 thru 0x45d2fcc (0xdb4 bytes) 2024-09-17T22:42:28.938 INFO:teuthology.orchestra.run.smithi115.stdout:5355 read 0x445c504 thru 0x446b4c6 (0xefc3 bytes) 2024-09-17T22:42:28.938 INFO:teuthology.orchestra.run.smithi115.stdout:5356 read 0x1cb91b0 thru 0x1cc40b4 (0xaf05 bytes) 2024-09-17T22:42:28.939 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.939 INFO:teuthology.orchestra.run.smithi115.stdout:5358 punch from 0x5044df1 to 0x504663f, (0x184e bytes) 2024-09-17T22:42:28.939 INFO:teuthology.orchestra.run.smithi115.stdout:5360 punch from 0x279864c to 0x2798e4b, (0x7ff bytes) 2024-09-17T22:42:28.939 INFO:teuthology.orchestra.run.smithi115.stdout:5361 write 0x3a2e9ed thru 0x3a3287c (0x3e90 bytes) 2024-09-17T22:42:28.940 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:42:28.942 INFO:teuthology.orchestra.run.smithi115.stdout:5362 punch from 0x228ce01 to 0x2293047, (0x6246 bytes) 2024-09-17T22:42:28.942 INFO:teuthology.orchestra.run.smithi115.stdout:5364 read 0x2f92a4 thru 0x304787 (0xb4e4 bytes) 2024-09-17T22:42:28.957 INFO:teuthology.orchestra.run.smithi115.stdout:5365 writesame 0xc30e588 thru 0xc316aae data_size 0x1(0x8527 bytes) 2024-09-17T22:42:28.957 INFO:teuthology.orchestra.run.smithi115.stdout:5366 writesame 0x1f3b403 thru 0x1f4ad91 data_size 0x1(0xf98f bytes) 2024-09-17T22:42:28.976 INFO:teuthology.orchestra.run.smithi115.stdout:5367 punch from 0x66542f9 to 0x66633e0, (0xf0e7 bytes) 2024-09-17T22:42:28.977 INFO:teuthology.orchestra.run.smithi115.stdout:5368 write 0xa5b34cf thru 0xa5b8959 (0x548b bytes) 2024-09-17T22:42:28.980 INFO:teuthology.orchestra.run.smithi115.stdout:5369 write 0x4369648 thru 0x436c3cd (0x2d86 bytes) 2024-09-17T22:42:28.982 INFO:teuthology.orchestra.run.smithi115.stdout:5370 write 0x30f8d40 thru 0x31024df (0x97a0 bytes) 2024-09-17T22:42:28.984 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:28.982+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:28.985 INFO:teuthology.orchestra.run.smithi115.stdout:5371 read 0x52ba628 thru 0x52c23f9 (0x7dd2 bytes) 2024-09-17T22:42:28.985 INFO:teuthology.orchestra.run.smithi115.stdout:5372 write 0x397ffae thru 0x3985744 (0x5797 bytes) 2024-09-17T22:42:28.987 INFO:teuthology.orchestra.run.smithi115.stdout:5373 read 0x8994fc3 thru 0x8998d9d (0x3ddb bytes) 2024-09-17T22:42:28.987 INFO:teuthology.orchestra.run.smithi115.stdout:5375 write 0x8eb4ce9 thru 0x8eb5a8a (0xda2 bytes) 2024-09-17T22:42:28.989 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.989 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:28.989 INFO:teuthology.orchestra.run.smithi115.stdout:5379 read 0x2232458 thru 0x2240b47 (0xe6f0 bytes) 2024-09-17T22:42:28.990 INFO:teuthology.orchestra.run.smithi115.stdout:5380 write 0x293fe46 thru 0x294c6f1 (0xc8ac bytes) 2024-09-17T22:42:28.991 INFO:teuthology.orchestra.run.smithi115.stdout:5384 writesame 0x475189e thru 0x475cf36 data_size 0x1(0xb699 bytes) 2024-09-17T22:42:29.006 INFO:teuthology.orchestra.run.smithi115.stdout:5385 read 0x75bf1e0 thru 0x75c37ef (0x4610 bytes) 2024-09-17T22:42:29.007 INFO:teuthology.orchestra.run.smithi115.stdout:5386 clone 35 order 25 su 2097152 sc 11 2024-09-17T22:42:29.045 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:29.045 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:29.056 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:42:29.093 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:42:29.094 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:42:29.097 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:29.095+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:29.106 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-09-17T22:42:28.320543+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1230319636,"num_objects":1875,"num_object_clones":421,"num_object_copies":3750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":453,"num_objects_misplaced":169,"num_objects_unfound":0,"num_objects_dirty":1875,"num_whiteouts":870,"num_read":246217,"num_read_kb":730301,"num_write":13719,"num_write_kb":261723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3161143,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11630,"ondisk_log_size":11630,"up":69,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":349276,"kb_used_data":44092,"kb_used_omap":12,"kb_used_meta":304819,"kb_avail":754625444,"statfs":{"total":773094113280,"available":772736454656,"internally_reserved":0,"allocated":45150208,"data_stored":320689195,"data_compressed":11449279,"data_compressed_allocated":28332032,"data_compressed_original":311781584,"omap_allocated":12750,"internal_metadata":312135218},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":62747799,"num_objects":141,"num_object_clones":13,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31,"num_objects_misplaced":16,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":5,"num_read":9975,"num_read_kb":42784,"num_write":2565,"num_write_kb":34145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.064866"},"pg_stats":[{"pgid":"3.1f","version":"80'355","reported_seq":6353,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T22:42:26.276022+0000","last_change":"2024-09-17T22:41:21.453392+0000","last_active":"2024-09-17T22:42:26.276022+0000","last_peered":"2024-09-17T22:42:26.276022+0000","last_clean":"2024-09-17T22:42:26.276022+0000","last_became_active":"2024-09-17T22:41:19.443658+0000","last_became_peered":"2024-09-17T22:41:19.443658+0000","last_unstale":"2024-09-17T22:42:26.276022+0000","last_undegraded":"2024-09-17T22:42:26.276022+0000","last_fullsized":"2024-09-17T22:42:26.276022+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:50:25.335161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047335399999999999,"stat_sum":{"num_bytes":34574835,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":21,"num_read":6798,"num_read_kb":12505,"num_write":392,"num_write_kb":2668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"80'690","reported_seq":6959,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T22:42:26.276097+0000","last_change":"2024-09-17T22:41:21.453587+0000","last_active":"2024-09-17T22:42:26.276097+0000","last_peered":"2024-09-17T22:42:26.276097+0000","last_clean":"2024-09-17T22:42:26.276097+0000","last_became_active":"2024-09-17T22:41:19.449357+0000","last_became_peered":"2024-09-17T22:41:19.449357+0000","last_unstale":"2024-09-17T22:42:26.276097+0000","last_undegraded":"2024-09-17T22:42:26.276097+0000","last_fullsized":"2024-09-17T22:42:26.276097+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:41:30.545725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066996099999999995,"stat_sum":{"num_bytes":75278952,"num_objects":74,"num_object_clones":15,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":32,"num_read":8773,"num_read_kb":25134,"num_write":1057,"num_write_kb":10815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"80'180","reported_seq":675,"reported_epoch":80,"state":"active+clean+remapped","last_fresh":"2024-09-17T22:42:26.319673+0000","last_change":"2024-09-17T22:41:57.183293+0000","last_active":"2024-09-17T22:42:26.319673+0000","last_peered":"2024-09-17T22:42:26.319673+0000","last_clean":"2024-09-17T22:42:26.319673+0000","last_became_active":"2024-09-17T22:41:57.133365+0000","last_became_peered":"2024-09-17T22:41:57.133365+0000","last_unstale":"2024-09-17T22:42:26.319673+0000","last_undegraded":"2024-09-17T22:42:26.319673+0000","last_fullsized":"2024-09-17T22:42:26.319673+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:32:49.043797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000382498,"stat_sum":{"num_bytes":32195522,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":47,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":21,"num_read":14393,"num_read_kb":45538,"num_write":162,"num_write_kb":8237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3161143,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[{"shards":"3,7","objects":47}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"80'381","reported_seq":1988,"reported_epoch":80,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:26.720596+0000","last_change":"2024-09-17T22:41:54.113405+0000","last_active":"2024-09-17T22:42:26.720596+0000","last_peered":"2024-09-17T22:42:26.720596+0000","last_clean":"2024-09-17T22:41:52.379845+0000","last_became_active":"2024-09-17T22:41:54.113405+0000","last_became_peered":"2024-09-17T22:41:54.113405+0000","last_unstale":"2024-09-17T22:42:26.720596+0000","last_undegraded":"2024-09-17T22:41:54.112692+0000","last_fullsized":"2024-09-17T22:41:54.112591+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:19.307042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001451078,"stat_sum":{"num_bytes":20790878,"num_objects":51,"num_object_clones":15,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":51,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":21,"num_read":2290,"num_read_kb":13808,"num_write":478,"num_write_kb":11820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":51}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"79'195","reported_seq":7584,"reported_epoch":80,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:26.272286+0000","last_change":"2024-09-17T22:41:54.118027+0000","last_active":"2024-09-17T22:42:26.272286+0000","last_peered":"2024-09-17T22:42:26.272286+0000","last_clean":"2024-09-17T22:41:52.368521+0000","last_became_active":"2024-09-17T22:41:54.118027+0000","last_became_peered":"2024-09-17T22:41:54.118027+0000","last_unstale":"2024-09-17T22:42:26.272286+0000","last_undegraded":"2024-09-17T22:41:54.116518+0000","last_fullsized":"2024-09-17T22:41:54.116411+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:32:14.049120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055813,"stat_sum":{"num_bytes":31562205,"num_objects":57,"num_object_clones":14,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":57,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":24,"num_read":7302,"num_read_kb":22730,"num_write":165,"num_write_kb":6187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":57}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"79'277","reported_seq":4196,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T22:42:27.319864+0000","last_change":"2024-09-17T22:41:21.446945+0000","last_active":"2024-09-17T22:42:27.319864+0000","last_peered":"2024-09-17T22:42:27.319864+0000","last_clean":"2024-09-17T22:42:27.319864+0000","last_became_active":"2024-09-17T22:41:19.446541+0000","last_became_peered":"2024-09-17T22:41:19.446541+0000","last_unstale":"2024-09-17T22:42:27.319864+0000","last_undegraded":"2024-09-17T22:42:27.319864+0000","last_fullsized":"2024-09-17T22:42:27.319864+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:53:10.059087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001630631,"stat_sum":{"num_bytes":35084120,"num_objects":69,"num_object_clones":11,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":38,"num_read":4340,"num_read_kb":18946,"num_write":272,"num_write_kb":4116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"79'257","reported_seq":5986,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T22:42:26.272238+0000","last_change":"2024-09-17T22:41:21.448965+0000","last_active":"2024-09-17T22:42:26.272238+0000","last_peered":"2024-09-17T22:42:26.272238+0000","last_clean":"2024-09-17T22:42:26.272238+0000","last_became_active":"2024-09-17T22:41:19.449940+0000","last_became_peered":"2024-09-17T22:41:19.449940+0000","last_unstale":"2024-09-17T22:42:26.272238+0000","last_undegraded":"2024-09-17T22:42:26.272238+0000","last_fullsized":"2024-09-17T22:42:26.272238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:41.249040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011385119999999999,"stat_sum":{"num_bytes":39791800,"num_objects":69,"num_object_clones":15,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":31,"num_read":5759,"num_read_kb":21348,"num_write":239,"num_write_kb":8944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"79'243","reported_seq":2553,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T22:42:26.275993+0000","last_change":"2024-09-17T22:41:21.453701+0000","last_active":"2024-09-17T22:42:26.275993+0000","last_peered":"2024-09-17T22:42:26.275993+0000","last_clean":"2024-09-17T22:42:26.275993+0000","last_became_active":"2024-09-17T22:41:19.446287+0000","last_became_peered":"2024-09-17T22:41:19.446287+0000","last_unstale":"2024-09-17T22:42:26.275993+0000","last_undegraded":"2024-09-17T22:42:26.275993+0000","last_fullsized":"2024-09-17T22:42:26.275993+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:05:21.962448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067184299999999998,"stat_sum":{"num_bytes":81648404,"num_objects":64,"num_object_clones":12,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":27,"num_read":3000,"num_read_kb":17287,"num_write":221,"num_write_kb":10286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"79'335","reported_seq":1881,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T22:42:26.275959+0000","last_change":"2024-09-17T22:41:21.453344+0000","last_active":"2024-09-17T22:42:26.275959+0000","last_peered":"2024-09-17T22:42:26.275959+0000","last_clean":"2024-09-17T22:42:26.275959+0000","last_became_active":"2024-09-17T22:41:19.445923+0000","last_became_peered":"2024-09-17T22:41:19.445923+0000","last_unstale":"2024-09-17T22:42:26.275959+0000","last_undegraded":"2024-09-17T22:42:26.275959+0000","last_fullsized":"2024-09-17T22:42:26.275959+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:29.464972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043560199999999999,"stat_sum":{"num_bytes":32816813,"num_objects":60,"num_object_clones":14,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":27,"num_read":2501,"num_read_kb":13949,"num_write":379,"num_write_kb":7873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"79'413","reported_seq":21766,"reported_epoch":79,"state":"active+clean+remapped","last_fresh":"2024-09-17T22:42:25.296246+0000","last_change":"2024-09-17T22:41:57.132629+0000","last_active":"2024-09-17T22:42:25.296246+0000","last_peered":"2024-09-17T22:42:25.296246+0000","last_clean":"2024-09-17T22:42:25.296246+0000","last_became_active":"2024-09-17T22:41:57.132018+0000","last_became_peered":"2024-09-17T22:41:57.132018+0000","last_unstale":"2024-09-17T22:42:25.296246+0000","last_undegraded":"2024-09-17T22:42:25.296246+0000","last_fullsized":"2024-09-17T22:42:25.296246+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:39:42.520285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000275164,"stat_sum":{"num_bytes":51445453,"num_objects":69,"num_object_clones":20,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":69,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":35,"num_read":22535,"num_read_kb":61116,"num_write":501,"num_write_kb":11324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":69}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"79'309","reported_seq":6661,"reported_epoch":80,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:26.296045+0000","last_change":"2024-09-17T22:41:54.119655+0000","last_active":"2024-09-17T22:42:26.296045+0000","last_peered":"2024-09-17T22:42:26.296045+0000","last_clean":"2024-09-17T22:41:52.245886+0000","last_became_active":"2024-09-17T22:41:54.119655+0000","last_became_peered":"2024-09-17T22:41:54.119655+0000","last_unstale":"2024-09-17T22:42:26.296045+0000","last_undegraded":"2024-09-17T22:41:54.119161+0000","last_fullsized":"2024-09-17T22:41:54.119071+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:34:39.254631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041055799999999999,"stat_sum":{"num_bytes":62426563,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":26,"num_read":7917,"num_read_kb":15365,"num_write":256,"num_write_kb":4114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":49}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"78'508","reported_seq":7409,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T22:42:25.258280+0000","last_change":"2024-09-17T22:41:21.446210+0000","last_active":"2024-09-17T22:42:25.258280+0000","last_peered":"2024-09-17T22:42:25.258280+0000","last_clean":"2024-09-17T22:42:25.258280+0000","last_became_active":"2024-09-17T22:41:19.451631+0000","last_became_peered":"2024-09-17T22:41:19.451631+0000","last_unstale":"2024-09-17T22:42:25.258280+0000","last_undegraded":"2024-09-17T22:42:25.258280+0000","last_fullsized":"2024-09-17T22:42:25.258280+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:07:37.889544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081193699999999999,"stat_sum":{"num_bytes":33009608,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":23,"num_read":8256,"num_read_kb":18206,"num_write":691,"num_write_kb":19427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":137,"reported_epoch":79,"state":"active+undersized","last_fresh":"2024-09-17T22:42:25.258739+0000","last_change":"2024-09-17T22:41:54.118987+0000","last_active":"2024-09-17T22:42:25.258739+0000","last_peered":"2024-09-17T22:42:25.258739+0000","last_clean":"2024-09-17T22:41:52.098057+0000","last_became_active":"2024-09-17T22:41:54.118987+0000","last_became_peered":"2024-09-17T22:41:54.118987+0000","last_unstale":"2024-09-17T22:42:25.258739+0000","last_undegraded":"2024-09-17T22:42:25.258739+0000","last_fullsized":"2024-09-17T22:41:54.114557+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.941682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"79'371","reported_seq":4547,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T22:42:26.272244+0000","last_change":"2024-09-17T22:41:21.448698+0000","last_active":"2024-09-17T22:42:26.272244+0000","last_peered":"2024-09-17T22:42:26.272244+0000","last_clean":"2024-09-17T22:42:26.272244+0000","last_became_active":"2024-09-17T22:41:19.448966+0000","last_became_peered":"2024-09-17T22:41:19.448966+0000","last_unstale":"2024-09-17T22:42:26.272244+0000","last_undegraded":"2024-09-17T22:42:26.272244+0000","last_fullsized":"2024-09-17T22:42:26.272244+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:11:57.273524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089570000000000003,"stat_sum":{"num_bytes":33075736,"num_objects":66,"num_object_clones":17,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":25,"num_read":5691,"num_read_kb":27095,"num_write":419,"num_write_kb":5342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":139,"reported_epoch":80,"state":"active+undersized","last_fresh":"2024-09-17T22:42:26.272355+0000","last_change":"2024-09-17T22:41:54.118378+0000","last_active":"2024-09-17T22:42:26.272355+0000","last_peered":"2024-09-17T22:42:26.272355+0000","last_clean":"2024-09-17T22:41:52.097696+0000","last_became_active":"2024-09-17T22:41:54.118378+0000","last_became_peered":"2024-09-17T22:41:54.118378+0000","last_unstale":"2024-09-17T22:42:26.272355+0000","last_undegraded":"2024-09-17T22:42:26.272355+0000","last_fullsized":"2024-09-17T22:41:54.116433+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:48:40.138581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"79'319","reported_seq":1340,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T22:42:26.272357+0000","last_change":"2024-09-17T22:41:21.448596+0000","last_active":"2024-09-17T22:42:26.272357+0000","last_peered":"2024-09-17T22:42:26.272357+0000","last_clean":"2024-09-17T22:42:26.272357+0000","last_became_active":"2024-09-17T22:41:19.451637+0000","last_became_peered":"2024-09-17T22:41:19.451637+0000","last_unstale":"2024-09-17T22:42:26.272357+0000","last_undegraded":"2024-09-17T22:42:26.272357+0000","last_fullsized":"2024-09-17T22:42:26.272357+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:13.759233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085737499999999996,"stat_sum":{"num_bytes":53858017,"num_objects":75,"num_object_clones":20,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":35,"num_read":1426,"num_read_kb":8598,"num_write":369,"num_write_kb":6993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":135,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T22:42:25.258732+0000","last_change":"2024-09-17T22:41:11.397753+0000","last_active":"2024-09-17T22:42:25.258732+0000","last_peered":"2024-09-17T22:42:25.258732+0000","last_clean":"2024-09-17T22:42:25.258732+0000","last_became_active":"2024-09-17T22:41:09.387651+0000","last_became_peered":"2024-09-17T22:41:09.387651+0000","last_unstale":"2024-09-17T22:42:25.258732+0000","last_undegraded":"2024-09-17T22:42:25.258732+0000","last_fullsized":"2024-09-17T22:42:25.258732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:14:30.915047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"79'400","reported_seq":946,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T22:42:26.295935+0000","last_change":"2024-09-17T22:41:21.466363+0000","last_active":"2024-09-17T22:42:26.295935+0000","last_peered":"2024-09-17T22:42:26.295935+0000","last_clean":"2024-09-17T22:42:26.295935+0000","last_became_active":"2024-09-17T22:41:19.444893+0000","last_became_peered":"2024-09-17T22:41:19.444893+0000","last_unstale":"2024-09-17T22:42:26.295935+0000","last_undegraded":"2024-09-17T22:42:26.295935+0000","last_fullsized":"2024-09-17T22:42:26.295935+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:29:39.792091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053617200000000002,"stat_sum":{"num_bytes":24915917,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":22,"num_read":1608,"num_read_kb":3590,"num_write":561,"num_write_kb":3243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":137,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T22:42:26.275879+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:42:26.275879+0000","last_peered":"2024-09-17T22:42:26.275879+0000","last_clean":"2024-09-17T22:42:26.275879+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:42:26.275879+0000","last_undegraded":"2024-09-17T22:42:26.275879+0000","last_fullsized":"2024-09-17T22:42:26.275879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"78'257","reported_seq":15030,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T22:42:25.263461+0000","last_change":"2024-09-17T22:41:21.516349+0000","last_active":"2024-09-17T22:42:25.263461+0000","last_peered":"2024-09-17T22:42:25.263461+0000","last_clean":"2024-09-17T22:42:25.263461+0000","last_became_active":"2024-09-17T22:41:19.451723+0000","last_became_peered":"2024-09-17T22:41:19.451723+0000","last_unstale":"2024-09-17T22:42:25.263461+0000","last_undegraded":"2024-09-17T22:42:25.263461+0000","last_fullsized":"2024-09-17T22:42:25.263461+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:07:21.986204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040033199999999999,"stat_sum":{"num_bytes":41417802,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":20,"num_read":15183,"num_read_kb":27687,"num_write":312,"num_write_kb":3903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"79'486","reported_seq":10399,"reported_epoch":79,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:25.284196+0000","last_change":"2024-09-17T22:41:54.119266+0000","last_active":"2024-09-17T22:42:25.284196+0000","last_peered":"2024-09-17T22:42:25.284196+0000","last_clean":"2024-09-17T22:41:52.262287+0000","last_became_active":"2024-09-17T22:41:54.119266+0000","last_became_peered":"2024-09-17T22:41:54.119266+0000","last_unstale":"2024-09-17T22:42:25.284196+0000","last_undegraded":"2024-09-17T22:41:54.116326+0000","last_fullsized":"2024-09-17T22:41:54.116242+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:28:00.198259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063898899999999996,"stat_sum":{"num_bytes":42579860,"num_objects":81,"num_object_clones":19,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":81,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":42,"num_read":13034,"num_read_kb":30492,"num_write":535,"num_write_kb":7915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":81}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":137,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T22:42:25.258833+0000","last_change":"2024-09-17T22:41:11.397488+0000","last_active":"2024-09-17T22:42:25.258833+0000","last_peered":"2024-09-17T22:42:25.258833+0000","last_clean":"2024-09-17T22:42:25.258833+0000","last_became_active":"2024-09-17T22:41:09.651390+0000","last_became_peered":"2024-09-17T22:41:09.651390+0000","last_unstale":"2024-09-17T22:42:25.258833+0000","last_undegraded":"2024-09-17T22:42:25.258833+0000","last_fullsized":"2024-09-17T22:42:25.258833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:21:00.598921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"79'261","reported_seq":3622,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T22:42:26.275851+0000","last_change":"2024-09-17T22:41:21.453941+0000","last_active":"2024-09-17T22:42:26.275851+0000","last_peered":"2024-09-17T22:42:26.275851+0000","last_clean":"2024-09-17T22:42:26.275851+0000","last_became_active":"2024-09-17T22:41:19.444314+0000","last_became_peered":"2024-09-17T22:41:19.444314+0000","last_unstale":"2024-09-17T22:42:26.275851+0000","last_undegraded":"2024-09-17T22:42:26.275851+0000","last_fullsized":"2024-09-17T22:42:26.275851+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:22:36.638483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085427099999999998,"stat_sum":{"num_bytes":43184098,"num_objects":55,"num_object_clones":10,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":25,"num_read":3768,"num_read_kb":20776,"num_write":257,"num_write_kb":8504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":137,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T22:42:26.270916+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:42:26.270916+0000","last_peered":"2024-09-17T22:42:26.270916+0000","last_clean":"2024-09-17T22:42:26.270916+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:42:26.270916+0000","last_undegraded":"2024-09-17T22:42:26.270916+0000","last_fullsized":"2024-09-17T22:42:26.270916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"79'282","reported_seq":6370,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T22:42:26.295905+0000","last_change":"2024-09-17T22:41:21.466094+0000","last_active":"2024-09-17T22:42:26.295905+0000","last_peered":"2024-09-17T22:42:26.295905+0000","last_clean":"2024-09-17T22:42:26.295905+0000","last_became_active":"2024-09-17T22:41:19.449791+0000","last_became_peered":"2024-09-17T22:41:19.449791+0000","last_unstale":"2024-09-17T22:42:26.295905+0000","last_undegraded":"2024-09-17T22:42:26.295905+0000","last_fullsized":"2024-09-17T22:42:26.295905+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:15:20.472831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042775300000000001,"stat_sum":{"num_bytes":17428062,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":20,"num_read":6628,"num_read_kb":7389,"num_write":349,"num_write_kb":3499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":135,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T22:42:25.258162+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:42:25.258162+0000","last_peered":"2024-09-17T22:42:25.258162+0000","last_clean":"2024-09-17T22:42:25.258162+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:42:25.258162+0000","last_undegraded":"2024-09-17T22:42:25.258162+0000","last_fullsized":"2024-09-17T22:42:25.258162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"80'642","reported_seq":8044,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T22:42:26.324863+0000","last_change":"2024-09-17T22:41:21.448409+0000","last_active":"2024-09-17T22:42:26.324863+0000","last_peered":"2024-09-17T22:42:26.324863+0000","last_clean":"2024-09-17T22:42:26.324863+0000","last_became_active":"2024-09-17T22:41:19.449279+0000","last_became_peered":"2024-09-17T22:41:19.449279+0000","last_unstale":"2024-09-17T22:42:26.324863+0000","last_undegraded":"2024-09-17T22:42:26.324863+0000","last_fullsized":"2024-09-17T22:42:26.324863+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:11:38.150480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075217299999999997,"stat_sum":{"num_bytes":26494996,"num_objects":60,"num_object_clones":14,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":28,"num_read":10640,"num_read_kb":22344,"num_write":883,"num_write_kb":2843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":136,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T22:42:25.258643+0000","last_change":"2024-09-17T22:41:11.398108+0000","last_active":"2024-09-17T22:42:25.258643+0000","last_peered":"2024-09-17T22:42:25.258643+0000","last_clean":"2024-09-17T22:42:25.258643+0000","last_became_active":"2024-09-17T22:41:09.388184+0000","last_became_peered":"2024-09-17T22:41:09.388184+0000","last_unstale":"2024-09-17T22:42:25.258643+0000","last_undegraded":"2024-09-17T22:42:25.258643+0000","last_fullsized":"2024-09-17T22:42:25.258643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:43.506329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":288,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T22:42:26.269112+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:42:26.269112+0000","last_peered":"2024-09-17T22:42:26.269112+0000","last_clean":"2024-09-17T22:42:26.269112+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:42:26.269112+0000","last_undegraded":"2024-09-17T22:42:26.269112+0000","last_fullsized":"2024-09-17T22:42:26.269112+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"79'503","reported_seq":11322,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T22:42:26.272230+0000","last_change":"2024-09-17T22:41:21.448156+0000","last_active":"2024-09-17T22:42:26.272230+0000","last_peered":"2024-09-17T22:42:26.272230+0000","last_clean":"2024-09-17T22:42:26.272230+0000","last_became_active":"2024-09-17T22:41:19.448956+0000","last_became_peered":"2024-09-17T22:41:19.448956+0000","last_unstale":"2024-09-17T22:42:26.272230+0000","last_undegraded":"2024-09-17T22:42:26.272230+0000","last_fullsized":"2024-09-17T22:42:26.272230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:51:40.948472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055851200000000005,"stat_sum":{"num_bytes":40017373,"num_objects":65,"num_object_clones":14,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":26,"num_read":12798,"num_read_kb":46194,"num_write":581,"num_write_kb":25112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"79'550","reported_seq":13903,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T22:42:25.307545+0000","last_change":"2024-09-17T22:41:21.445421+0000","last_active":"2024-09-17T22:42:25.307545+0000","last_peered":"2024-09-17T22:42:25.307545+0000","last_clean":"2024-09-17T22:42:25.307545+0000","last_became_active":"2024-09-17T22:41:19.449003+0000","last_became_peered":"2024-09-17T22:41:19.449003+0000","last_unstale":"2024-09-17T22:42:25.307545+0000","last_undegraded":"2024-09-17T22:42:25.307545+0000","last_fullsized":"2024-09-17T22:42:25.307545+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:43.072023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051465400000000002,"stat_sum":{"num_bytes":18178323,"num_objects":61,"num_object_clones":17,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":23,"num_read":15301,"num_read_kb":38443,"num_write":780,"num_write_kb":9175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"78'326","reported_seq":7921,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T22:42:25.258729+0000","last_change":"2024-09-17T22:41:21.445728+0000","last_active":"2024-09-17T22:42:25.258729+0000","last_peered":"2024-09-17T22:42:25.258729+0000","last_clean":"2024-09-17T22:42:25.258729+0000","last_became_active":"2024-09-17T22:41:19.448011+0000","last_became_peered":"2024-09-17T22:41:19.448011+0000","last_unstale":"2024-09-17T22:42:25.258729+0000","last_undegraded":"2024-09-17T22:42:25.258729+0000","last_fullsized":"2024-09-17T22:42:25.258729+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:55:35.265979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068494200000000002,"stat_sum":{"num_bytes":65517493,"num_objects":67,"num_object_clones":17,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":31,"num_read":8027,"num_read_kb":37446,"num_write":364,"num_write_kb":8597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"79'303","reported_seq":2177,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T22:42:26.270995+0000","last_change":"2024-09-17T22:41:21.462738+0000","last_active":"2024-09-17T22:42:26.270995+0000","last_peered":"2024-09-17T22:42:26.270995+0000","last_clean":"2024-09-17T22:42:26.270995+0000","last_became_active":"2024-09-17T22:41:19.447215+0000","last_became_peered":"2024-09-17T22:41:19.447215+0000","last_unstale":"2024-09-17T22:42:26.270995+0000","last_undegraded":"2024-09-17T22:42:26.270995+0000","last_fullsized":"2024-09-17T22:42:26.270995+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:20:16.811228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042703000000000001,"stat_sum":{"num_bytes":29991346,"num_objects":55,"num_object_clones":11,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":24,"num_read":3176,"num_read_kb":13965,"num_write":324,"num_write_kb":5664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"79'225","reported_seq":20895,"reported_epoch":80,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:26.271233+0000","last_change":"2024-09-17T22:41:54.119555+0000","last_active":"2024-09-17T22:42:26.271233+0000","last_peered":"2024-09-17T22:42:26.271233+0000","last_clean":"2024-09-17T22:41:52.249181+0000","last_became_active":"2024-09-17T22:41:54.119555+0000","last_became_peered":"2024-09-17T22:41:54.119555+0000","last_unstale":"2024-09-17T22:42:26.271233+0000","last_undegraded":"2024-09-17T22:41:54.118897+0000","last_fullsized":"2024-09-17T22:41:54.118785+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:48:16.795506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000395712,"stat_sum":{"num_bytes":80658686,"num_objects":55,"num_object_clones":9,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":55,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":28,"num_read":21133,"num_read_kb":59442,"num_write":201,"num_write_kb":12476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":55}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"79'253","reported_seq":1318,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T22:42:25.291358+0000","last_change":"2024-09-17T22:41:21.446562+0000","last_active":"2024-09-17T22:42:25.291358+0000","last_peered":"2024-09-17T22:42:25.291358+0000","last_clean":"2024-09-17T22:42:25.291358+0000","last_became_active":"2024-09-17T22:41:19.447631+0000","last_became_peered":"2024-09-17T22:41:19.447631+0000","last_unstale":"2024-09-17T22:42:25.291358+0000","last_undegraded":"2024-09-17T22:42:25.291358+0000","last_fullsized":"2024-09-17T22:42:25.291358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:57:56.548895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000402657,"stat_sum":{"num_bytes":23500280,"num_objects":47,"num_object_clones":9,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":22,"num_read":1295,"num_read_kb":6048,"num_write":280,"num_write_kb":5460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"79'242","reported_seq":8046,"reported_epoch":79,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:25.291707+0000","last_change":"2024-09-17T22:41:54.119016+0000","last_active":"2024-09-17T22:42:25.291707+0000","last_peered":"2024-09-17T22:42:25.291707+0000","last_clean":"2024-09-17T22:41:52.398679+0000","last_became_active":"2024-09-17T22:41:54.119016+0000","last_became_peered":"2024-09-17T22:41:54.119016+0000","last_unstale":"2024-09-17T22:42:25.291707+0000","last_undegraded":"2024-09-17T22:41:54.116346+0000","last_fullsized":"2024-09-17T22:41:54.116297+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:45:48.020538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038538199999999998,"stat_sum":{"num_bytes":23666153,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":29,"num_read":8314,"num_read_kb":21847,"num_write":200,"num_write_kb":10603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":54}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"80'174","reported_seq":1248,"reported_epoch":80,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:26.617870+0000","last_change":"2024-09-17T22:41:54.113116+0000","last_active":"2024-09-17T22:42:26.617870+0000","last_peered":"2024-09-17T22:42:26.617870+0000","last_clean":"2024-09-17T22:41:52.346319+0000","last_became_active":"2024-09-17T22:41:54.113116+0000","last_became_peered":"2024-09-17T22:41:54.113116+0000","last_unstale":"2024-09-17T22:42:26.617870+0000","last_undegraded":"2024-09-17T22:41:54.112692+0000","last_fullsized":"2024-09-17T22:41:54.112614+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:54:21.670495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000602379,"stat_sum":{"num_bytes":22649535,"num_objects":56,"num_object_clones":11,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":56,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":30,"num_read":1004,"num_read_kb":4925,"num_write":154,"num_write_kb":4074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":56}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"79'245","reported_seq":1501,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-17T22:42:26.779237+0000","last_change":"2024-09-17T22:41:21.446576+0000","last_active":"2024-09-17T22:42:26.779237+0000","last_peered":"2024-09-17T22:42:26.779237+0000","last_clean":"2024-09-17T22:42:26.779237+0000","last_became_active":"2024-09-17T22:41:19.447419+0000","last_became_peered":"2024-09-17T22:41:19.447419+0000","last_unstale":"2024-09-17T22:42:26.779237+0000","last_undegraded":"2024-09-17T22:42:26.779237+0000","last_fullsized":"2024-09-17T22:42:26.779237+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:07:26.226263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001865071,"stat_sum":{"num_bytes":12365359,"num_objects":49,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":26,"num_read":2587,"num_read_kb":7701,"num_write":215,"num_write_kb":2857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"79'451","reported_seq":2895,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T22:42:25.258584+0000","last_change":"2024-09-17T22:41:21.445226+0000","last_active":"2024-09-17T22:42:25.258584+0000","last_peered":"2024-09-17T22:42:25.258584+0000","last_clean":"2024-09-17T22:42:25.258584+0000","last_became_active":"2024-09-17T22:41:19.454151+0000","last_became_peered":"2024-09-17T22:41:19.454151+0000","last_unstale":"2024-09-17T22:42:25.258584+0000","last_undegraded":"2024-09-17T22:42:25.258584+0000","last_fullsized":"2024-09-17T22:42:25.258584+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:06:28.641650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050545600000000005,"stat_sum":{"num_bytes":25236168,"num_objects":69,"num_object_clones":21,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":34,"num_read":4583,"num_read_kb":11742,"num_write":551,"num_write_kb":3346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"79'717","reported_seq":10726,"reported_epoch":80,"state":"active+clean+remapped","last_fresh":"2024-09-17T22:42:26.295970+0000","last_change":"2024-09-17T22:41:57.132756+0000","last_active":"2024-09-17T22:42:26.295970+0000","last_peered":"2024-09-17T22:42:26.295970+0000","last_clean":"2024-09-17T22:42:26.295970+0000","last_became_active":"2024-09-17T22:41:57.131666+0000","last_became_peered":"2024-09-17T22:41:57.131666+0000","last_unstale":"2024-09-17T22:42:26.295970+0000","last_undegraded":"2024-09-17T22:42:26.295970+0000","last_fullsized":"2024-09-17T22:42:26.295970+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:34:27.299661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048487299999999998,"stat_sum":{"num_bytes":55795846,"num_objects":53,"num_object_clones":11,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":53,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":23,"num_read":13668,"num_read_kb":42221,"num_write":952,"num_write_kb":21487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6,3],"avail_no_missing":["6","3"],"object_location_counts":[{"shards":"3,6","objects":53}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"78'360","reported_seq":1975,"reported_epoch":79,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:25.258614+0000","last_change":"2024-09-17T22:41:54.118972+0000","last_active":"2024-09-17T22:42:25.258614+0000","last_peered":"2024-09-17T22:42:25.258614+0000","last_clean":"2024-09-17T22:41:52.267782+0000","last_became_active":"2024-09-17T22:41:54.118972+0000","last_became_peered":"2024-09-17T22:41:54.118972+0000","last_unstale":"2024-09-17T22:42:25.258614+0000","last_undegraded":"2024-09-17T22:41:54.114427+0000","last_fullsized":"2024-09-17T22:41:54.114366+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:44:19.564658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062146200000000003,"stat_sum":{"num_bytes":18573046,"num_objects":50,"num_object_clones":8,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":31,"num_read":2373,"num_read_kb":6324,"num_write":458,"num_write_kb":2401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":50}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1229729249,"num_objects":1871,"num_object_clones":421,"num_object_copies":3742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":453,"num_objects_misplaced":169,"num_objects_unfound":0,"num_objects_dirty":1871,"num_whiteouts":870,"num_read":246101,"num_read_kb":730201,"num_write":13558,"num_write_kb":259305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3161143,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38326272,"data_stored":317691090,"data_compressed":11570674,"data_compressed_allocated":28471296,"data_compressed_original":311949520,"omap_allocated":0,"internal_metadata":0},"log_size":11510,"ondisk_log_size":11510,"up":53,"acting":56,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":14,"acting":14,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542165,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40476,"kb_used_data":5124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331364,"statfs":{"total":96636764160,"available":96595316736,"internally_reserved":0,"allocated":5246976,"data_stored":34898849,"data_compressed":1309729,"data_compressed_allocated":3137536,"data_compressed_original":33788274,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542156,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34156,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337684,"statfs":{"total":96636764160,"available":96601788416,"internally_reserved":0,"allocated":3043328,"data_stored":23926169,"data_compressed":437861,"data_compressed_allocated":1896448,"data_compressed_original":23315877,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41140,"kb_used_data":5788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330700,"statfs":{"total":96636764160,"available":96594636800,"internally_reserved":0,"allocated":5926912,"data_stored":49752327,"data_compressed":1413258,"data_compressed_allocated":4018176,"data_compressed_original":48747863,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542163,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45404,"kb_used_data":5728,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326436,"statfs":{"total":96636764160,"available":96590270464,"internally_reserved":0,"allocated":5865472,"data_stored":36697195,"data_compressed":1585694,"data_compressed_allocated":3600384,"data_compressed_original":35459870,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45364,"kb_used_data":5856,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326476,"statfs":{"total":96636764160,"available":96590311424,"internally_reserved":0,"allocated":5996544,"data_stored":42914166,"data_compressed":1491041,"data_compressed_allocated":3727360,"data_compressed_original":41730657,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44716,"kb_used_data":5204,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327124,"statfs":{"total":96636764160,"available":96590974976,"internally_reserved":0,"allocated":5328896,"data_stored":35817964,"data_compressed":1381833,"data_compressed_allocated":3084288,"data_compressed_original":34605454,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52308,"kb_used_data":7232,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94319532,"statfs":{"total":96636764160,"available":96583200768,"internally_reserved":0,"allocated":7405568,"data_stored":53677809,"data_compressed":2115240,"data_compressed_allocated":4907008,"data_compressed_original":52375328,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45712,"kb_used_data":6188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326128,"statfs":{"total":96636764160,"available":96589955072,"internally_reserved":0,"allocated":6336512,"data_stored":43004716,"data_compressed":1714623,"data_compressed_allocated":3960832,"data_compressed_original":41758261,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2514944,"data_stored":23924031,"data_compressed":438892,"data_compressed_allocated":1921024,"data_compressed_original":23549349,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4251648,"data_stored":33876498,"data_compressed":1304751,"data_compressed_allocated":3100672,"data_compressed_original":33214834,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5529600,"data_stored":43070368,"data_compressed":1754085,"data_compressed_allocated":4022272,"data_compressed_original":42208821,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6680576,"data_stored":53937721,"data_compressed":2206098,"data_compressed_allocated":5033984,"data_compressed_original":53006112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4407296,"data_stored":35392448,"data_compressed":1381833,"data_compressed_allocated":3084288,"data_compressed_original":34605454,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5005312,"data_stored":41891853,"data_compressed":1486063,"data_compressed_allocated":3690496,"data_compressed_original":41157217,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4935680,"data_stored":36271510,"data_compressed":1585694,"data_compressed_allocated":3600384,"data_compressed_original":35459870,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5001216,"data_stored":49326661,"data_compressed":1413258,"data_compressed_allocated":4018176,"data_compressed_original":48747863,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:42:29.108 INFO:tasks.thrashosds.thrasher:Setting 3.1c to [1, 3] 2024-09-17T22:42:29.108 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.1c', '1', '3'] 2024-09-17T22:42:29.108 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.1c 1 3 2024-09-17T22:42:29.161 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:29.162 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:29.171 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:42:29.183 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:29.181+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:29.284 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:29.281+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:29.321 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:29.321 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:29.330 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:42:29.415 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:29.413+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:29.474 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-17T22:42:29.482 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:29.483 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:29.486 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:29.483+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:29.494 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:42:29.586 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:29.584+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:29.680 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:29.681 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:29.687 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:29.684+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:29.692 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:42:29.772 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:29.772 INFO:teuthology.orchestra.run.smithi115.stdout:5388 writesame 0x2840841 thru 0x284138c data_size 0x6(0xb4c bytes) 2024-09-17T22:42:29.782 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:29.782 INFO:teuthology.orchestra.run.smithi115.stdout:5391 punch from 0x49b6cba to 0x49c30ba, (0xc400 bytes) 2024-09-17T22:42:29.782 INFO:teuthology.orchestra.run.smithi115.stdout:5394 punch from 0x7502937 to 0x750ed8d, (0xc456 bytes) 2024-09-17T22:42:29.787 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:29.785+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:29.795 INFO:teuthology.orchestra.run.smithi115.stdout:5395 writesame 0xe9bb1cb thru 0xe9c09f2 data_size 0xe(0x5828 bytes) 2024-09-17T22:42:29.799 INFO:teuthology.orchestra.run.smithi115.stdout:5396 read 0x2584f01 thru 0x2591df9 (0xcef9 bytes) 2024-09-17T22:42:29.800 INFO:teuthology.orchestra.run.smithi115.stdout:5399 write 0x9fdafc4 thru 0x9fdf477 (0x44b4 bytes) 2024-09-17T22:42:29.807 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:29.807 INFO:teuthology.orchestra.run.smithi115.stdout:5402 write 0x81a78d3 thru 0x81b248f (0xabbd bytes) 2024-09-17T22:42:29.809 INFO:teuthology.orchestra.run.smithi115.stdout:5403 punch from 0x5661fb6 to 0x5662941, (0x98b bytes) 2024-09-17T22:42:29.809 INFO:teuthology.orchestra.run.smithi115.stdout:5405 read 0x7a61013 thru 0x7a6a4fb (0x94e9 bytes) 2024-09-17T22:42:29.809 INFO:teuthology.orchestra.run.smithi115.stdout:5407 write 0x65635c9 thru 0x656c62b (0x9063 bytes) 2024-09-17T22:42:29.810 INFO:teuthology.orchestra.run.smithi115.stdout:5408 punch from 0x655efac to 0x656308e, (0x40e2 bytes) 2024-09-17T22:42:29.811 INFO:teuthology.orchestra.run.smithi115.stdout:5409 write 0x9be9481 thru 0x9bf6dbb (0xd93b bytes) 2024-09-17T22:42:29.817 INFO:teuthology.orchestra.run.smithi115.stdout:5410 read 0xaa20951 thru 0xaa2b3cb (0xaa7b bytes) 2024-09-17T22:42:29.817 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:29.817 INFO:teuthology.orchestra.run.smithi115.stdout:5412 write 0xcaf48a2 thru 0xcb028ce (0xe02d bytes) 2024-09-17T22:42:29.819 INFO:teuthology.orchestra.run.smithi115.stdout:5413 read 0x48195ad thru 0x4826f0e (0xd962 bytes) 2024-09-17T22:42:29.819 INFO:teuthology.orchestra.run.smithi115.stdout:5414 read 0x68267f0 thru 0x6834d89 (0xe59a bytes) 2024-09-17T22:42:29.822 INFO:teuthology.orchestra.run.smithi115.stdout:5416 trunc from 0xe9c09f3 to 0xd2346cd 2024-09-17T22:42:29.842 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:29.842 INFO:teuthology.orchestra.run.smithi115.stdout:5418 punch from 0xc1f331a to 0xc1f37ab, (0x491 bytes) 2024-09-17T22:42:29.850 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:29.850 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:29.854 INFO:teuthology.orchestra.run.smithi115.stdout:5419 trunc from 0xd2346cd to 0x2f699d6 2024-09-17T22:42:29.861 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:42:29.862 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:42:29.863 INFO:teuthology.orchestra.run.smithi115.stdout:5420 read 0x25136af thru 0x2514940 (0x1292 bytes) 2024-09-17T22:42:29.864 INFO:teuthology.orchestra.run.smithi115.stdout:5422 read 0xcb06ee thru 0xcb270f (0x2022 bytes) 2024-09-17T22:42:29.864 INFO:teuthology.orchestra.run.smithi115.stdout:5423 read 0x143bb52 thru 0x1443e7d (0x832c bytes) 2024-09-17T22:42:29.866 INFO:teuthology.orchestra.run.smithi115.stdout:5424 trunc from 0x2f699d6 to 0xcaa703 2024-09-17T22:42:29.895 INFO:teuthology.orchestra.run.smithi115.stdout:5425 write 0x7da8f82 thru 0x7dab491 (0x2510 bytes) 2024-09-17T22:42:29.895 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:29.893+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:29.899 INFO:teuthology.orchestra.run.smithi115.stdout:5426 writesame 0x4e1c0a8 thru 0x4e1ca46 data_size 0x1(0x99f bytes) 2024-09-17T22:42:29.907 INFO:teuthology.orchestra.run.smithi115.stdout:5428 write 0x7fddf03 thru 0x7feaca4 (0xcda2 bytes) 2024-09-17T22:42:29.913 INFO:teuthology.orchestra.run.smithi115.stdout:5430 read 0x43b3802 thru 0x43b7628 (0x3e27 bytes) 2024-09-17T22:42:29.913 INFO:teuthology.orchestra.run.smithi115.stdout:5431 write 0x1f19566 thru 0x1f280ab (0xeb46 bytes) 2024-09-17T22:42:29.916 INFO:teuthology.orchestra.run.smithi115.stdout:5433 punch from 0x6a6161c to 0x6a6f36c, (0xdd50 bytes) 2024-09-17T22:42:29.916 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:29.916 INFO:teuthology.orchestra.run.smithi115.stdout:5435 punch from 0x860f10 to 0x8698e6, (0x89d6 bytes) 2024-09-17T22:42:29.916 INFO:teuthology.orchestra.run.smithi115.stdout:5436 punch from 0x18f7d11 to 0x1904f2b, (0xd21a bytes) 2024-09-17T22:42:29.929 INFO:teuthology.orchestra.run.smithi115.stdout:5437 writesame 0xdcdd87d thru 0xdced0ae data_size 0x1(0xf832 bytes) 2024-09-17T22:42:29.930 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:29.930 INFO:teuthology.orchestra.run.smithi115.stdout:5439 writesame 0x67a85aa thru 0x67b7aa4 data_size 0x25(0xf4fb bytes) 2024-09-17T22:42:29.935 INFO:teuthology.orchestra.run.smithi115.stdout:5440 trunc from 0xdced0af to 0xbb94137 2024-09-17T22:42:29.943 INFO:teuthology.orchestra.run.smithi115.stdout:5442 write 0x4e2852f thru 0x4e2b96f (0x3441 bytes) 2024-09-17T22:42:29.944 INFO:teuthology.orchestra.run.smithi115.stdout:5443 read 0x9325b3d thru 0x932af8c (0x5450 bytes) 2024-09-17T22:42:29.945 INFO:teuthology.orchestra.run.smithi115.stdout:5444 write 0x129c63b thru 0x129e5e4 (0x1faa bytes) 2024-09-17T22:42:29.948 INFO:teuthology.orchestra.run.smithi115.stdout:5447 write 0x5b65c81 thru 0x5b75512 (0xf892 bytes) 2024-09-17T22:42:29.949 INFO:teuthology.orchestra.run.smithi115.stdout:5449 write 0x3563135 thru 0x356a19f (0x706b bytes) 2024-09-17T22:42:29.951 INFO:teuthology.orchestra.run.smithi115.stdout:5450 clone 36 order 18 su 32768 sc 15 2024-09-17T22:42:29.978 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:29.980 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:29.980 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:29.990 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:42:29.991 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:29.987+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:29.996 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:29.997 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:30.014 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:30.014 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:30.030 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:30.030 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:30.047 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:30.047 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:30.064 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:30.064 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:30.081 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:30.081 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:30.090 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:30.087+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:30.098 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:30.098 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:30.102 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:30.102 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:30.103 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:30.104 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:30.104 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:30.104 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:30.113 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:42:30.191 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:30.189+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:30.255 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:30.255 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:30.255 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:30.255 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:30.255 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:30.264 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:42:30.294 INFO:teuthology.orchestra.run.smithi115.stderr:set 3.1c pg_upmap mapping to [1,3] 2024-09-17T22:42:30.294 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:30.292+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:30.297 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:42:30.377 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:30.377 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:30.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:30.377 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:30.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.378 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4229.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-17T22:42:30.378 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:30.378 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 80.979427630000004, 2024-09-17T22:42:30.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25708877200000002, 2024-09-17T22:42:30.378 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.378 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.378 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4229", 2024-09-17T22:42:30.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/3080427038", 2024-09-17T22:42:30.378 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2 2024-09-17T22:42:30.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.379 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:30.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:30.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394611+0000", 2024-09-17T22:42:30.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3149999999999999e-06 2024-09-17T22:42:30.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394619+0000", 2024-09-17T22:42:30.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240000000000003e-06 2024-09-17T22:42:30.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394632+0000", 2024-09-17T22:42:30.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2921e-05 2024-09-17T22:42:30.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394639+0000", 2024-09-17T22:42:30.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-09-17T22:42:30.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394703+0000", 2024-09-17T22:42:30.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4407999999999997e-05 2024-09-17T22:42:30.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:42:30.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394711+0000", 2024-09-17T22:42:30.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5939999999999996e-06 2024-09-17T22:42:30.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651545+0000", 2024-09-17T22:42:30.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25683409200000001 2024-09-17T22:42:30.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651698+0000", 2024-09-17T22:42:30.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000153678 2024-09-17T22:42:30.383 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.383 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.383 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.383 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38736 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:30.383 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:30.384 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.596831625, 2024-09-17T22:42:30.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051395418999999998, 2024-09-17T22:42:30.384 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.384 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.384 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:30.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:30.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.384 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38736 2024-09-17T22:42:30.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.385 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:30.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:30.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:30.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6899999999999998e-07 2024-09-17T22:42:30.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777212+0000", 2024-09-17T22:42:30.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1240000000000002e-06 2024-09-17T22:42:30.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777219+0000", 2024-09-17T22:42:30.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5409999999999997e-06 2024-09-17T22:42:30.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777224+0000", 2024-09-17T22:42:30.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:42:30.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.826131+0000", 2024-09-17T22:42:30.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048907369999999999 2024-09-17T22:42:30.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:30.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828457+0000", 2024-09-17T22:42:30.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023256209999999999 2024-09-17T22:42:30.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828601+0000", 2024-09-17T22:42:30.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014438799999999999 2024-09-17T22:42:30.389 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.389 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.389 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.390 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38752 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 673792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:30.390 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:30.390 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.596807554000002, 2024-09-17T22:42:30.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051582301999999997, 2024-09-17T22:42:30.390 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.390 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.390 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:30.390 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.390 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:30.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.391 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38752 2024-09-17T22:42:30.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.391 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:30.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:30.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:30.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3000000000000001e-07 2024-09-17T22:42:30.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777232+0000", 2024-09-17T22:42:30.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9709999999999998e-06 2024-09-17T22:42:30.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777238+0000", 2024-09-17T22:42:30.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7209999999999999e-06 2024-09-17T22:42:30.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777243+0000", 2024-09-17T22:42:30.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5210000000000004e-06 2024-09-17T22:42:30.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828633+0000", 2024-09-17T22:42:30.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051390915000000002 2024-09-17T22:42:30.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:30.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828711+0000", 2024-09-17T22:42:30.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7474999999999999e-05 2024-09-17T22:42:30.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828812+0000", 2024-09-17T22:42:30.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010116900000000001 2024-09-17T22:42:30.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.394 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.395 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.395 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38753 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:30.395 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:30.395 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.596788281000002, 2024-09-17T22:42:30.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051770444999999998, 2024-09-17T22:42:30.395 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.395 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.395 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:30.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:30.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.396 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38753 2024-09-17T22:42:30.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.396 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:30.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:30.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:30.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:42:30.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777251+0000", 2024-09-17T22:42:30.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8059999999999999e-06 2024-09-17T22:42:30.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777257+0000", 2024-09-17T22:42:30.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2480000000000001e-06 2024-09-17T22:42:30.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777262+0000", 2024-09-17T22:42:30.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000004e-06 2024-09-17T22:42:30.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828842+0000", 2024-09-17T22:42:30.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051580203999999998 2024-09-17T22:42:30.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:30.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828917+0000", 2024-09-17T22:42:30.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4980000000000004e-05 2024-09-17T22:42:30.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829019+0000", 2024-09-17T22:42:30.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102248 2024-09-17T22:42:30.400 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.400 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.400 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.400 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38766 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:30.400 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:30.400 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.596769295999998, 2024-09-17T22:42:30.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051958839999999999, 2024-09-17T22:42:30.401 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.401 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.401 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:30.401 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.401 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:30.401 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.401 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38766 2024-09-17T22:42:30.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.401 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:30.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:30.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777269+0000", 2024-09-17T22:42:30.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:42:30.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777271+0000", 2024-09-17T22:42:30.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-17T22:42:30.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777276+0000", 2024-09-17T22:42:30.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:42:30.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777281+0000", 2024-09-17T22:42:30.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:42:30.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829055+0000", 2024-09-17T22:42:30.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051773570999999997 2024-09-17T22:42:30.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:30.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829128+0000", 2024-09-17T22:42:30.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3131000000000002e-05 2024-09-17T22:42:30.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829227+0000", 2024-09-17T22:42:30.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8800000000000003e-05 2024-09-17T22:42:30.406 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.406 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.406 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.406 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38767 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:30.406 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:30.406 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.596749312999997, 2024-09-17T22:42:30.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137018, 2024-09-17T22:42:30.407 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.407 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.407 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:30.407 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.407 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:30.407 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.407 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38767 2024-09-17T22:42:30.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.407 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:30.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:30.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:42:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777290+0000", 2024-09-17T22:42:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.88e-06 2024-09-17T22:42:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777296+0000", 2024-09-17T22:42:30.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-09-17T22:42:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777300+0000", 2024-09-17T22:42:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4510000000000002e-06 2024-09-17T22:42:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829257+0000", 2024-09-17T22:42:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051956975000000002 2024-09-17T22:42:30.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829328+0000", 2024-09-17T22:42:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1026999999999996e-05 2024-09-17T22:42:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829425+0000", 2024-09-17T22:42:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.679e-05 2024-09-17T22:42:30.411 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38784 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 675840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.596730858999997, 2024-09-17T22:42:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052344522999999997, 2024-09-17T22:42:30.412 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.413 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.413 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:30.413 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.413 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:30.413 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.413 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38784 2024-09-17T22:42:30.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.413 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777307+0000", 2024-09-17T22:42:30.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:42:30.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777309+0000", 2024-09-17T22:42:30.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.063e-06 2024-09-17T22:42:30.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777314+0000", 2024-09-17T22:42:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-17T22:42:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777319+0000", 2024-09-17T22:42:30.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999996e-06 2024-09-17T22:42:30.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829456+0000", 2024-09-17T22:42:30.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137151 2024-09-17T22:42:30.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:30.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829527+0000", 2024-09-17T22:42:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1588000000000006e-05 2024-09-17T22:42:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829651+0000", 2024-09-17T22:42:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012365 2024-09-17T22:42:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.418 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38785 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 749568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.596712744000001, 2024-09-17T22:42:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052546572999999999, 2024-09-17T22:42:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.419 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38785 2024-09-17T22:42:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:42:30.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777327+0000", 2024-09-17T22:42:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:42:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777333+0000", 2024-09-17T22:42:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7309999999999998e-06 2024-09-17T22:42:30.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777337+0000", 2024-09-17T22:42:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:42:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829686+0000", 2024-09-17T22:42:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052349159999999999 2024-09-17T22:42:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829766+0000", 2024-09-17T22:42:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9326999999999995e-05 2024-09-17T22:42:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829871+0000", 2024-09-17T22:42:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000105466 2024-09-17T22:42:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38798 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.596079136, 2024-09-17T22:42:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052117564999999998, 2024-09-17T22:42:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.425 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38798 2024-09-17T22:42:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777959+0000", 2024-09-17T22:42:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9599999999999999e-07 2024-09-17T22:42:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777963+0000", 2024-09-17T22:42:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2660000000000003e-06 2024-09-17T22:42:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777971+0000", 2024-09-17T22:42:30.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3489999999999992e-06 2024-09-17T22:42:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777978+0000", 2024-09-17T22:42:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:42:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829906+0000", 2024-09-17T22:42:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051927635999999999 2024-09-17T22:42:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829986+0000", 2024-09-17T22:42:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0444000000000004e-05 2024-09-17T22:42:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830076+0000", 2024-09-17T22:42:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9473000000000002e-05 2024-09-17T22:42:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.429 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38799 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 882688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:30.430 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:30.430 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.596047697000003, 2024-09-17T22:42:30.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052359606000000003, 2024-09-17T22:42:30.430 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.430 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.430 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:30.430 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.430 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:30.430 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.430 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38799 2024-09-17T22:42:30.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.430 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:30.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:30.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:30.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8299999999999997e-07 2024-09-17T22:42:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777992+0000", 2024-09-17T22:42:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.176e-06 2024-09-17T22:42:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777998+0000", 2024-09-17T22:42:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.173e-06 2024-09-17T22:42:30.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778003+0000", 2024-09-17T22:42:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6349999999999997e-06 2024-09-17T22:42:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830112+0000", 2024-09-17T22:42:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052109366999999997 2024-09-17T22:42:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:30.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830192+0000", 2024-09-17T22:42:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9814000000000005e-05 2024-09-17T22:42:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830349+0000", 2024-09-17T22:42:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015695800000000001 2024-09-17T22:42:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38814 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:30.434 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.595764293000002, 2024-09-17T22:42:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052274339000000003, 2024-09-17T22:42:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38814 2024-09-17T22:42:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:30.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:42:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778276+0000", 2024-09-17T22:42:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2029999999999999e-06 2024-09-17T22:42:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778282+0000", 2024-09-17T22:42:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0059999999999998e-06 2024-09-17T22:42:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778287+0000", 2024-09-17T22:42:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0039999999999999e-06 2024-09-17T22:42:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830382+0000", 2024-09-17T22:42:30.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052095379999999997 2024-09-17T22:42:30.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:30.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830455+0000", 2024-09-17T22:42:30.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3127000000000006e-05 2024-09-17T22:42:30.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830547+0000", 2024-09-17T22:42:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2207999999999995e-05 2024-09-17T22:42:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38828 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 163840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.595581885000001, 2024-09-17T22:42:30.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052466039999999999, 2024-09-17T22:42:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38828 2024-09-17T22:42:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778456+0000", 2024-09-17T22:42:30.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7599999999999996e-07 2024-09-17T22:42:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778460+0000", 2024-09-17T22:42:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8229999999999997e-06 2024-09-17T22:42:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778470+0000", 2024-09-17T22:42:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8199999999999992e-06 2024-09-17T22:42:30.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778476+0000", 2024-09-17T22:42:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2589999999999997e-06 2024-09-17T22:42:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830578+0000", 2024-09-17T22:42:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052102273999999997 2024-09-17T22:42:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830667+0000", 2024-09-17T22:42:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8701000000000003e-05 2024-09-17T22:42:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830921+0000", 2024-09-17T22:42:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025438700000000002 2024-09-17T22:42:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.444 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38829 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.595550488000001, 2024-09-17T22:42:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052648882000000001, 2024-09-17T22:42:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.445 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:30.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:30.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.446 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38829 2024-09-17T22:42:30.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.446 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:30.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:30.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:30.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7e-07 2024-09-17T22:42:30.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778490+0000", 2024-09-17T22:42:30.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:42:30.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778495+0000", 2024-09-17T22:42:30.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9329999999999998e-06 2024-09-17T22:42:30.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778500+0000", 2024-09-17T22:42:30.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9509999999999997e-06 2024-09-17T22:42:30.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830958+0000", 2024-09-17T22:42:30.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05245785 2024-09-17T22:42:30.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:30.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831051+0000", 2024-09-17T22:42:30.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2989e-05 2024-09-17T22:42:30.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.450 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.450 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831136+0000", 2024-09-17T22:42:30.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4448000000000002e-05 2024-09-17T22:42:30.450 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38891 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.592238430000002, 2024-09-17T22:42:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050847934999999997, 2024-09-17T22:42:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.451 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38891 2024-09-17T22:42:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:30.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-09-17T22:42:30.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781802+0000", 2024-09-17T22:42:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:42:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781810+0000", 2024-09-17T22:42:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8019999999999992e-06 2024-09-17T22:42:30.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781816+0000", 2024-09-17T22:42:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.285e-06 2024-09-17T22:42:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832485+0000", 2024-09-17T22:42:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050668462999999997 2024-09-17T22:42:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:30.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832565+0000", 2024-09-17T22:42:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9692999999999997e-05 2024-09-17T22:42:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832647+0000", 2024-09-17T22:42:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2224000000000004e-05 2024-09-17T22:42:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38892 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.592210180999999, 2024-09-17T22:42:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050946120999999997, 2024-09-17T22:42:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.457 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38892 2024-09-17T22:42:30.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.458 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:30.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781828+0000", 2024-09-17T22:42:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5400000000000002e-07 2024-09-17T22:42:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781830+0000", 2024-09-17T22:42:30.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1959999999999998e-06 2024-09-17T22:42:30.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781837+0000", 2024-09-17T22:42:30.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8310000000000003e-06 2024-09-17T22:42:30.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781843+0000", 2024-09-17T22:42:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1e-06 2024-09-17T22:42:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832674+0000", 2024-09-17T22:42:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050831182000000003 2024-09-17T22:42:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832725+0000", 2024-09-17T22:42:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1588000000000001e-05 2024-09-17T22:42:30.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832773+0000", 2024-09-17T22:42:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7769999999999998e-05 2024-09-17T22:42:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38909 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 36864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.592187410999998, 2024-09-17T22:42:30.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051046518999999999, 2024-09-17T22:42:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38909 2024-09-17T22:42:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:30.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:42:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781852+0000", 2024-09-17T22:42:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0590000000000001e-06 2024-09-17T22:42:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781859+0000", 2024-09-17T22:42:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3380000000000001e-06 2024-09-17T22:42:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781864+0000", 2024-09-17T22:42:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:42:30.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832804+0000", 2024-09-17T22:42:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050939614000000001 2024-09-17T22:42:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832845+0000", 2024-09-17T22:42:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1014999999999999e-05 2024-09-17T22:42:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832896+0000", 2024-09-17T22:42:30.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1836999999999999e-05 2024-09-17T22:42:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38910 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.592167101000001, 2024-09-17T22:42:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051216818999999997, 2024-09-17T22:42:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:30.467 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38910 2024-09-17T22:42:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:30.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781871+0000", 2024-09-17T22:42:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:42:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781873+0000", 2024-09-17T22:42:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:42:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781879+0000", 2024-09-17T22:42:30.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0700000000000003e-06 2024-09-17T22:42:30.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781884+0000", 2024-09-17T22:42:30.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-09-17T22:42:30.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832932+0000", 2024-09-17T22:42:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051048219999999998 2024-09-17T22:42:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833001+0000", 2024-09-17T22:42:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8504999999999995e-05 2024-09-17T22:42:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833087+0000", 2024-09-17T22:42:30.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6464000000000004e-05 2024-09-17T22:42:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38923 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.592146780999997, 2024-09-17T22:42:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051379573999999997, 2024-09-17T22:42:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.472 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.473 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:30.473 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.473 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:30.473 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.473 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38923 2024-09-17T22:42:30.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.473 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:30.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:30.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:42:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781893+0000", 2024-09-17T22:42:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-09-17T22:42:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781900+0000", 2024-09-17T22:42:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3860000000000004e-06 2024-09-17T22:42:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781904+0000", 2024-09-17T22:42:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9300000000000002e-06 2024-09-17T22:42:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833115+0000", 2024-09-17T22:42:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051210845999999997 2024-09-17T22:42:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833188+0000", 2024-09-17T22:42:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2276999999999999e-05 2024-09-17T22:42:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833270+0000", 2024-09-17T22:42:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2501000000000002e-05 2024-09-17T22:42:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38941 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.591354246000002, 2024-09-17T22:42:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050969459000000002, 2024-09-17T22:42:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:30.478 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38941 2024-09-17T22:42:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782684+0000", 2024-09-17T22:42:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:42:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782686+0000", 2024-09-17T22:42:30.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.396e-06 2024-09-17T22:42:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782693+0000", 2024-09-17T22:42:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1999999999999997e-06 2024-09-17T22:42:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782699+0000", 2024-09-17T22:42:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7910000000000001e-06 2024-09-17T22:42:30.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833471+0000", 2024-09-17T22:42:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050772333000000003 2024-09-17T22:42:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833536+0000", 2024-09-17T22:42:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4380000000000004e-05 2024-09-17T22:42:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833653+0000", 2024-09-17T22:42:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011691799999999999 2024-09-17T22:42:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:151189 3.14 3:2b696b13:::rbd_data.115f14f2d1d5.0000000000000009:e [read 3127296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:30.483 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 40.661133501000002, 2024-09-17T22:42:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.054254700000000003, 2024-09-17T22:42:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 151189 2024-09-17T22:42:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:30.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713051+0000", 2024-09-17T22:42:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014722100000000001 2024-09-17T22:42:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713055+0000", 2024-09-17T22:42:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.16e-06 2024-09-17T22:42:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713065+0000", 2024-09-17T22:42:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8390000000000002e-06 2024-09-17T22:42:30.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713073+0000", 2024-09-17T22:42:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0390000000000005e-06 2024-09-17T22:42:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762644+0000", 2024-09-17T22:42:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049571160000000003 2024-09-17T22:42:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762739+0000", 2024-09-17T22:42:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4565e-05 2024-09-17T22:42:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.767159+0000", 2024-09-17T22:42:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044197159999999997 2024-09-17T22:42:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:30.488 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:30.489 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:30.490 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:42:30.491 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:30.487+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:30.591 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:30.588+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:30.594 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:30.594 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:30.594 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:30.595 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:30.603 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:42:30.692 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:30.689+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:30.706 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:30.706 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:30.706 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:30.706 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:30.706 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:30.714 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:42:30.793 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:30.791+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:30.857 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158673 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.613098915000002, 2024-09-17T22:42:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039400614, 2024-09-17T22:42:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.858 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158673 2024-09-17T22:42:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.04e-07 2024-09-17T22:42:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241563+0000", 2024-09-17T22:42:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5799999999999998e-07 2024-09-17T22:42:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241566+0000", 2024-09-17T22:42:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8279999999999999e-06 2024-09-17T22:42:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241568+0000", 2024-09-17T22:42:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2709999999999999e-06 2024-09-17T22:42:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280954+0000", 2024-09-17T22:42:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039386051999999998 2024-09-17T22:42:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280962+0000", 2024-09-17T22:42:30.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3010000000000007e-06 2024-09-17T22:42:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158675 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.613089623999997, 2024-09-17T22:42:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039408033000000002, 2024-09-17T22:42:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158675 2024-09-17T22:42:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.36e-07 2024-09-17T22:42:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241572+0000", 2024-09-17T22:42:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6599999999999994e-07 2024-09-17T22:42:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241575+0000", 2024-09-17T22:42:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8389999999999998e-06 2024-09-17T22:42:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241577+0000", 2024-09-17T22:42:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4119999999999999e-06 2024-09-17T22:42:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280975+0000", 2024-09-17T22:42:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039397205999999997 2024-09-17T22:42:30.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280979+0000", 2024-09-17T22:42:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.374e-06 2024-09-17T22:42:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158677 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.867 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.613067180999998, 2024-09-17T22:42:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039409307999999997, 2024-09-17T22:42:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158677 2024-09-17T22:42:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.868 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241594+0000", 2024-09-17T22:42:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:42:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241595+0000", 2024-09-17T22:42:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2729999999999999e-06 2024-09-17T22:42:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241602+0000", 2024-09-17T22:42:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:42:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241606+0000", 2024-09-17T22:42:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3019999999999999e-06 2024-09-17T22:42:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280994+0000", 2024-09-17T22:42:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039388484000000001 2024-09-17T22:42:30.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281003+0000", 2024-09-17T22:42:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5459999999999992e-06 2024-09-17T22:42:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.872 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158679 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.613051503000001, 2024-09-17T22:42:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039413403999999999, 2024-09-17T22:42:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.873 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158679 2024-09-17T22:42:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4999999999999999e-07 2024-09-17T22:42:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241613+0000", 2024-09-17T22:42:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.207e-06 2024-09-17T22:42:30.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241616+0000", 2024-09-17T22:42:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.275e-06 2024-09-17T22:42:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241618+0000", 2024-09-17T22:42:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:42:30.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281017+0000", 2024-09-17T22:42:30.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039398385000000001 2024-09-17T22:42:30.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281022+0000", 2024-09-17T22:42:30.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.896e-06 2024-09-17T22:42:30.877 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.878 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.878 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.878 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158681 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.878 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:30.878 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.613039325000003, 2024-09-17T22:42:30.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039425292000000001, 2024-09-17T22:42:30.878 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.878 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.879 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.879 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.879 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.879 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.879 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158681 2024-09-17T22:42:30.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.879 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:30.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:30.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:30.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:42:30.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241622+0000", 2024-09-17T22:42:30.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:42:30.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241625+0000", 2024-09-17T22:42:30.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9629999999999998e-06 2024-09-17T22:42:30.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241628+0000", 2024-09-17T22:42:30.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4339999999999999e-06 2024-09-17T22:42:30.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281039+0000", 2024-09-17T22:42:30.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039410672000000001 2024-09-17T22:42:30.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281047+0000", 2024-09-17T22:42:30.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9969999999999998e-06 2024-09-17T22:42:30.883 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.883 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.883 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.883 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158682 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.883 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:30.883 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.613030041000002, 2024-09-17T22:42:30.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039436455000000002, 2024-09-17T22:42:30.883 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158682 2024-09-17T22:42:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:30.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:30.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:30.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:42:30.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241632+0000", 2024-09-17T22:42:30.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:42:30.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241635+0000", 2024-09-17T22:42:30.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8940000000000002e-06 2024-09-17T22:42:30.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241637+0000", 2024-09-17T22:42:30.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3659999999999999e-06 2024-09-17T22:42:30.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281061+0000", 2024-09-17T22:42:30.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039423664999999997 2024-09-17T22:42:30.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281067+0000", 2024-09-17T22:42:30.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.285e-06 2024-09-17T22:42:30.887 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.887 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.887 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.888 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158684 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.888 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:30.888 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.613020695000003, 2024-09-17T22:42:30.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462980000000002, 2024-09-17T22:42:30.888 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.888 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.888 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158684 2024-09-17T22:42:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241641+0000", 2024-09-17T22:42:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:42:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241644+0000", 2024-09-17T22:42:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8870000000000001e-06 2024-09-17T22:42:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241646+0000", 2024-09-17T22:42:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3530000000000002e-06 2024-09-17T22:42:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281093+0000", 2024-09-17T22:42:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039446947000000003 2024-09-17T22:42:30.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281103+0000", 2024-09-17T22:42:30.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6299999999999993e-06 2024-09-17T22:42:30.892 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.892 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.892 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.892 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158686 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.892 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:30.892 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.613011524999997, 2024-09-17T22:42:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039474214000000001, 2024-09-17T22:42:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158686 2024-09-17T22:42:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:42:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241650+0000", 2024-09-17T22:42:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2999999999999999e-07 2024-09-17T22:42:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241653+0000", 2024-09-17T22:42:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9689999999999999e-06 2024-09-17T22:42:30.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241655+0000", 2024-09-17T22:42:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3460000000000001e-06 2024-09-17T22:42:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281118+0000", 2024-09-17T22:42:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462876000000001 2024-09-17T22:42:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281123+0000", 2024-09-17T22:42:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8840000000000002e-06 2024-09-17T22:42:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158688 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:30.897 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.613002293999998, 2024-09-17T22:42:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039490458999999999, 2024-09-17T22:42:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158688 2024-09-17T22:42:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:42:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241659+0000", 2024-09-17T22:42:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3699999999999999e-07 2024-09-17T22:42:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241662+0000", 2024-09-17T22:42:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8499999999999998e-06 2024-09-17T22:42:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241665+0000", 2024-09-17T22:42:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.356e-06 2024-09-17T22:42:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281141+0000", 2024-09-17T22:42:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039476031000000002 2024-09-17T22:42:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281149+0000", 2024-09-17T22:42:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0709999999999995e-06 2024-09-17T22:42:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158690 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.612993234999998, 2024-09-17T22:42:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039502378999999997, 2024-09-17T22:42:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.903 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158690 2024-09-17T22:42:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:30.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241668+0000", 2024-09-17T22:42:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241669+0000", 2024-09-17T22:42:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1009999999999999e-06 2024-09-17T22:42:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241672+0000", 2024-09-17T22:42:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9059999999999998e-06 2024-09-17T22:42:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241674+0000", 2024-09-17T22:42:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4439999999999998e-06 2024-09-17T22:42:30.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281162+0000", 2024-09-17T22:42:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039488189 2024-09-17T22:42:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281170+0000", 2024-09-17T22:42:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5310000000000004e-06 2024-09-17T22:42:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158692 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.612983802000002, 2024-09-17T22:42:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039531064999999997, 2024-09-17T22:42:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.908 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158692 2024-09-17T22:42:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241678+0000", 2024-09-17T22:42:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0800000000000003e-07 2024-09-17T22:42:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241681+0000", 2024-09-17T22:42:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8729999999999999e-06 2024-09-17T22:42:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241683+0000", 2024-09-17T22:42:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.447e-06 2024-09-17T22:42:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281199+0000", 2024-09-17T22:42:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039515388999999998 2024-09-17T22:42:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281208+0000", 2024-09-17T22:42:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2399999999999996e-06 2024-09-17T22:42:30.912 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158694 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.612974631, 2024-09-17T22:42:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039543986000000003, 2024-09-17T22:42:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.913 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158694 2024-09-17T22:42:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:42:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241687+0000", 2024-09-17T22:42:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:42:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241691+0000", 2024-09-17T22:42:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.811e-06 2024-09-17T22:42:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241693+0000", 2024-09-17T22:42:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.385e-06 2024-09-17T22:42:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281223+0000", 2024-09-17T22:42:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039529332 2024-09-17T22:42:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281230+0000", 2024-09-17T22:42:30.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2139999999999999e-06 2024-09-17T22:42:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158696 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.612964243, 2024-09-17T22:42:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039570458000000003, 2024-09-17T22:42:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158696 2024-09-17T22:42:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:42:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:42:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:42:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.33e-07 2024-09-17T22:42:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241700+0000", 2024-09-17T22:42:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8949999999999998e-06 2024-09-17T22:42:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241703+0000", 2024-09-17T22:42:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3159999999999999e-06 2024-09-17T22:42:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281257+0000", 2024-09-17T22:42:30.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039553862000000002 2024-09-17T22:42:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281267+0000", 2024-09-17T22:42:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.024e-05 2024-09-17T22:42:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158698 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.612954164999998, 2024-09-17T22:42:30.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039576493999999997, 2024-09-17T22:42:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158698 2024-09-17T22:42:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241707+0000", 2024-09-17T22:42:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:42:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241708+0000", 2024-09-17T22:42:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5000000000000001e-07 2024-09-17T22:42:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241711+0000", 2024-09-17T22:42:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9670000000000001e-06 2024-09-17T22:42:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241713+0000", 2024-09-17T22:42:30.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2689999999999998e-06 2024-09-17T22:42:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281278+0000", 2024-09-17T22:42:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039565206999999998 2024-09-17T22:42:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281283+0000", 2024-09-17T22:42:30.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:42:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158700 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.612944954, 2024-09-17T22:42:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039604292999999999, 2024-09-17T22:42:30.929 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158700 2024-09-17T22:42:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:30.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241717+0000", 2024-09-17T22:42:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.78e-07 2024-09-17T22:42:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241720+0000", 2024-09-17T22:42:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8119999999999999e-06 2024-09-17T22:42:30.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241722+0000", 2024-09-17T22:42:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-17T22:42:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281311+0000", 2024-09-17T22:42:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039588894999999999 2024-09-17T22:42:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281320+0000", 2024-09-17T22:42:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9989999999999997e-06 2024-09-17T22:42:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158702 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:30.934 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.612933632000001, 2024-09-17T22:42:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039616142, 2024-09-17T22:42:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158702 2024-09-17T22:42:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4400000000000001e-07 2024-09-17T22:42:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241728+0000", 2024-09-17T22:42:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0839999999999999e-06 2024-09-17T22:42:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241731+0000", 2024-09-17T22:42:30.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0690000000000002e-06 2024-09-17T22:42:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241734+0000", 2024-09-17T22:42:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4159999999999998e-06 2024-09-17T22:42:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281334+0000", 2024-09-17T22:42:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039600110000000001 2024-09-17T22:42:30.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281343+0000", 2024-09-17T22:42:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2189999999999993e-06 2024-09-17T22:42:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158704 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.612923768000002, 2024-09-17T22:42:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039641485999999997, 2024-09-17T22:42:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158704 2024-09-17T22:42:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:42:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241738+0000", 2024-09-17T22:42:30.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0589999999999999e-06 2024-09-17T22:42:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241741+0000", 2024-09-17T22:42:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.943e-06 2024-09-17T22:42:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241744+0000", 2024-09-17T22:42:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4819999999999999e-06 2024-09-17T22:42:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281370+0000", 2024-09-17T22:42:30.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039625661 2024-09-17T22:42:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281378+0000", 2024-09-17T22:42:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1310000000000006e-06 2024-09-17T22:42:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158705 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.612913218000003, 2024-09-17T22:42:30.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039652637999999997, 2024-09-17T22:42:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158705 2024-09-17T22:42:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:30.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:42:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-07 2024-09-17T22:42:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:42:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0599999999999999e-07 2024-09-17T22:42:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241751+0000", 2024-09-17T22:42:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.926e-06 2024-09-17T22:42:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241754+0000", 2024-09-17T22:42:30.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4360000000000001e-06 2024-09-17T22:42:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281392+0000", 2024-09-17T22:42:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039638566 2024-09-17T22:42:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281400+0000", 2024-09-17T22:42:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.571e-06 2024-09-17T22:42:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.948 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158707 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.612892117000001, 2024-09-17T22:42:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039654809999999999, 2024-09-17T22:42:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158707 2024-09-17T22:42:30.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241769+0000", 2024-09-17T22:42:30.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4299999999999999e-07 2024-09-17T22:42:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241770+0000", 2024-09-17T22:42:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3769999999999999e-06 2024-09-17T22:42:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241773+0000", 2024-09-17T22:42:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2109999999999998e-06 2024-09-17T22:42:30.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241776+0000", 2024-09-17T22:42:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5069999999999999e-06 2024-09-17T22:42:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281415+0000", 2024-09-17T22:42:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039639499000000002 2024-09-17T22:42:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281423+0000", 2024-09-17T22:42:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8730000000000007e-06 2024-09-17T22:42:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158709 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.612881508000001, 2024-09-17T22:42:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039661937000000001, 2024-09-17T22:42:30.954 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158709 2024-09-17T22:42:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:30.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242843+0000", 2024-09-17T22:42:30.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010634780000000001 2024-09-17T22:42:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242845+0000", 2024-09-17T22:42:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7209999999999999e-06 2024-09-17T22:42:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242851+0000", 2024-09-17T22:42:30.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6949999999999997e-06 2024-09-17T22:42:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242855+0000", 2024-09-17T22:42:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7960000000000002e-06 2024-09-17T22:42:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281434+0000", 2024-09-17T22:42:30.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038579519 2024-09-17T22:42:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281441+0000", 2024-09-17T22:42:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7279999999999998e-06 2024-09-17T22:42:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:30.959 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:30.959 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:30.962 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:42:30.962 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:30.958+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:31.062 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:31.060+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:31.106 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:31.106 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:31.106 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:31.106 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:31.114 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:42:31.198 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:31.196+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:31.258 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:31.258 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:31.258 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:31.258 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:31.258 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:31.264 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:31.261+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:31.267 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:42:31.296 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is set 2024-09-17T22:42:31.365 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:31.362+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:31.413 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:31.413 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:31.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:31.413 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157229 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4073472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.263844800000001, 2024-09-17T22:42:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041957959000000003, 2024-09-17T22:42:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.414 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157229 2024-09-17T22:42:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146207+0000", 2024-09-17T22:42:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0299999999999999e-07 2024-09-17T22:42:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146215+0000", 2024-09-17T22:42:31.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3059999999999998e-06 2024-09-17T22:42:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146222+0000", 2024-09-17T22:42:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1180000000000003e-06 2024-09-17T22:42:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146227+0000", 2024-09-17T22:42:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1749999999999998e-06 2024-09-17T22:42:31.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.187985+0000", 2024-09-17T22:42:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041758391999999998 2024-09-17T22:42:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188065+0000", 2024-09-17T22:42:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0290000000000005e-05 2024-09-17T22:42:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188164+0000", 2024-09-17T22:42:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9074999999999996e-05 2024-09-17T22:42:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157231 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4079616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.419 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.263707175999997, 2024-09-17T22:42:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042034581000000001, 2024-09-17T22:42:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.420 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157231 2024-09-17T22:42:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:31.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146345+0000", 2024-09-17T22:42:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:42:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146347+0000", 2024-09-17T22:42:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.357e-06 2024-09-17T22:42:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146352+0000", 2024-09-17T22:42:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T22:42:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146361+0000", 2024-09-17T22:42:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0219999999999993e-06 2024-09-17T22:42:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188199+0000", 2024-09-17T22:42:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041838130000000001 2024-09-17T22:42:31.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188285+0000", 2024-09-17T22:42:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5160000000000005e-05 2024-09-17T22:42:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188379+0000", 2024-09-17T22:42:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3949000000000005e-05 2024-09-17T22:42:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157233 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4085760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.263491027000001, 2024-09-17T22:42:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042008960999999997, 2024-09-17T22:42:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157233 2024-09-17T22:42:31.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146561+0000", 2024-09-17T22:42:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6300000000000005e-07 2024-09-17T22:42:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146564+0000", 2024-09-17T22:42:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-17T22:42:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146575+0000", 2024-09-17T22:42:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0891e-05 2024-09-17T22:42:31.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146579+0000", 2024-09-17T22:42:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.284e-06 2024-09-17T22:42:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188407+0000", 2024-09-17T22:42:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041828390999999999 2024-09-17T22:42:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188476+0000", 2024-09-17T22:42:31.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9129999999999997e-05 2024-09-17T22:42:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188569+0000", 2024-09-17T22:42:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2669000000000003e-05 2024-09-17T22:42:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157235 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4091904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.263297029999997, 2024-09-17T22:42:31.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042047517, 2024-09-17T22:42:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157235 2024-09-17T22:42:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146755+0000", 2024-09-17T22:42:31.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0900000000000001e-07 2024-09-17T22:42:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146758+0000", 2024-09-17T22:42:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.399e-06 2024-09-17T22:42:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146768+0000", 2024-09-17T22:42:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4730000000000006e-06 2024-09-17T22:42:31.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146774+0000", 2024-09-17T22:42:31.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6939999999999998e-06 2024-09-17T22:42:31.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188616+0000", 2024-09-17T22:42:31.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041841481999999999 2024-09-17T22:42:31.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188700+0000", 2024-09-17T22:42:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4227999999999994e-05 2024-09-17T22:42:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188802+0000", 2024-09-17T22:42:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101532 2024-09-17T22:42:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157237 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4098048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.263266758999997, 2024-09-17T22:42:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165452999999999, 2024-09-17T22:42:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.435 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157237 2024-09-17T22:42:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:31.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146785+0000", 2024-09-17T22:42:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6800000000000001e-07 2024-09-17T22:42:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146787+0000", 2024-09-17T22:42:31.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.046e-06 2024-09-17T22:42:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146794+0000", 2024-09-17T22:42:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9249999999999998e-06 2024-09-17T22:42:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146799+0000", 2024-09-17T22:42:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9289999999999997e-06 2024-09-17T22:42:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188844+0000", 2024-09-17T22:42:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042045386999999997 2024-09-17T22:42:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188888+0000", 2024-09-17T22:42:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3760000000000001e-05 2024-09-17T22:42:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188950+0000", 2024-09-17T22:42:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1938000000000002e-05 2024-09-17T22:42:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157239 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4104192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:31.440 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.263238373, 2024-09-17T22:42:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042347193999999998, 2024-09-17T22:42:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157239 2024-09-17T22:42:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.441 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:31.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:42:31.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146816+0000", 2024-09-17T22:42:31.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4700000000000001e-06 2024-09-17T22:42:31.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146822+0000", 2024-09-17T22:42:31.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.579e-06 2024-09-17T22:42:31.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146826+0000", 2024-09-17T22:42:31.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3780000000000001e-06 2024-09-17T22:42:31.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188991+0000", 2024-09-17T22:42:31.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165553000000001 2024-09-17T22:42:31.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189068+0000", 2024-09-17T22:42:31.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6548e-05 2024-09-17T22:42:31.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189160+0000", 2024-09-17T22:42:31.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2163000000000005e-05 2024-09-17T22:42:31.445 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.446 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.446 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.446 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157242 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4110336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.446 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:31.446 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.263200587, 2024-09-17T22:42:31.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042499783999999999, 2024-09-17T22:42:31.446 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.446 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.446 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.447 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.447 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.447 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.447 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157242 2024-09-17T22:42:31.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.447 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:31.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:31.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:31.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:42:31.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146852+0000", 2024-09-17T22:42:31.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.37e-06 2024-09-17T22:42:31.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146856+0000", 2024-09-17T22:42:31.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1690000000000002e-06 2024-09-17T22:42:31.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146863+0000", 2024-09-17T22:42:31.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9920000000000001e-06 2024-09-17T22:42:31.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189193+0000", 2024-09-17T22:42:31.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042329355999999999 2024-09-17T22:42:31.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189246+0000", 2024-09-17T22:42:31.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3643000000000001e-05 2024-09-17T22:42:31.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189350+0000", 2024-09-17T22:42:31.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010394 2024-09-17T22:42:31.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.451 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.451 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157244 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4116480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.451 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:31.451 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.263181375999999, 2024-09-17T22:42:31.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042675684999999998, 2024-09-17T22:42:31.451 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.451 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.451 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.451 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.452 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157244 2024-09-17T22:42:31.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.452 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:31.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:31.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:31.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1e-07 2024-09-17T22:42:31.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146871+0000", 2024-09-17T22:42:31.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.189e-06 2024-09-17T22:42:31.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146877+0000", 2024-09-17T22:42:31.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8100000000000003e-06 2024-09-17T22:42:31.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146880+0000", 2024-09-17T22:42:31.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.83e-06 2024-09-17T22:42:31.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189388+0000", 2024-09-17T22:42:31.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042507637000000001 2024-09-17T22:42:31.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189443+0000", 2024-09-17T22:42:31.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5742999999999998e-05 2024-09-17T22:42:31.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189545+0000", 2024-09-17T22:42:31.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102166 2024-09-17T22:42:31.455 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.455 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.455 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.456 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157246 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4122624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.456 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:31.456 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.263152324000004, 2024-09-17T22:42:31.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042847083000000001, 2024-09-17T22:42:31.456 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.456 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.456 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.456 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.456 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.456 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.457 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157246 2024-09-17T22:42:31.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.457 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:31.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:31.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:31.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0100000000000001e-07 2024-09-17T22:42:31.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146901+0000", 2024-09-17T22:42:31.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.531e-06 2024-09-17T22:42:31.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146905+0000", 2024-09-17T22:42:31.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0759999999999996e-06 2024-09-17T22:42:31.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146908+0000", 2024-09-17T22:42:31.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.083e-06 2024-09-17T22:42:31.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189585+0000", 2024-09-17T22:42:31.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042677009000000002 2024-09-17T22:42:31.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189656+0000", 2024-09-17T22:42:31.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0712999999999999e-05 2024-09-17T22:42:31.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189746+0000", 2024-09-17T22:42:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0370000000000004e-05 2024-09-17T22:42:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157247 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4128768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.263080488, 2024-09-17T22:42:31.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043016916000000002, 2024-09-17T22:42:31.462 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.462 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.462 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.462 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.462 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.462 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157247 2024-09-17T22:42:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:31.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147071+0000", 2024-09-17T22:42:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100769 2024-09-17T22:42:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147081+0000", 2024-09-17T22:42:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9229999999999997e-06 2024-09-17T22:42:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147090+0000", 2024-09-17T22:42:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.935e-06 2024-09-17T22:42:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147097+0000", 2024-09-17T22:42:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T22:42:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189787+0000", 2024-09-17T22:42:31.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042690525 2024-09-17T22:42:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189884+0000", 2024-09-17T22:42:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6858e-05 2024-09-17T22:42:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189988+0000", 2024-09-17T22:42:31.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103336 2024-09-17T22:42:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157250 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4134912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.262937027, 2024-09-17T22:42:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043099446, 2024-09-17T22:42:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.467 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157250 2024-09-17T22:42:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147235+0000", 2024-09-17T22:42:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120975 2024-09-17T22:42:31.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147238+0000", 2024-09-17T22:42:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5289999999999998e-06 2024-09-17T22:42:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147244+0000", 2024-09-17T22:42:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6050000000000003e-06 2024-09-17T22:42:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147249+0000", 2024-09-17T22:42:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4939999999999997e-06 2024-09-17T22:42:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190020+0000", 2024-09-17T22:42:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042771300999999998 2024-09-17T22:42:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190102+0000", 2024-09-17T22:42:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1600000000000005e-05 2024-09-17T22:42:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190214+0000", 2024-09-17T22:42:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011194200000000001 2024-09-17T22:42:31.472 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157252 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4141056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.262794358000001, 2024-09-17T22:42:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043165508999999998, 2024-09-17T22:42:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.473 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157252 2024-09-17T22:42:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:31.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147402+0000", 2024-09-17T22:42:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014475600000000001 2024-09-17T22:42:31.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147411+0000", 2024-09-17T22:42:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4350000000000003e-06 2024-09-17T22:42:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147420+0000", 2024-09-17T22:42:31.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4790000000000002e-06 2024-09-17T22:42:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147427+0000", 2024-09-17T22:42:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5400000000000001e-06 2024-09-17T22:42:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190254+0000", 2024-09-17T22:42:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042826731 2024-09-17T22:42:31.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190326+0000", 2024-09-17T22:42:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1798999999999995e-05 2024-09-17T22:42:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190422+0000", 2024-09-17T22:42:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6768999999999995e-05 2024-09-17T22:42:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157253 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4147200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.262607992, 2024-09-17T22:42:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043157752000000001, 2024-09-17T22:42:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.479 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157253 2024-09-17T22:42:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147590+0000", 2024-09-17T22:42:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014709 2024-09-17T22:42:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147593+0000", 2024-09-17T22:42:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5840000000000002e-06 2024-09-17T22:42:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147598+0000", 2024-09-17T22:42:31.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2469999999999999e-06 2024-09-17T22:42:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147602+0000", 2024-09-17T22:42:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0049999999999998e-06 2024-09-17T22:42:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190475+0000", 2024-09-17T22:42:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872767999999999 2024-09-17T22:42:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190528+0000", 2024-09-17T22:42:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2889999999999997e-05 2024-09-17T22:42:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190601+0000", 2024-09-17T22:42:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3168000000000001e-05 2024-09-17T22:42:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.483 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157255 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4153344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.262442010999997, 2024-09-17T22:42:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043132251000000003, 2024-09-17T22:42:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157255 2024-09-17T22:42:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147820+0000", 2024-09-17T22:42:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021123600000000001 2024-09-17T22:42:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147824+0000", 2024-09-17T22:42:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6780000000000002e-06 2024-09-17T22:42:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147833+0000", 2024-09-17T22:42:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4250000000000004e-06 2024-09-17T22:42:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147842+0000", 2024-09-17T22:42:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0679999999999997e-06 2024-09-17T22:42:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190632+0000", 2024-09-17T22:42:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042790863999999998 2024-09-17T22:42:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190674+0000", 2024-09-17T22:42:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1804000000000002e-05 2024-09-17T22:42:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190741+0000", 2024-09-17T22:42:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7175999999999995e-05 2024-09-17T22:42:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157257 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4159488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.488 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.262201521000001, 2024-09-17T22:42:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.04307035, 2024-09-17T22:42:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157257 2024-09-17T22:42:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.489 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148844+0000", 2024-09-17T22:42:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00099455599999999996 2024-09-17T22:42:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148848+0000", 2024-09-17T22:42:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0180000000000003e-06 2024-09-17T22:42:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148858+0000", 2024-09-17T22:42:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.825e-06 2024-09-17T22:42:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148865+0000", 2024-09-17T22:42:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8290000000000002e-06 2024-09-17T22:42:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.492 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190789+0000", 2024-09-17T22:42:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041924630999999997 2024-09-17T22:42:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190860+0000", 2024-09-17T22:42:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0557999999999998e-05 2024-09-17T22:42:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190920+0000", 2024-09-17T22:42:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9932999999999997e-05 2024-09-17T22:42:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157259 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4165632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:31.494 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.261175502, 2024-09-17T22:42:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042261411999999998, 2024-09-17T22:42:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157259 2024-09-17T22:42:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.496 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:42:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148878+0000", 2024-09-17T22:42:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:42:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148885+0000", 2024-09-17T22:42:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7290000000000003e-06 2024-09-17T22:42:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148893+0000", 2024-09-17T22:42:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8110000000000003e-06 2024-09-17T22:42:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190967+0000", 2024-09-17T22:42:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042073948999999999 2024-09-17T22:42:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191069+0000", 2024-09-17T22:42:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102075 2024-09-17T22:42:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191137+0000", 2024-09-17T22:42:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8281000000000006e-05 2024-09-17T22:42:31.499 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157261 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4171776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.261125894999999, 2024-09-17T22:42:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042437255, 2024-09-17T22:42:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.500 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157261 2024-09-17T22:42:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:31.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148926+0000", 2024-09-17T22:42:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T22:42:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148928+0000", 2024-09-17T22:42:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4329999999999998e-06 2024-09-17T22:42:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148936+0000", 2024-09-17T22:42:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6969999999999993e-06 2024-09-17T22:42:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148942+0000", 2024-09-17T22:42:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9479999999999996e-06 2024-09-17T22:42:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191185+0000", 2024-09-17T22:42:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042243147000000002 2024-09-17T22:42:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191266+0000", 2024-09-17T22:42:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1186000000000005e-05 2024-09-17T22:42:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191362+0000", 2024-09-17T22:42:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6384000000000004e-05 2024-09-17T22:42:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157263 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4177920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:31.505 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.261101496000002, 2024-09-17T22:42:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042636395000000001, 2024-09-17T22:42:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157263 2024-09-17T22:42:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.37e-07 2024-09-17T22:42:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148952+0000", 2024-09-17T22:42:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.88e-06 2024-09-17T22:42:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148959+0000", 2024-09-17T22:42:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5649999999999998e-06 2024-09-17T22:42:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148963+0000", 2024-09-17T22:42:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8570000000000003e-06 2024-09-17T22:42:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191408+0000", 2024-09-17T22:42:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042444707999999998 2024-09-17T22:42:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191489+0000", 2024-09-17T22:42:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0612999999999995e-05 2024-09-17T22:42:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191586+0000", 2024-09-17T22:42:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7235e-05 2024-09-17T22:42:31.509 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157265 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4184064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.261079946999999, 2024-09-17T22:42:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042791830000000003, 2024-09-17T22:42:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157265 2024-09-17T22:42:31.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148972+0000", 2024-09-17T22:42:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:42:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148973+0000", 2024-09-17T22:42:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.894e-06 2024-09-17T22:42:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148985+0000", 2024-09-17T22:42:31.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1995e-05 2024-09-17T22:42:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148990+0000", 2024-09-17T22:42:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8269999999999997e-06 2024-09-17T22:42:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191630+0000", 2024-09-17T22:42:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042639639 2024-09-17T22:42:31.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191686+0000", 2024-09-17T22:42:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6066000000000002e-05 2024-09-17T22:42:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191763+0000", 2024-09-17T22:42:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7035999999999999e-05 2024-09-17T22:42:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.514 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157267 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4190208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 41.261054133999998, 2024-09-17T22:42:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043028813999999999, 2024-09-17T22:42:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.515 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157267 2024-09-17T22:42:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:42:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148999+0000", 2024-09-17T22:42:31.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.432e-06 2024-09-17T22:42:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149002+0000", 2024-09-17T22:42:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7850000000000002e-06 2024-09-17T22:42:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149005+0000", 2024-09-17T22:42:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:42:31.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191878+0000", 2024-09-17T22:42:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872312000000003 2024-09-17T22:42:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191961+0000", 2024-09-17T22:42:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3418000000000007e-05 2024-09-17T22:42:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.192026+0000", 2024-09-17T22:42:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4689000000000004e-05 2024-09-17T22:42:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.520 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.520 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:31.521 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:42:31.522 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:31.519+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:31.623 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:31.620+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:31.626 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:31.626 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:31.626 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:31.626 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:31.635 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:42:31.723 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:31.721+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:31.738 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:31.738 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:31.738 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:31.738 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:31.738 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:31.746 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:42:31.825 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:31.822+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:31.890 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:31.891 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:31.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:31.891 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:31.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.891 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188693 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5881856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:31.891 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:31.891 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.060304712000001, 2024-09-17T22:42:31.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028193223, 2024-09-17T22:42:31.891 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.891 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.891 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.892 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188693 2024-09-17T22:42:31.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.892 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:31.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:31.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:31.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:42:31.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827090+0000", 2024-09-17T22:42:31.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2170000000000001e-06 2024-09-17T22:42:31.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827098+0000", 2024-09-17T22:42:31.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1510000000000004e-06 2024-09-17T22:42:31.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827103+0000", 2024-09-17T22:42:31.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0839999999999999e-06 2024-09-17T22:42:31.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855158+0000", 2024-09-17T22:42:31.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028055217 2024-09-17T22:42:31.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855201+0000", 2024-09-17T22:42:31.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3272000000000002e-05 2024-09-17T22:42:31.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855280+0000", 2024-09-17T22:42:31.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8845999999999997e-05 2024-09-17T22:42:31.896 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.896 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.896 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.897 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188694 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5971968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:31.897 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:31.897 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.060280825, 2024-09-17T22:42:31.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028312278, 2024-09-17T22:42:31.897 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.897 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.897 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.897 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.897 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.897 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.898 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188694 2024-09-17T22:42:31.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.898 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:31.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:31.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:31.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:42:31.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827114+0000", 2024-09-17T22:42:31.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3029999999999998e-06 2024-09-17T22:42:31.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827121+0000", 2024-09-17T22:42:31.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3699999999999997e-06 2024-09-17T22:42:31.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827126+0000", 2024-09-17T22:42:31.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7350000000000004e-06 2024-09-17T22:42:31.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855307+0000", 2024-09-17T22:42:31.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028180851999999999 2024-09-17T22:42:31.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855346+0000", 2024-09-17T22:42:31.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9274999999999999e-05 2024-09-17T22:42:31.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855423+0000", 2024-09-17T22:42:31.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7294999999999998e-05 2024-09-17T22:42:31.901 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.901 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.901 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.901 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188695 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:31.901 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:31.902 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.060260047, 2024-09-17T22:42:31.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028435876999999998, 2024-09-17T22:42:31.902 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.902 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.902 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.902 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.902 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.902 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.902 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188695 2024-09-17T22:42:31.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.902 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:31.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:31.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:31.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:42:31.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827138+0000", 2024-09-17T22:42:31.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2580000000000001e-06 2024-09-17T22:42:31.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827145+0000", 2024-09-17T22:42:31.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0129999999999996e-06 2024-09-17T22:42:31.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827150+0000", 2024-09-17T22:42:31.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3749999999999996e-06 2024-09-17T22:42:31.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855447+0000", 2024-09-17T22:42:31.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028297224999999999 2024-09-17T22:42:31.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855487+0000", 2024-09-17T22:42:31.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0454000000000003e-05 2024-09-17T22:42:31.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855567+0000", 2024-09-17T22:42:31.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0149999999999997e-05 2024-09-17T22:42:31.906 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.906 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.906 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.906 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188696 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:31.906 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:31.906 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.060236263, 2024-09-17T22:42:31.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028543570000000001, 2024-09-17T22:42:31.906 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.906 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.906 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.907 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.907 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.907 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.907 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188696 2024-09-17T22:42:31.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.907 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:31.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:31.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827156+0000", 2024-09-17T22:42:31.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T22:42:31.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827158+0000", 2024-09-17T22:42:31.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.875e-06 2024-09-17T22:42:31.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827164+0000", 2024-09-17T22:42:31.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7950000000000002e-06 2024-09-17T22:42:31.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827169+0000", 2024-09-17T22:42:31.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2880000000000003e-06 2024-09-17T22:42:31.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855582+0000", 2024-09-17T22:42:31.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028412929 2024-09-17T22:42:31.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855624+0000", 2024-09-17T22:42:31.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2484000000000002e-05 2024-09-17T22:42:31.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855699+0000", 2024-09-17T22:42:31.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4869000000000004e-05 2024-09-17T22:42:31.911 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.911 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.911 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.911 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188697 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6242304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:31.912 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:31.912 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.060204733999999, 2024-09-17T22:42:31.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028658598, 2024-09-17T22:42:31.912 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.912 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.912 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.912 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.912 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.912 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.912 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188697 2024-09-17T22:42:31.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.913 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:31.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:31.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:31.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8800000000000003e-07 2024-09-17T22:42:31.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827190+0000", 2024-09-17T22:42:31.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.351e-06 2024-09-17T22:42:31.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827198+0000", 2024-09-17T22:42:31.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.909e-06 2024-09-17T22:42:31.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827203+0000", 2024-09-17T22:42:31.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4020000000000001e-06 2024-09-17T22:42:31.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855720+0000", 2024-09-17T22:42:31.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028516909999999999 2024-09-17T22:42:31.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855762+0000", 2024-09-17T22:42:31.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2184999999999997e-05 2024-09-17T22:42:31.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855845+0000", 2024-09-17T22:42:31.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3353e-05 2024-09-17T22:42:31.917 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.917 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.917 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.917 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202883 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2342912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:31.917 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:42:31.917 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.162568352999999, 2024-09-17T22:42:31.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028098636, 2024-09-17T22:42:31.918 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.918 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.918 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.918 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.918 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202883 2024-09-17T22:42:31.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.918 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:42:31.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:42:31.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724824+0000", 2024-09-17T22:42:31.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:42:31.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724826+0000", 2024-09-17T22:42:31.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.086e-06 2024-09-17T22:42:31.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724832+0000", 2024-09-17T22:42:31.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9100000000000002e-06 2024-09-17T22:42:31.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724837+0000", 2024-09-17T22:42:31.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9439999999999996e-06 2024-09-17T22:42:31.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752790+0000", 2024-09-17T22:42:31.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027953657999999999 2024-09-17T22:42:31.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752848+0000", 2024-09-17T22:42:31.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7677999999999999e-05 2024-09-17T22:42:31.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752922+0000", 2024-09-17T22:42:31.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3957999999999999e-05 2024-09-17T22:42:31.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.922 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.923 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.923 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202884 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2473984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:31.923 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:42:31.923 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.162540128, 2024-09-17T22:42:31.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028206208999999999, 2024-09-17T22:42:31.923 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.923 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.923 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.924 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.924 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.924 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.924 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202884 2024-09-17T22:42:31.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.924 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:42:31.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:42:31.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724852+0000", 2024-09-17T22:42:31.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2599999999999998e-07 2024-09-17T22:42:31.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724854+0000", 2024-09-17T22:42:31.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.26e-06 2024-09-17T22:42:31.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724871+0000", 2024-09-17T22:42:31.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6589999999999999e-05 2024-09-17T22:42:31.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724877+0000", 2024-09-17T22:42:31.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.054e-06 2024-09-17T22:42:31.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752951+0000", 2024-09-17T22:42:31.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028073747 2024-09-17T22:42:31.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752993+0000", 2024-09-17T22:42:31.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2082000000000001e-05 2024-09-17T22:42:31.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753058+0000", 2024-09-17T22:42:31.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5049999999999996e-05 2024-09-17T22:42:31.927 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.927 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.927 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.928 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202885 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:31.928 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:31.928 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.162506824999999, 2024-09-17T22:42:31.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028333028, 2024-09-17T22:42:31.928 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.928 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.928 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.928 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.928 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.928 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.928 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202885 2024-09-17T22:42:31.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.929 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:31.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:31.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:31.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3300000000000002e-07 2024-09-17T22:42:31.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724892+0000", 2024-09-17T22:42:31.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2779999999999999e-06 2024-09-17T22:42:31.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724899+0000", 2024-09-17T22:42:31.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9110000000000004e-06 2024-09-17T22:42:31.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724904+0000", 2024-09-17T22:42:31.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2360000000000004e-06 2024-09-17T22:42:31.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753094+0000", 2024-09-17T22:42:31.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02819033 2024-09-17T22:42:31.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753142+0000", 2024-09-17T22:42:31.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.829e-05 2024-09-17T22:42:31.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753218+0000", 2024-09-17T22:42:31.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5450000000000004e-05 2024-09-17T22:42:31.932 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.932 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.932 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.932 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202886 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:31.932 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:31.933 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.162481627, 2024-09-17T22:42:31.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028442512, 2024-09-17T22:42:31.933 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.933 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.933 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.933 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.933 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.933 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.933 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202886 2024-09-17T22:42:31.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.933 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:31.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:31.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:31.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:42:31.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724912+0000", 2024-09-17T22:42:31.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.967e-06 2024-09-17T22:42:31.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724918+0000", 2024-09-17T22:42:31.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8069999999999996e-06 2024-09-17T22:42:31.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724923+0000", 2024-09-17T22:42:31.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.386e-06 2024-09-17T22:42:31.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753244+0000", 2024-09-17T22:42:31.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028321567999999998 2024-09-17T22:42:31.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753289+0000", 2024-09-17T22:42:31.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4620000000000003e-05 2024-09-17T22:42:31.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753353+0000", 2024-09-17T22:42:31.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3795999999999997e-05 2024-09-17T22:42:31.937 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.937 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.938 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.938 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202887 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:31.938 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:31.938 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.162463713999999, 2024-09-17T22:42:31.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028565838999999999, 2024-09-17T22:42:31.938 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.938 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.938 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.938 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.939 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.939 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.939 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202887 2024-09-17T22:42:31.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.939 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:31.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:31.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:31.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:42:31.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724930+0000", 2024-09-17T22:42:31.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8980000000000001e-06 2024-09-17T22:42:31.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724936+0000", 2024-09-17T22:42:31.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7849999999999997e-06 2024-09-17T22:42:31.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724942+0000", 2024-09-17T22:42:31.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9649999999999996e-06 2024-09-17T22:42:31.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753380+0000", 2024-09-17T22:42:31.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028438049 2024-09-17T22:42:31.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753425+0000", 2024-09-17T22:42:31.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5163000000000001e-05 2024-09-17T22:42:31.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753494+0000", 2024-09-17T22:42:31.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8588000000000001e-05 2024-09-17T22:42:31.943 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.943 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.943 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.944 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202888 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:31.944 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:42:31.944 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.162444216000001, 2024-09-17T22:42:31.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028703314000000001, 2024-09-17T22:42:31.944 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.944 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.944 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.944 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.944 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.944 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.944 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202888 2024-09-17T22:42:31.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.945 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:42:31.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:42:31.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724948+0000", 2024-09-17T22:42:31.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:42:31.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724950+0000", 2024-09-17T22:42:31.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8029999999999999e-06 2024-09-17T22:42:31.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724955+0000", 2024-09-17T22:42:31.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8760000000000002e-06 2024-09-17T22:42:31.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724960+0000", 2024-09-17T22:42:31.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5979999999999999e-06 2024-09-17T22:42:31.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753527+0000", 2024-09-17T22:42:31.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028566864000000001 2024-09-17T22:42:31.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753568+0000", 2024-09-17T22:42:31.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0775000000000002e-05 2024-09-17T22:42:31.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753651+0000", 2024-09-17T22:42:31.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3039000000000002e-05 2024-09-17T22:42:31.949 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.949 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.949 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.949 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202889 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:31.949 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:31.949 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.162426644, 2024-09-17T22:42:31.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028820964000000001, 2024-09-17T22:42:31.949 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.950 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.950 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.950 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.950 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.950 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.950 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202889 2024-09-17T22:42:31.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.950 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:31.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:31.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:31.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2599999999999998e-07 2024-09-17T22:42:31.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724967+0000", 2024-09-17T22:42:31.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T22:42:31.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724973+0000", 2024-09-17T22:42:31.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6060000000000002e-06 2024-09-17T22:42:31.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724978+0000", 2024-09-17T22:42:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8230000000000002e-06 2024-09-17T22:42:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753687+0000", 2024-09-17T22:42:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028709495000000002 2024-09-17T22:42:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753727+0000", 2024-09-17T22:42:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9953999999999997e-05 2024-09-17T22:42:31.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753786+0000", 2024-09-17T22:42:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8941999999999999e-05 2024-09-17T22:42:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202890 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.162409075999999, 2024-09-17T22:42:31.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028933838, 2024-09-17T22:42:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202890 2024-09-17T22:42:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5999999999999999e-07 2024-09-17T22:42:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724985+0000", 2024-09-17T22:42:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8169999999999999e-06 2024-09-17T22:42:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724991+0000", 2024-09-17T22:42:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:42:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724995+0000", 2024-09-17T22:42:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4530000000000004e-06 2024-09-17T22:42:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753811+0000", 2024-09-17T22:42:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028815584000000002 2024-09-17T22:42:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753854+0000", 2024-09-17T22:42:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3356999999999998e-05 2024-09-17T22:42:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753916+0000", 2024-09-17T22:42:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2488000000000002e-05 2024-09-17T22:42:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.958 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202891 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.162391618999999, 2024-09-17T22:42:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029067554999999998, 2024-09-17T22:42:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202891 2024-09-17T22:42:31.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:42:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725002+0000", 2024-09-17T22:42:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:42:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725008+0000", 2024-09-17T22:42:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6629999999999998e-06 2024-09-17T22:42:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.961 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725016+0000", 2024-09-17T22:42:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8269999999999998e-06 2024-09-17T22:42:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753949+0000", 2024-09-17T22:42:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028932625999999999 2024-09-17T22:42:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753994+0000", 2024-09-17T22:42:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5717000000000003e-05 2024-09-17T22:42:31.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754068+0000", 2024-09-17T22:42:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3312000000000006e-05 2024-09-17T22:42:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202892 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:31.963 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:31.964 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.162348912000001, 2024-09-17T22:42:31.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029159939999999999, 2024-09-17T22:42:31.964 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.964 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.964 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.964 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202892 2024-09-17T22:42:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7999999999999995e-07 2024-09-17T22:42:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725045+0000", 2024-09-17T22:42:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-17T22:42:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725052+0000", 2024-09-17T22:42:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3629999999999999e-06 2024-09-17T22:42:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725056+0000", 2024-09-17T22:42:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3250000000000001e-06 2024-09-17T22:42:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754094+0000", 2024-09-17T22:42:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029038270000000001 2024-09-17T22:42:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754134+0000", 2024-09-17T22:42:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9722999999999999e-05 2024-09-17T22:42:31.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754203+0000", 2024-09-17T22:42:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8524999999999999e-05 2024-09-17T22:42:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202893 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:31.969 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:42:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.162329335000001, 2024-09-17T22:42:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029341651, 2024-09-17T22:42:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202893 2024-09-17T22:42:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:42:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:42:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725063+0000", 2024-09-17T22:42:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6699999999999999e-07 2024-09-17T22:42:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725065+0000", 2024-09-17T22:42:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2120000000000002e-06 2024-09-17T22:42:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725071+0000", 2024-09-17T22:42:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5169999999999996e-06 2024-09-17T22:42:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725076+0000", 2024-09-17T22:42:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.78e-06 2024-09-17T22:42:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754233+0000", 2024-09-17T22:42:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029156277000000001 2024-09-17T22:42:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754298+0000", 2024-09-17T22:42:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5320000000000005e-05 2024-09-17T22:42:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.974 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754404+0000", 2024-09-17T22:42:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106078 2024-09-17T22:42:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202894 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:42:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.162309455999999, 2024-09-17T22:42:31.975 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029456572, 2024-09-17T22:42:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202894 2024-09-17T22:42:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.976 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:42:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:42:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.977 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725083+0000", 2024-09-17T22:42:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5900000000000002e-07 2024-09-17T22:42:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725085+0000", 2024-09-17T22:42:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.979e-06 2024-09-17T22:42:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725091+0000", 2024-09-17T22:42:31.978 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1480000000000003e-06 2024-09-17T22:42:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725095+0000", 2024-09-17T22:42:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1339999999999997e-06 2024-09-17T22:42:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754429+0000", 2024-09-17T22:42:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029333758000000001 2024-09-17T22:42:31.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754471+0000", 2024-09-17T22:42:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2274999999999998e-05 2024-09-17T22:42:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754539+0000", 2024-09-17T22:42:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7818999999999996e-05 2024-09-17T22:42:31.980 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202895 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.162291059999999, 2024-09-17T22:42:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029635578999999999, 2024-09-17T22:42:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.981 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202895 2024-09-17T22:42:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.982 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.08e-07 2024-09-17T22:42:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725107+0000", 2024-09-17T22:42:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9020000000000004e-06 2024-09-17T22:42:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.983 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725113+0000", 2024-09-17T22:42:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7400000000000001e-06 2024-09-17T22:42:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725117+0000", 2024-09-17T22:42:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:42:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754567+0000", 2024-09-17T22:42:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029449411000000002 2024-09-17T22:42:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.984 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754638+0000", 2024-09-17T22:42:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1432000000000003e-05 2024-09-17T22:42:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754736+0000", 2024-09-17T22:42:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8066000000000006e-05 2024-09-17T22:42:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.985 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202896 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:42:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.162269329000001, 2024-09-17T22:42:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029760534000000002, 2024-09-17T22:42:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202896 2024-09-17T22:42:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.986 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:42:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:42:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725123+0000", 2024-09-17T22:42:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:42:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.987 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725125+0000", 2024-09-17T22:42:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.863e-06 2024-09-17T22:42:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725130+0000", 2024-09-17T22:42:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7250000000000002e-06 2024-09-17T22:42:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725135+0000", 2024-09-17T22:42:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:42:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.988 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754763+0000", 2024-09-17T22:42:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029628202999999999 2024-09-17T22:42:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754805+0000", 2024-09-17T22:42:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2370000000000003e-05 2024-09-17T22:42:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754883+0000", 2024-09-17T22:42:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7565999999999996e-05 2024-09-17T22:42:31.989 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202897 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.162251969, 2024-09-17T22:42:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029947972999999999, 2024-09-17T22:42:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:31.990 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202897 2024-09-17T22:42:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:42:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.992 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725146+0000", 2024-09-17T22:42:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6030000000000004e-06 2024-09-17T22:42:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725152+0000", 2024-09-17T22:42:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9970000000000004e-06 2024-09-17T22:42:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.993 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725156+0000", 2024-09-17T22:42:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-06 2024-09-17T22:42:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754909+0000", 2024-09-17T22:42:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029752958999999999 2024-09-17T22:42:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:31.994 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754987+0000", 2024-09-17T22:42:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8208999999999996e-05 2024-09-17T22:42:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.755088+0000", 2024-09-17T22:42:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100348 2024-09-17T22:42:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.995 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:31.996 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:31.996 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:31.997 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:42:31.998 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:31.995+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:32.098 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:32.095+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:32.100 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:32.101 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:32.101 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:32.101 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:32.103 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:42:32.109 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:42:32.199 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:32.196+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:32.212 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:32.212 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:32.212 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:32.212 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:32.213 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:32.218 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:32.221 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:42:32.235 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:32.235 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:32.252 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:32.252 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:32.267 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:32.268 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:32.284 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:32.285 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:32.300 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:32.297+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:32.301 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:32.301 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:32.318 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:32.319 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:32.330 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:32.330 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:32.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:32.330 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:32.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.330 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202974 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.330 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725913+0000", 2024-09-17T22:42:32.330 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.601048714999999, 2024-09-17T22:42:32.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022636643000000001, 2024-09-17T22:42:32.331 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.331 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.331 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.331 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.331 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.331 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.331 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202974 2024-09-17T22:42:32.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.331 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725913+0000", 2024-09-17T22:42:32.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725913+0000", 2024-09-17T22:42:32.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725913+0000", 2024-09-17T22:42:32.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5199999999999997e-07 2024-09-17T22:42:32.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725916+0000", 2024-09-17T22:42:32.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8100000000000002e-06 2024-09-17T22:42:32.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725924+0000", 2024-09-17T22:42:32.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1289999999999996e-06 2024-09-17T22:42:32.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725930+0000", 2024-09-17T22:42:32.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3469999999999998e-06 2024-09-17T22:42:32.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748360+0000", 2024-09-17T22:42:32.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022430410000000001 2024-09-17T22:42:32.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748443+0000", 2024-09-17T22:42:32.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3355000000000004e-05 2024-09-17T22:42:32.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748550+0000", 2024-09-17T22:42:32.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010603999999999999 2024-09-17T22:42:32.334 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.335 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.335 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.335 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202975 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.335 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725937+0000", 2024-09-17T22:42:32.335 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.601024676, 2024-09-17T22:42:32.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022787498, 2024-09-17T22:42:32.335 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.335 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.335 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.335 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.335 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.335 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.336 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202975 2024-09-17T22:42:32.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.336 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725937+0000", 2024-09-17T22:42:32.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725937+0000", 2024-09-17T22:42:32.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725937+0000", 2024-09-17T22:42:32.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4299999999999998e-07 2024-09-17T22:42:32.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725939+0000", 2024-09-17T22:42:32.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0669999999999999e-06 2024-09-17T22:42:32.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725946+0000", 2024-09-17T22:42:32.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6390000000000003e-06 2024-09-17T22:42:32.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725951+0000", 2024-09-17T22:42:32.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0089999999999998e-06 2024-09-17T22:42:32.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748593+0000", 2024-09-17T22:42:32.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022642343999999998 2024-09-17T22:42:32.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748657+0000", 2024-09-17T22:42:32.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3133999999999996e-05 2024-09-17T22:42:32.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748724+0000", 2024-09-17T22:42:32.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7861999999999995e-05 2024-09-17T22:42:32.339 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.339 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.339 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.339 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202976 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.340 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725958+0000", 2024-09-17T22:42:32.340 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.601003495, 2024-09-17T22:42:32.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022890631000000002, 2024-09-17T22:42:32.340 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.340 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.340 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.340 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.340 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.340 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.340 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202976 2024-09-17T22:42:32.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.340 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725958+0000", 2024-09-17T22:42:32.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725958+0000", 2024-09-17T22:42:32.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725959+0000", 2024-09-17T22:42:32.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7199999999999999e-07 2024-09-17T22:42:32.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725961+0000", 2024-09-17T22:42:32.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9860000000000001e-06 2024-09-17T22:42:32.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725967+0000", 2024-09-17T22:42:32.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6630000000000004e-06 2024-09-17T22:42:32.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725972+0000", 2024-09-17T22:42:32.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0549999999999999e-06 2024-09-17T22:42:32.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748753+0000", 2024-09-17T22:42:32.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022780959 2024-09-17T22:42:32.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748792+0000", 2024-09-17T22:42:32.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8581999999999998e-05 2024-09-17T22:42:32.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748849+0000", 2024-09-17T22:42:32.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6913999999999998e-05 2024-09-17T22:42:32.344 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.344 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.344 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.344 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202977 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.345 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:42:32.345 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.600982538, 2024-09-17T22:42:32.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023068750999999998, 2024-09-17T22:42:32.345 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.345 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.345 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.346 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202977 2024-09-17T22:42:32.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.346 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:42:32.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:42:32.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725980+0000", 2024-09-17T22:42:32.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:42:32.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725982+0000", 2024-09-17T22:42:32.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0669999999999999e-06 2024-09-17T22:42:32.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725988+0000", 2024-09-17T22:42:32.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6610000000000003e-06 2024-09-17T22:42:32.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725993+0000", 2024-09-17T22:42:32.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:42:32.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748888+0000", 2024-09-17T22:42:32.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022894930000000001 2024-09-17T22:42:32.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748951+0000", 2024-09-17T22:42:32.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3638000000000003e-05 2024-09-17T22:42:32.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749048+0000", 2024-09-17T22:42:32.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6343999999999998e-05 2024-09-17T22:42:32.350 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.350 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.350 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.350 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202978 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2211840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.351 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:32.351 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.600962025999999, 2024-09-17T22:42:32.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023238050999999999, 2024-09-17T22:42:32.351 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.351 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.351 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.351 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.351 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.351 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.351 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202978 2024-09-17T22:42:32.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.352 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:32.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:32.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:32.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.58e-07 2024-09-17T22:42:32.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726002+0000", 2024-09-17T22:42:32.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.193e-06 2024-09-17T22:42:32.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726009+0000", 2024-09-17T22:42:32.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8000000000000001e-06 2024-09-17T22:42:32.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726014+0000", 2024-09-17T22:42:32.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9840000000000001e-06 2024-09-17T22:42:32.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749090+0000", 2024-09-17T22:42:32.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023075917000000001 2024-09-17T22:42:32.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749163+0000", 2024-09-17T22:42:32.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3219999999999994e-05 2024-09-17T22:42:32.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749238+0000", 2024-09-17T22:42:32.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4479000000000003e-05 2024-09-17T22:42:32.356 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.356 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.356 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.356 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202979 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2342912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.356 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:42:32.356 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.600940574999999, 2024-09-17T22:42:32.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023393536999999999, 2024-09-17T22:42:32.356 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.356 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.357 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.357 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.357 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.357 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.357 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202979 2024-09-17T22:42:32.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.357 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:42:32.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:42:32.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726022+0000", 2024-09-17T22:42:32.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:42:32.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726024+0000", 2024-09-17T22:42:32.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1969999999999999e-06 2024-09-17T22:42:32.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726030+0000", 2024-09-17T22:42:32.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.691e-06 2024-09-17T22:42:32.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726035+0000", 2024-09-17T22:42:32.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8029999999999996e-06 2024-09-17T22:42:32.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749280+0000", 2024-09-17T22:42:32.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023244668 2024-09-17T22:42:32.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749344+0000", 2024-09-17T22:42:32.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4418000000000006e-05 2024-09-17T22:42:32.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749415+0000", 2024-09-17T22:42:32.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0272999999999996e-05 2024-09-17T22:42:32.361 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.361 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.361 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.361 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202980 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2473984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.361 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:32.361 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.600919806, 2024-09-17T22:42:32.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023562221000000001, 2024-09-17T22:42:32.361 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.362 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.362 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.362 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.362 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.362 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.362 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202980 2024-09-17T22:42:32.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.362 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:32.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:32.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:32.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:42:32.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726044+0000", 2024-09-17T22:42:32.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9489999999999999e-06 2024-09-17T22:42:32.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726066+0000", 2024-09-17T22:42:32.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1508000000000001e-05 2024-09-17T22:42:32.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726072+0000", 2024-09-17T22:42:32.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3450000000000002e-06 2024-09-17T22:42:32.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749455+0000", 2024-09-17T22:42:32.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023382824999999999 2024-09-17T22:42:32.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749527+0000", 2024-09-17T22:42:32.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1728000000000002e-05 2024-09-17T22:42:32.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749604+0000", 2024-09-17T22:42:32.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7428999999999993e-05 2024-09-17T22:42:32.366 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.366 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.366 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.366 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202981 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.366 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:42:32.366 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.600881531000001, 2024-09-17T22:42:32.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023677800999999998, 2024-09-17T22:42:32.367 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.367 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.367 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.367 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.367 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.367 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.367 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202981 2024-09-17T22:42:32.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.367 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:42:32.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:42:32.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726081+0000", 2024-09-17T22:42:32.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5499999999999998e-07 2024-09-17T22:42:32.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726083+0000", 2024-09-17T22:42:32.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1780000000000002e-06 2024-09-17T22:42:32.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726090+0000", 2024-09-17T22:42:32.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8510000000000001e-06 2024-09-17T22:42:32.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726101+0000", 2024-09-17T22:42:32.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.133e-05 2024-09-17T22:42:32.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749639+0000", 2024-09-17T22:42:32.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023537679999999998 2024-09-17T22:42:32.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749688+0000", 2024-09-17T22:42:32.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9548000000000001e-05 2024-09-17T22:42:32.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749758+0000", 2024-09-17T22:42:32.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9758999999999994e-05 2024-09-17T22:42:32.371 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.371 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.371 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.371 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202982 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.371 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:32.371 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.600853947999999, 2024-09-17T22:42:32.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023841469000000001, 2024-09-17T22:42:32.372 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.372 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.372 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.372 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.372 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.372 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.372 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202982 2024-09-17T22:42:32.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.372 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:32.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:32.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:32.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:42:32.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726110+0000", 2024-09-17T22:42:32.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.902e-06 2024-09-17T22:42:32.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726117+0000", 2024-09-17T22:42:32.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6189999999999996e-06 2024-09-17T22:42:32.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726122+0000", 2024-09-17T22:42:32.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.198e-06 2024-09-17T22:42:32.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749795+0000", 2024-09-17T22:42:32.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023673397999999998 2024-09-17T22:42:32.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749877+0000", 2024-09-17T22:42:32.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1910999999999996e-05 2024-09-17T22:42:32.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749949+0000", 2024-09-17T22:42:32.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2001000000000004e-05 2024-09-17T22:42:32.376 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.376 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.376 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.377 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202983 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.377 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:32.377 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.600832888999999, 2024-09-17T22:42:32.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024014081, 2024-09-17T22:42:32.377 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.377 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.377 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.377 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.377 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.377 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.378 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202983 2024-09-17T22:42:32.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.378 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:32.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:32.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:32.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:42:32.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726131+0000", 2024-09-17T22:42:32.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9429999999999999e-06 2024-09-17T22:42:32.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726138+0000", 2024-09-17T22:42:32.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T22:42:32.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726143+0000", 2024-09-17T22:42:32.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0629999999999996e-06 2024-09-17T22:42:32.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749991+0000", 2024-09-17T22:42:32.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023847671000000001 2024-09-17T22:42:32.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750065+0000", 2024-09-17T22:42:32.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4569999999999999e-05 2024-09-17T22:42:32.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750143+0000", 2024-09-17T22:42:32.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7649000000000001e-05 2024-09-17T22:42:32.382 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.382 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.382 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.382 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202984 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.382 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:42:32.382 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.600812188000001, 2024-09-17T22:42:32.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024177605000000001, 2024-09-17T22:42:32.383 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.383 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.383 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.383 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202984 2024-09-17T22:42:32.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.383 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:42:32.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:42:32.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726150+0000", 2024-09-17T22:42:32.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T22:42:32.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726152+0000", 2024-09-17T22:42:32.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:42:32.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726162+0000", 2024-09-17T22:42:32.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7839999999999998e-06 2024-09-17T22:42:32.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726167+0000", 2024-09-17T22:42:32.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5690000000000004e-06 2024-09-17T22:42:32.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750184+0000", 2024-09-17T22:42:32.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024016909999999999 2024-09-17T22:42:32.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750255+0000", 2024-09-17T22:42:32.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1137999999999996e-05 2024-09-17T22:42:32.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750327+0000", 2024-09-17T22:42:32.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1903000000000006e-05 2024-09-17T22:42:32.387 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.387 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.387 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.387 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202985 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.388 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:32.388 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.600786801, 2024-09-17T22:42:32.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024280500999999999, 2024-09-17T22:42:32.388 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.388 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.388 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.388 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.388 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202985 2024-09-17T22:42:32.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.388 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:32.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:32.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:32.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8100000000000003e-07 2024-09-17T22:42:32.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726182+0000", 2024-09-17T22:42:32.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:42:32.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726190+0000", 2024-09-17T22:42:32.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:42:32.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726195+0000", 2024-09-17T22:42:32.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2809999999999999e-06 2024-09-17T22:42:32.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750366+0000", 2024-09-17T22:42:32.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024171635 2024-09-17T22:42:32.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750410+0000", 2024-09-17T22:42:32.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3823999999999999e-05 2024-09-17T22:42:32.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750455+0000", 2024-09-17T22:42:32.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5039e-05 2024-09-17T22:42:32.392 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.392 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.392 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.392 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202986 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.393 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:42:32.393 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.600759221000001, 2024-09-17T22:42:32.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024453077, 2024-09-17T22:42:32.393 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.393 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.393 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.393 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.393 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.393 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.393 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202986 2024-09-17T22:42:32.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.393 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:42:32.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:42:32.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726203+0000", 2024-09-17T22:42:32.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T22:42:32.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726205+0000", 2024-09-17T22:42:32.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0250000000000001e-06 2024-09-17T22:42:32.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726211+0000", 2024-09-17T22:42:32.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1340000000000001e-06 2024-09-17T22:42:32.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726222+0000", 2024-09-17T22:42:32.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1105e-05 2024-09-17T22:42:32.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750520+0000", 2024-09-17T22:42:32.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024297462999999998 2024-09-17T22:42:32.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750585+0000", 2024-09-17T22:42:32.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5940999999999998e-05 2024-09-17T22:42:32.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750655+0000", 2024-09-17T22:42:32.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0012000000000007e-05 2024-09-17T22:42:32.397 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.397 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.397 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.397 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202987 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.397 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:32.397 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.600732840999999, 2024-09-17T22:42:32.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024622569, 2024-09-17T22:42:32.397 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.397 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.397 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.398 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.398 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.398 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.398 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202987 2024-09-17T22:42:32.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.398 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:32.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:32.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:32.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:42:32.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726238+0000", 2024-09-17T22:42:32.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7430000000000008e-06 2024-09-17T22:42:32.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726245+0000", 2024-09-17T22:42:32.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6640000000000001e-06 2024-09-17T22:42:32.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726249+0000", 2024-09-17T22:42:32.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9119999999999997e-06 2024-09-17T22:42:32.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750694+0000", 2024-09-17T22:42:32.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024444105000000001 2024-09-17T22:42:32.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750767+0000", 2024-09-17T22:42:32.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3577000000000004e-05 2024-09-17T22:42:32.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750851+0000", 2024-09-17T22:42:32.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4189000000000003e-05 2024-09-17T22:42:32.401 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.401 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.402 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.402 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202988 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.402 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:32.402 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.600705979000001, 2024-09-17T22:42:32.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024808883, 2024-09-17T22:42:32.402 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.402 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.402 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.402 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.403 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.403 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.403 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202988 2024-09-17T22:42:32.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.403 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:32.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:32.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:32.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5499999999999999e-07 2024-09-17T22:42:32.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726258+0000", 2024-09-17T22:42:32.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8020000000000001e-06 2024-09-17T22:42:32.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726272+0000", 2024-09-17T22:42:32.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3828e-05 2024-09-17T22:42:32.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726277+0000", 2024-09-17T22:42:32.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2379999999999997e-06 2024-09-17T22:42:32.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750895+0000", 2024-09-17T22:42:32.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024618543 2024-09-17T22:42:32.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750978+0000", 2024-09-17T22:42:32.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3030999999999998e-05 2024-09-17T22:42:32.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751065+0000", 2024-09-17T22:42:32.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6086000000000002e-05 2024-09-17T22:42:32.407 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.407 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.407 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.408 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202989 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.408 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:42:32.408 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.600677105999999, 2024-09-17T22:42:32.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024976788, 2024-09-17T22:42:32.408 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.408 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.408 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.408 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.408 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.409 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.409 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202989 2024-09-17T22:42:32.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.409 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:42:32.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:42:32.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726285+0000", 2024-09-17T22:42:32.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T22:42:32.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726287+0000", 2024-09-17T22:42:32.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9860000000000001e-06 2024-09-17T22:42:32.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726294+0000", 2024-09-17T22:42:32.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2849999999999997e-06 2024-09-17T22:42:32.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726299+0000", 2024-09-17T22:42:32.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0150000000000003e-06 2024-09-17T22:42:32.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751109+0000", 2024-09-17T22:42:32.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024810175 2024-09-17T22:42:32.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751184+0000", 2024-09-17T22:42:32.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4640000000000004e-05 2024-09-17T22:42:32.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751261+0000", 2024-09-17T22:42:32.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7317000000000005e-05 2024-09-17T22:42:32.413 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.413 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.413 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.413 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202990 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.414 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:32.414 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.600656891, 2024-09-17T22:42:32.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025156767999999999, 2024-09-17T22:42:32.414 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.414 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.414 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.414 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.414 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202990 2024-09-17T22:42:32.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.415 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:32.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:32.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:32.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:42:32.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726307+0000", 2024-09-17T22:42:32.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7090000000000001e-06 2024-09-17T22:42:32.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726312+0000", 2024-09-17T22:42:32.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4909999999999996e-06 2024-09-17T22:42:32.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726317+0000", 2024-09-17T22:42:32.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3800000000000004e-06 2024-09-17T22:42:32.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751308+0000", 2024-09-17T22:42:32.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024991143 2024-09-17T22:42:32.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751385+0000", 2024-09-17T22:42:32.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6793999999999997e-05 2024-09-17T22:42:32.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751461+0000", 2024-09-17T22:42:32.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6891999999999995e-05 2024-09-17T22:42:32.419 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.419 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.419 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.419 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202991 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.419 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:32.419 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.600638857, 2024-09-17T22:42:32.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025289477000000001, 2024-09-17T22:42:32.419 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.419 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.419 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.420 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.420 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.420 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.420 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202991 2024-09-17T22:42:32.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.420 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:32.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:32.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:32.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.27e-07 2024-09-17T22:42:32.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726325+0000", 2024-09-17T22:42:32.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1459999999999999e-06 2024-09-17T22:42:32.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726333+0000", 2024-09-17T22:42:32.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1289999999999996e-06 2024-09-17T22:42:32.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726339+0000", 2024-09-17T22:42:32.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2530000000000004e-06 2024-09-17T22:42:32.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751499+0000", 2024-09-17T22:42:32.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025160109 2024-09-17T22:42:32.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751543+0000", 2024-09-17T22:42:32.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4317000000000003e-05 2024-09-17T22:42:32.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751612+0000", 2024-09-17T22:42:32.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9096000000000004e-05 2024-09-17T22:42:32.423 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.423 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.423 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.424 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202992 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.424 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:32.424 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.600616793, 2024-09-17T22:42:32.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025420585999999998, 2024-09-17T22:42:32.424 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.424 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.424 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.424 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.424 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.424 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.424 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202992 2024-09-17T22:42:32.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.425 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:32.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:32.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:32.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5199999999999998e-07 2024-09-17T22:42:32.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726347+0000", 2024-09-17T22:42:32.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9939999999999999e-06 2024-09-17T22:42:32.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726358+0000", 2024-09-17T22:42:32.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0421999999999999e-05 2024-09-17T22:42:32.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726363+0000", 2024-09-17T22:42:32.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:42:32.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751644+0000", 2024-09-17T22:42:32.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025280727999999999 2024-09-17T22:42:32.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751712+0000", 2024-09-17T22:42:32.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8207999999999995e-05 2024-09-17T22:42:32.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.428 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751765+0000", 2024-09-17T22:42:32.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3533999999999999e-05 2024-09-17T22:42:32.428 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.428 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.428 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.428 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202993 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:32.429 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:32.429 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 14.600592802, 2024-09-17T22:42:32.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025516259999999999, 2024-09-17T22:42:32.429 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:32.429 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:32.429 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:32.429 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:32.429 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:32.429 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:32.429 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202993 2024-09-17T22:42:32.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.430 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:32.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:32.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:32.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:32.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:32.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:32.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:32.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:32.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:42:32.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:32.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726371+0000", 2024-09-17T22:42:32.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T22:42:32.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:32.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726380+0000", 2024-09-17T22:42:32.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9109999999999999e-06 2024-09-17T22:42:32.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:32.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726385+0000", 2024-09-17T22:42:32.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7380000000000002e-06 2024-09-17T22:42:32.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:32.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751785+0000", 2024-09-17T22:42:32.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025400526 2024-09-17T22:42:32.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:32.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751824+0000", 2024-09-17T22:42:32.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8989999999999998e-05 2024-09-17T22:42:32.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:32.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:32.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:32.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751885+0000", 2024-09-17T22:42:32.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0937000000000003e-05 2024-09-17T22:42:32.434 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.434 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.434 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.434 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:32.434 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:32.434 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:32.435 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:32.436 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:32.437 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:42:32.437 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:32.438 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:32.438 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone35 from 0xbb94137 (overlap 0xcaa703) to 0xb6631c 2024-09-17T22:42:32.438 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:32.433+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:32.465 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-17T22:42:32.537 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:32.534+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:32.585 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:32.585 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:32.595 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:42:32.638 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:32.636+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:32.680 INFO:teuthology.orchestra.run.smithi115.stdout:5455 writesame 0x344a4a thru 0x34d9e8 data_size 0x1(0x8f9f bytes) 2024-09-17T22:42:32.701 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:32.701 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:32.704 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:32.704 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:32.704 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:32.704 INFO:teuthology.orchestra.run.smithi115.stdout:5460 write 0xc33883 thru 0xc35ad6 (0x2254 bytes) 2024-09-17T22:42:32.709 INFO:teuthology.orchestra.run.smithi115.stdout:5462 write 0x5cbc232 thru 0x5cc6b76 (0xa945 bytes) 2024-09-17T22:42:32.711 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:42:32.717 INFO:teuthology.orchestra.run.smithi115.stdout:5463 read 0x5707344 thru 0x5715d40 (0xe9fd bytes) 2024-09-17T22:42:32.717 INFO:teuthology.orchestra.run.smithi115.stdout:5465 write 0x5b56e73 thru 0x5b5e057 (0x71e5 bytes) 2024-09-17T22:42:32.718 INFO:teuthology.orchestra.run.smithi115.stdout:5466 read 0x6f87283 thru 0x6f967d1 (0xf54f bytes) 2024-09-17T22:42:32.721 INFO:teuthology.orchestra.run.smithi115.stdout:5467 trunc from 0xbb94137 to 0x9d584f7 2024-09-17T22:42:32.738 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:32.736+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:32.740 INFO:teuthology.orchestra.run.smithi115.stdout:5469 punch from 0x4cc4c0b to 0x4ccff93, (0xb388 bytes) 2024-09-17T22:42:32.740 INFO:teuthology.orchestra.run.smithi115.stdout:5470 write 0x5e64e5f thru 0x5e6d840 (0x89e2 bytes) 2024-09-17T22:42:32.756 INFO:teuthology.orchestra.run.smithi115.stdout:5471 trunc from 0x9d584f7 to 0x8aff78 2024-09-17T22:42:32.801 INFO:teuthology.orchestra.run.smithi115.stdout:5472 trunc from 0x8aff78 to 0x45314dc 2024-09-17T22:42:32.808 INFO:teuthology.orchestra.run.smithi115.stdout:5473 punch from 0x2f2058 to 0x2f542d, (0x33d5 bytes) 2024-09-17T22:42:32.820 INFO:teuthology.orchestra.run.smithi115.stdout:5475 writesame 0x98125ae thru 0x981abca data_size 0x13(0x861d bytes) 2024-09-17T22:42:32.821 INFO:teuthology.orchestra.run.smithi115.stdout:5476 punch from 0x46c58a7 to 0x46c9a2d, (0x4186 bytes) 2024-09-17T22:42:32.821 INFO:teuthology.orchestra.run.smithi115.stdout:5477 read 0x811a969 thru 0x811af1c (0x5b4 bytes) 2024-09-17T22:42:32.821 INFO:teuthology.orchestra.run.smithi115.stdout:5478 read 0x1440265 thru 0x1448e9b (0x8c37 bytes) 2024-09-17T22:42:32.821 INFO:teuthology.orchestra.run.smithi115.stdout:5479 write 0x96b0cf1 thru 0x96b4043 (0x3353 bytes) 2024-09-17T22:42:32.837 INFO:teuthology.orchestra.run.smithi115.stdout:5480 writesame 0xed7d487 thru 0xed8854c data_size 0x22(0xb0c6 bytes) 2024-09-17T22:42:32.837 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:32.837 INFO:teuthology.orchestra.run.smithi115.stdout:5482 writesame 0xc73d6d7 thru 0xc7470ee data_size 0x4(0x9a18 bytes) 2024-09-17T22:42:32.838 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:32.838 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:32.839 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:32.837+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:32.845 INFO:teuthology.orchestra.run.smithi115.stdout:5485 write 0x862597d thru 0x8633ae1 (0xe165 bytes) 2024-09-17T22:42:32.848 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:42:32.854 INFO:teuthology.orchestra.run.smithi115.stdout:5486 read 0x88bc489 thru 0x88c1f67 (0x5adf bytes) 2024-09-17T22:42:32.864 INFO:teuthology.orchestra.run.smithi115.stdout:5487 trunc from 0xed8854d to 0x6ce6a0a 2024-09-17T22:42:32.876 INFO:teuthology.orchestra.run.smithi115.stdout:5488 writesame 0x11b0853 thru 0x11bde4a data_size 0x8(0xd5f8 bytes) 2024-09-17T22:42:32.890 INFO:teuthology.orchestra.run.smithi115.stdout:5489 trunc from 0x6ce6a0a to 0xcc2ae2c 2024-09-17T22:42:32.897 INFO:teuthology.orchestra.run.smithi115.stdout:5490 write 0x43f5a7 thru 0x448636 (0x9090 bytes) 2024-09-17T22:42:32.916 INFO:teuthology.orchestra.run.smithi115.stdout:5492 trunc from 0xcc2ae2c to 0x8581d3e 2024-09-17T22:42:32.921 INFO:teuthology.orchestra.run.smithi115.stdout:5493 writesame 0x2acb655 thru 0x2accbfc data_size 0x2(0x15a8 bytes) 2024-09-17T22:42:32.925 INFO:teuthology.orchestra.run.smithi115.stdout:5498 read 0x2017a7d thru 0x20181f9 (0x77d bytes) 2024-09-17T22:42:32.925 INFO:teuthology.orchestra.run.smithi115.stdout:5499 read 0x474f6dc thru 0x4757f51 (0x8876 bytes) 2024-09-17T22:42:32.925 INFO:teuthology.orchestra.run.smithi115.stdout:5500 read 0x7d49823 thru 0x7d4b6c2 (0x1ea0 bytes) 2024-09-17T22:42:32.931 INFO:teuthology.orchestra.run.smithi115.stdout:5501 write 0x883e77c thru 0x88455fd (0x6e82 bytes) 2024-09-17T22:42:32.939 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:32.937+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:32.945 INFO:teuthology.orchestra.run.smithi115.stdout:5502 write 0xb22c36d thru 0xb22da09 (0x169d bytes) 2024-09-17T22:42:32.948 INFO:teuthology.orchestra.run.smithi115.stdout:5503 clone 37 order 24 su 131072 sc 2 2024-09-17T22:42:32.972 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:32.972 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:32.981 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:42:33.064 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:33.062+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:33.123 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:33.123 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:33.132 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:42:33.173 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:33.170+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:33.242 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:33.240+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:33.275 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:33.275 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:33.284 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:42:33.312 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:42:33.343 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:33.341+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:33.392 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:33.392 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:33.402 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:42:33.444 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:33.441+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:33.545 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:33.542+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:33.554 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:33.554 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:33.566 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:42:33.645 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:33.643+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:33.669 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:33.669 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:33.678 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:42:33.746 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:33.744+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:33.783 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:33.783 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:33.791 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:42:33.847 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:33.844+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:33.895 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:33.895 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:33.905 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:42:33.948 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:33.945+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:34.047 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:34.045+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:34.048 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:34.048 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:34.056 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:42:34.148 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:34.145+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:34.200 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:34.200 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:34.209 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:42:34.249 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:34.247+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:34.306 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is set 2024-09-17T22:42:34.359 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:34.359 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:34.366 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:34.363+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:34.368 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:42:34.369 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone36 from 0xb22da0a (overlap 0x8aff78) to 0x457fbc 2024-09-17T22:42:34.438 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:42:34.470 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-17T22:42:34.470 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:34.468+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:34.484 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:34.484 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:34.485 INFO:teuthology.orchestra.run.smithi115.stdout:5504 writesame 0x7b41599 thru 0x7b4ac76 data_size 0x7b(0x96de bytes) 2024-09-17T22:42:34.492 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:34.492 INFO:teuthology.orchestra.run.smithi115.stdout:5506 punch from 0x8bc2d47 to 0x8bd2577, (0xf830 bytes) 2024-09-17T22:42:34.493 INFO:teuthology.orchestra.run.smithi115.stdout:5507 punch from 0x22bcde5 to 0x22bfbf6, (0x2e11 bytes) 2024-09-17T22:42:34.494 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:42:34.496 INFO:teuthology.orchestra.run.smithi115.stdout:5508 trunc from 0xb22da0a to 0x7c01d16 2024-09-17T22:42:34.507 INFO:teuthology.orchestra.run.smithi115.stdout:5509 read 0x38a3d25 thru 0x38a94aa (0x5786 bytes) 2024-09-17T22:42:34.508 INFO:teuthology.orchestra.run.smithi115.stdout:5510 read 0x121164e thru 0x121bea2 (0xa855 bytes) 2024-09-17T22:42:34.508 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:34.508 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:34.517 INFO:teuthology.orchestra.run.smithi115.stdout:5515 write 0xb7228d8 thru 0xb727062 (0x478b bytes) 2024-09-17T22:42:34.533 INFO:teuthology.orchestra.run.smithi115.stdout:5516 trunc from 0xb727063 to 0x17e6535 2024-09-17T22:42:34.551 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:34.549+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:34.561 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:34.570 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:34.570 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:34.570 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:34.578 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:34.579 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:34.590 INFO:teuthology.orchestra.run.smithi115.stdout:5520 write 0xe17124e thru 0xe180090 (0xee43 bytes) 2024-09-17T22:42:34.598 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:34.598 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:34.600 INFO:teuthology.orchestra.run.smithi115.stdout:5521 read 0x7930497 thru 0x793392d (0x3497 bytes) 2024-09-17T22:42:34.600 INFO:teuthology.orchestra.run.smithi115.stdout:5522 punch from 0x10ab8fb to 0x10aebec, (0x32f1 bytes) 2024-09-17T22:42:34.600 INFO:teuthology.orchestra.run.smithi115.stdout:5523 read 0xa7be249 thru 0xa7c3092 (0x4e4a bytes) 2024-09-17T22:42:34.600 INFO:teuthology.orchestra.run.smithi115.stdout:5526 writesame 0xa60d5d1 thru 0xa616fb7 data_size 0x3(0x99e7 bytes) 2024-09-17T22:42:34.607 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:34.607 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:34.609 INFO:teuthology.orchestra.run.smithi115.stdout:5528 read 0xa7d5cf0 thru 0xa7da676 (0x4987 bytes) 2024-09-17T22:42:34.609 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:34.617 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:42:34.617 INFO:teuthology.orchestra.run.smithi115.stdout:5532 trunc from 0xe180091 to 0x909b8d1 2024-09-17T22:42:34.618 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:34.618 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:34.628 INFO:teuthology.orchestra.run.smithi115.stdout:5533 read 0x8a9f130 thru 0x8aa4fa7 (0x5e78 bytes) 2024-09-17T22:42:34.628 INFO:teuthology.orchestra.run.smithi115.stdout:5535 write 0x7a7c783 thru 0x7a802b2 (0x3b30 bytes) 2024-09-17T22:42:34.633 INFO:teuthology.orchestra.run.smithi115.stdout:5536 write 0x63741e3 thru 0x63816a1 (0xd4bf bytes) 2024-09-17T22:42:34.634 INFO:teuthology.orchestra.run.smithi115.stdout:5538 read 0x728b10 thru 0x72ad6c (0x225d bytes) 2024-09-17T22:42:34.635 INFO:teuthology.orchestra.run.smithi115.stdout:5539 writesame 0x5054008 thru 0x5059d0d data_size 0x51(0x5d06 bytes) 2024-09-17T22:42:34.635 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:34.636 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:34.639 INFO:teuthology.orchestra.run.smithi115.stdout:5540 writesame 0x6964451 thru 0x697432a data_size 0x1(0xfeda bytes) 2024-09-17T22:42:34.653 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:34.653 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:34.661 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:34.659+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:34.662 INFO:teuthology.orchestra.run.smithi115.stdout:5542 punch from 0x48183ab to 0x481f478, (0x70cd bytes) 2024-09-17T22:42:34.662 INFO:teuthology.orchestra.run.smithi115.stdout:5543 read 0x3cba2fc thru 0x3cc2c07 (0x890c bytes) 2024-09-17T22:42:34.662 INFO:teuthology.orchestra.run.smithi115.stdout:5544 write 0x8788bb0 thru 0x878b6c4 (0x2b15 bytes) 2024-09-17T22:42:34.665 INFO:teuthology.orchestra.run.smithi115.stdout:5545 write 0x1ccefe6 thru 0x1cd72e9 (0x8304 bytes) 2024-09-17T22:42:34.674 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:34.674 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:34.677 INFO:teuthology.orchestra.run.smithi115.stdout:5546 writesame 0xd0f21f8 thru 0xd0f91e0 data_size 0x1(0x6fe9 bytes) 2024-09-17T22:42:34.677 INFO:teuthology.orchestra.run.smithi115.stdout:5547 read 0x2cc537 thru 0x2cd926 (0x13f0 bytes) 2024-09-17T22:42:34.678 INFO:teuthology.orchestra.run.smithi115.stdout:5548 writesame 0xa1c075 thru 0xa21d88 data_size 0x2e(0x5d14 bytes) 2024-09-17T22:42:34.679 INFO:teuthology.orchestra.run.smithi115.stdout:5551 write 0x2076602 thru 0x2079075 (0x2a74 bytes) 2024-09-17T22:42:34.683 INFO:teuthology.orchestra.run.smithi115.stdout:5553 writesame 0x6118590 thru 0x6123f5e data_size 0x1(0xb9cf bytes) 2024-09-17T22:42:34.691 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:34.692 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:34.695 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:34.696 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:34.699 INFO:teuthology.orchestra.run.smithi115.stdout:5554 writesame 0x7ed7d8c thru 0x7edf864 data_size 0xb(0x7ad9 bytes) 2024-09-17T22:42:34.701 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:34.701 INFO:teuthology.orchestra.run.smithi115.stdout:5556 read 0x4c0ba9e thru 0x4c145a7 (0x8b0a bytes) 2024-09-17T22:42:34.701 INFO:teuthology.orchestra.run.smithi115.stdout:5557 punch from 0x3347df0 to 0x3357738, (0xf948 bytes) 2024-09-17T22:42:34.708 INFO:teuthology.orchestra.run.smithi115.stdout:5558 trunc from 0xd0f91e1 to 0x75db004 2024-09-17T22:42:34.727 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:34.727 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:34.727 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:34.727 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:34.727 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:34.736 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:42:34.737 INFO:teuthology.orchestra.run.smithi115.stdout:5559 writesame 0xead1674 thru 0xeade974 data_size 0x1(0xd301 bytes) 2024-09-17T22:42:34.740 INFO:teuthology.orchestra.run.smithi115.stdout:5560 trunc from 0xeade975 to 0xbb1e51a 2024-09-17T22:42:34.745 INFO:teuthology.orchestra.run.smithi115.stdout:5561 writesame 0x72f7823 thru 0x7307742 data_size 0x10(0xff20 bytes) 2024-09-17T22:42:34.748 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:34.748 INFO:teuthology.orchestra.run.smithi115.stdout:5563 writesame 0x922a3bf thru 0x9233653 data_size 0x4f(0x9295 bytes) 2024-09-17T22:42:34.751 INFO:teuthology.orchestra.run.smithi115.stdout:5564 punch from 0x79d52db to 0x79dc2d1, (0x6ff6 bytes) 2024-09-17T22:42:34.751 INFO:teuthology.orchestra.run.smithi115.stdout:5565 punch from 0x6333f30 to 0x633cf57, (0x9027 bytes) 2024-09-17T22:42:34.751 INFO:teuthology.orchestra.run.smithi115.stdout:5566 writesame 0x8efdeff thru 0x8f0556c data_size 0x5d(0x766e bytes) 2024-09-17T22:42:34.760 INFO:teuthology.orchestra.run.smithi115.stdout:5568 trunc from 0xbb1e51a to 0x6ba14f0 2024-09-17T22:42:34.777 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:34.775+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:34.780 INFO:teuthology.orchestra.run.smithi115.stdout:5569 writesame 0x83b7377 thru 0x83c481d data_size 0x4d(0xd4a7 bytes) 2024-09-17T22:42:34.784 INFO:teuthology.orchestra.run.smithi115.stdout:5571 trunc from 0x83c481e to 0x4f2195f 2024-09-17T22:42:34.796 INFO:teuthology.orchestra.run.smithi115.stdout:5572 punch from 0x3e0c945 to 0x3e0e06d, (0x1728 bytes) 2024-09-17T22:42:34.796 INFO:teuthology.orchestra.run.smithi115.stdout:5573 read 0x9a4071 thru 0x9a432f (0x2bf bytes) 2024-09-17T22:42:34.797 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:34.797 INFO:teuthology.orchestra.run.smithi115.stdout:5577 punch from 0x410578b to 0x41104ab, (0xad20 bytes) 2024-09-17T22:42:34.797 INFO:teuthology.orchestra.run.smithi115.stdout:5578 read 0x31cc170 thru 0x31d5e47 (0x9cd8 bytes) 2024-09-17T22:42:34.797 INFO:teuthology.orchestra.run.smithi115.stdout:5579 write 0x30fa5fb thru 0x31029ab (0x83b1 bytes) 2024-09-17T22:42:34.799 INFO:teuthology.orchestra.run.smithi115.stdout:5581 read 0x2f5ea6f thru 0x2f68828 (0x9dba bytes) 2024-09-17T22:42:34.800 INFO:teuthology.orchestra.run.smithi115.stdout:5584 punch from 0x692634 to 0x692be4, (0x5b0 bytes) 2024-09-17T22:42:34.805 INFO:teuthology.orchestra.run.smithi115.stdout:5585 writesame 0x4feb371 thru 0x4ff430f data_size 0x1(0x8f9f bytes) 2024-09-17T22:42:34.811 INFO:teuthology.orchestra.run.smithi115.stdout:5586 write 0x5950c7f thru 0x595724b (0x65cd bytes) 2024-09-17T22:42:34.827 INFO:teuthology.orchestra.run.smithi115.stdout:5587 writesame 0xd75418a thru 0xd75d11a data_size 0x1(0x8f91 bytes) 2024-09-17T22:42:34.829 INFO:teuthology.orchestra.run.smithi115.stdout:5590 trunc from 0xd75d11b to 0xc6e0efb 2024-09-17T22:42:34.843 INFO:teuthology.orchestra.run.smithi115.stdout:5591 write 0xe2ab8ed thru 0xe2b0b17 (0x522b bytes) 2024-09-17T22:42:34.845 INFO:teuthology.orchestra.run.smithi115.stdout:5592 read 0x2682b99 thru 0x268d01f (0xa487 bytes) 2024-09-17T22:42:34.846 INFO:teuthology.orchestra.run.smithi115.stdout:5594 punch from 0xdfc2194 to 0xdfc7482, (0x52ee bytes) 2024-09-17T22:42:34.846 INFO:teuthology.orchestra.run.smithi115.stdout:5595 write 0xc6442a6 thru 0xc644577 (0x2d2 bytes) 2024-09-17T22:42:34.849 INFO:teuthology.orchestra.run.smithi115.stdout:5596 writesame 0x58ec5ad thru 0x58f4f56 data_size 0x2(0x89aa bytes) 2024-09-17T22:42:34.853 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:34.850+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:34.857 INFO:teuthology.orchestra.run.smithi115.stdout:5597 writesame 0xdf72f04 thru 0xdf8144e data_size 0x1(0xe54b bytes) 2024-09-17T22:42:34.883 INFO:teuthology.orchestra.run.smithi115.stdout:5598 punch from 0x42b3dda to 0x42b5e3c, (0x2062 bytes) 2024-09-17T22:42:34.884 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:34.884 INFO:teuthology.orchestra.run.smithi115.stdout:5601 writesame 0x502ae32 thru 0x5035330 data_size 0x1(0xa4ff bytes) 2024-09-17T22:42:34.887 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:34.887 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:34.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:34.887 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:34.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.887 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188693 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5881856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:34.887 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:34.887 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.056744169000002, 2024-09-17T22:42:34.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028193223, 2024-09-17T22:42:34.887 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.888 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.888 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.888 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188693 2024-09-17T22:42:34.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.888 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:34.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:34.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:34.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:42:34.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827090+0000", 2024-09-17T22:42:34.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2170000000000001e-06 2024-09-17T22:42:34.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827098+0000", 2024-09-17T22:42:34.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1510000000000004e-06 2024-09-17T22:42:34.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827103+0000", 2024-09-17T22:42:34.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0839999999999999e-06 2024-09-17T22:42:34.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855158+0000", 2024-09-17T22:42:34.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028055217 2024-09-17T22:42:34.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855201+0000", 2024-09-17T22:42:34.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3272000000000002e-05 2024-09-17T22:42:34.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855280+0000", 2024-09-17T22:42:34.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8845999999999997e-05 2024-09-17T22:42:34.891 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.891 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.891 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.891 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188694 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5971968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:34.891 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:34.892 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.056720282000001, 2024-09-17T22:42:34.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028312278, 2024-09-17T22:42:34.892 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.892 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.892 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.892 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188694 2024-09-17T22:42:34.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.892 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:34.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:34.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:34.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:42:34.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827114+0000", 2024-09-17T22:42:34.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3029999999999998e-06 2024-09-17T22:42:34.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827121+0000", 2024-09-17T22:42:34.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3699999999999997e-06 2024-09-17T22:42:34.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827126+0000", 2024-09-17T22:42:34.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7350000000000004e-06 2024-09-17T22:42:34.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855307+0000", 2024-09-17T22:42:34.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028180851999999999 2024-09-17T22:42:34.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855346+0000", 2024-09-17T22:42:34.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9274999999999999e-05 2024-09-17T22:42:34.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855423+0000", 2024-09-17T22:42:34.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7294999999999998e-05 2024-09-17T22:42:34.895 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.895 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.895 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.896 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188695 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:34.896 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:34.896 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.056699504000001, 2024-09-17T22:42:34.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028435876999999998, 2024-09-17T22:42:34.896 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.896 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.896 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.896 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188695 2024-09-17T22:42:34.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.896 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:34.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:34.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:34.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:42:34.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827138+0000", 2024-09-17T22:42:34.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2580000000000001e-06 2024-09-17T22:42:34.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827145+0000", 2024-09-17T22:42:34.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0129999999999996e-06 2024-09-17T22:42:34.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827150+0000", 2024-09-17T22:42:34.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3749999999999996e-06 2024-09-17T22:42:34.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855447+0000", 2024-09-17T22:42:34.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028297224999999999 2024-09-17T22:42:34.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855487+0000", 2024-09-17T22:42:34.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0454000000000003e-05 2024-09-17T22:42:34.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855567+0000", 2024-09-17T22:42:34.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0149999999999997e-05 2024-09-17T22:42:34.900 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.900 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.900 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.900 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188696 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:34.900 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:34.900 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.056675720000001, 2024-09-17T22:42:34.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028543570000000001, 2024-09-17T22:42:34.900 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.900 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.901 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.901 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188696 2024-09-17T22:42:34.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.901 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:34.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:34.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827156+0000", 2024-09-17T22:42:34.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T22:42:34.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827158+0000", 2024-09-17T22:42:34.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.875e-06 2024-09-17T22:42:34.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827164+0000", 2024-09-17T22:42:34.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7950000000000002e-06 2024-09-17T22:42:34.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827169+0000", 2024-09-17T22:42:34.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2880000000000003e-06 2024-09-17T22:42:34.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855582+0000", 2024-09-17T22:42:34.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028412929 2024-09-17T22:42:34.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855624+0000", 2024-09-17T22:42:34.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2484000000000002e-05 2024-09-17T22:42:34.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855699+0000", 2024-09-17T22:42:34.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4869000000000004e-05 2024-09-17T22:42:34.904 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.904 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.904 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.905 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188697 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6242304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:34.905 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:34.905 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.056644191, 2024-09-17T22:42:34.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028658598, 2024-09-17T22:42:34.905 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.905 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.905 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.905 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188697 2024-09-17T22:42:34.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.906 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:34.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:34.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:34.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8800000000000003e-07 2024-09-17T22:42:34.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827190+0000", 2024-09-17T22:42:34.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.351e-06 2024-09-17T22:42:34.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827198+0000", 2024-09-17T22:42:34.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.909e-06 2024-09-17T22:42:34.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827203+0000", 2024-09-17T22:42:34.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4020000000000001e-06 2024-09-17T22:42:34.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855720+0000", 2024-09-17T22:42:34.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028516909999999999 2024-09-17T22:42:34.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855762+0000", 2024-09-17T22:42:34.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2184999999999997e-05 2024-09-17T22:42:34.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855845+0000", 2024-09-17T22:42:34.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3353e-05 2024-09-17T22:42:34.909 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.909 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.909 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.909 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202883 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2342912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:34.909 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:42:34.909 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.159007809999999, 2024-09-17T22:42:34.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028098636, 2024-09-17T22:42:34.910 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.910 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.910 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.910 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202883 2024-09-17T22:42:34.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.910 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:42:34.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:42:34.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724824+0000", 2024-09-17T22:42:34.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:42:34.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724826+0000", 2024-09-17T22:42:34.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.086e-06 2024-09-17T22:42:34.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724832+0000", 2024-09-17T22:42:34.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9100000000000002e-06 2024-09-17T22:42:34.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724837+0000", 2024-09-17T22:42:34.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9439999999999996e-06 2024-09-17T22:42:34.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752790+0000", 2024-09-17T22:42:34.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027953657999999999 2024-09-17T22:42:34.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752848+0000", 2024-09-17T22:42:34.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7677999999999999e-05 2024-09-17T22:42:34.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752922+0000", 2024-09-17T22:42:34.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3957999999999999e-05 2024-09-17T22:42:34.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.913 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.914 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202884 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2473984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:34.914 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:42:34.914 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.158979585000001, 2024-09-17T22:42:34.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028206208999999999, 2024-09-17T22:42:34.914 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.914 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.914 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.914 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202884 2024-09-17T22:42:34.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.915 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:42:34.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:42:34.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724852+0000", 2024-09-17T22:42:34.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2599999999999998e-07 2024-09-17T22:42:34.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724854+0000", 2024-09-17T22:42:34.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.26e-06 2024-09-17T22:42:34.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724871+0000", 2024-09-17T22:42:34.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6589999999999999e-05 2024-09-17T22:42:34.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724877+0000", 2024-09-17T22:42:34.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.054e-06 2024-09-17T22:42:34.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752951+0000", 2024-09-17T22:42:34.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028073747 2024-09-17T22:42:34.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752993+0000", 2024-09-17T22:42:34.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2082000000000001e-05 2024-09-17T22:42:34.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753058+0000", 2024-09-17T22:42:34.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5049999999999996e-05 2024-09-17T22:42:34.918 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.918 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.918 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.918 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202885 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:34.919 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:34.919 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.158946281999999, 2024-09-17T22:42:34.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028333028, 2024-09-17T22:42:34.919 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.919 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.919 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.919 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.919 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.919 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.919 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202885 2024-09-17T22:42:34.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.919 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:34.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:34.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:34.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3300000000000002e-07 2024-09-17T22:42:34.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724892+0000", 2024-09-17T22:42:34.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2779999999999999e-06 2024-09-17T22:42:34.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724899+0000", 2024-09-17T22:42:34.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9110000000000004e-06 2024-09-17T22:42:34.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724904+0000", 2024-09-17T22:42:34.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2360000000000004e-06 2024-09-17T22:42:34.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753094+0000", 2024-09-17T22:42:34.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02819033 2024-09-17T22:42:34.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753142+0000", 2024-09-17T22:42:34.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.829e-05 2024-09-17T22:42:34.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753218+0000", 2024-09-17T22:42:34.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5450000000000004e-05 2024-09-17T22:42:34.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.922 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.923 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202886 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:34.923 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:34.923 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.158921083999999, 2024-09-17T22:42:34.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028442512, 2024-09-17T22:42:34.923 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.923 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.923 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.923 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202886 2024-09-17T22:42:34.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.924 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:34.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:34.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:34.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:42:34.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724912+0000", 2024-09-17T22:42:34.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.967e-06 2024-09-17T22:42:34.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724918+0000", 2024-09-17T22:42:34.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8069999999999996e-06 2024-09-17T22:42:34.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724923+0000", 2024-09-17T22:42:34.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.386e-06 2024-09-17T22:42:34.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753244+0000", 2024-09-17T22:42:34.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028321567999999998 2024-09-17T22:42:34.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753289+0000", 2024-09-17T22:42:34.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4620000000000003e-05 2024-09-17T22:42:34.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753353+0000", 2024-09-17T22:42:34.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3795999999999997e-05 2024-09-17T22:42:34.927 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.927 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.927 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.927 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202887 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:34.927 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:34.927 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.158903170999999, 2024-09-17T22:42:34.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028565838999999999, 2024-09-17T22:42:34.927 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.927 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.927 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.927 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.927 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.927 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.928 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202887 2024-09-17T22:42:34.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.928 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:34.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:34.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:34.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:42:34.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724930+0000", 2024-09-17T22:42:34.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8980000000000001e-06 2024-09-17T22:42:34.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724936+0000", 2024-09-17T22:42:34.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7849999999999997e-06 2024-09-17T22:42:34.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724942+0000", 2024-09-17T22:42:34.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9649999999999996e-06 2024-09-17T22:42:34.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753380+0000", 2024-09-17T22:42:34.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028438049 2024-09-17T22:42:34.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753425+0000", 2024-09-17T22:42:34.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5163000000000001e-05 2024-09-17T22:42:34.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753494+0000", 2024-09-17T22:42:34.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8588000000000001e-05 2024-09-17T22:42:34.931 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.931 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.931 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.931 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202888 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:34.931 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:42:34.931 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.158883672999998, 2024-09-17T22:42:34.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028703314000000001, 2024-09-17T22:42:34.931 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.931 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.931 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.932 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.932 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202888 2024-09-17T22:42:34.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.932 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:42:34.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:42:34.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724948+0000", 2024-09-17T22:42:34.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:42:34.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724950+0000", 2024-09-17T22:42:34.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8029999999999999e-06 2024-09-17T22:42:34.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724955+0000", 2024-09-17T22:42:34.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8760000000000002e-06 2024-09-17T22:42:34.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724960+0000", 2024-09-17T22:42:34.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5979999999999999e-06 2024-09-17T22:42:34.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753527+0000", 2024-09-17T22:42:34.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028566864000000001 2024-09-17T22:42:34.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753568+0000", 2024-09-17T22:42:34.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0775000000000002e-05 2024-09-17T22:42:34.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753651+0000", 2024-09-17T22:42:34.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3039000000000002e-05 2024-09-17T22:42:34.935 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.935 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.935 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.935 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202889 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:34.935 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:34.935 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.158866101000001, 2024-09-17T22:42:34.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028820964000000001, 2024-09-17T22:42:34.936 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.936 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.936 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.936 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.936 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202889 2024-09-17T22:42:34.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.936 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:34.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:34.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:34.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2599999999999998e-07 2024-09-17T22:42:34.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724967+0000", 2024-09-17T22:42:34.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T22:42:34.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724973+0000", 2024-09-17T22:42:34.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6060000000000002e-06 2024-09-17T22:42:34.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724978+0000", 2024-09-17T22:42:34.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8230000000000002e-06 2024-09-17T22:42:34.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753687+0000", 2024-09-17T22:42:34.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028709495000000002 2024-09-17T22:42:34.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753727+0000", 2024-09-17T22:42:34.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9953999999999997e-05 2024-09-17T22:42:34.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753786+0000", 2024-09-17T22:42:34.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8941999999999999e-05 2024-09-17T22:42:34.939 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.939 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.939 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.940 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202890 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:34.940 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:34.940 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.158848533, 2024-09-17T22:42:34.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028933838, 2024-09-17T22:42:34.940 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.940 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.940 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.940 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.940 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202890 2024-09-17T22:42:34.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.940 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:34.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:34.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:34.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5999999999999999e-07 2024-09-17T22:42:34.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724985+0000", 2024-09-17T22:42:34.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8169999999999999e-06 2024-09-17T22:42:34.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724991+0000", 2024-09-17T22:42:34.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:42:34.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.942 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.942 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724995+0000", 2024-09-17T22:42:34.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4530000000000004e-06 2024-09-17T22:42:34.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753811+0000", 2024-09-17T22:42:34.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028815584000000002 2024-09-17T22:42:34.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753854+0000", 2024-09-17T22:42:34.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3356999999999998e-05 2024-09-17T22:42:34.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753916+0000", 2024-09-17T22:42:34.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2488000000000002e-05 2024-09-17T22:42:34.943 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.944 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.944 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.944 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202891 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:34.944 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:34.944 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.158831075999998, 2024-09-17T22:42:34.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029067554999999998, 2024-09-17T22:42:34.944 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.944 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.944 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.944 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.944 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.944 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.944 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202891 2024-09-17T22:42:34.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.945 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:34.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:34.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:34.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:42:34.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725002+0000", 2024-09-17T22:42:34.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:42:34.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725008+0000", 2024-09-17T22:42:34.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6629999999999998e-06 2024-09-17T22:42:34.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725016+0000", 2024-09-17T22:42:34.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8269999999999998e-06 2024-09-17T22:42:34.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753949+0000", 2024-09-17T22:42:34.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028932625999999999 2024-09-17T22:42:34.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.947 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753994+0000", 2024-09-17T22:42:34.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5717000000000003e-05 2024-09-17T22:42:34.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.947 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754068+0000", 2024-09-17T22:42:34.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3312000000000006e-05 2024-09-17T22:42:34.948 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.948 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.948 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.948 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202892 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:34.948 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:34.948 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.158788369, 2024-09-17T22:42:34.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029159939999999999, 2024-09-17T22:42:34.948 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.948 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.948 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.949 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.949 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.949 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.949 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202892 2024-09-17T22:42:34.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.949 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:34.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:34.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:34.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7999999999999995e-07 2024-09-17T22:42:34.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725045+0000", 2024-09-17T22:42:34.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-17T22:42:34.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725052+0000", 2024-09-17T22:42:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3629999999999999e-06 2024-09-17T22:42:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725056+0000", 2024-09-17T22:42:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3250000000000001e-06 2024-09-17T22:42:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754094+0000", 2024-09-17T22:42:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029038270000000001 2024-09-17T22:42:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754134+0000", 2024-09-17T22:42:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9722999999999999e-05 2024-09-17T22:42:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754203+0000", 2024-09-17T22:42:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8524999999999999e-05 2024-09-17T22:42:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202893 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:42:34.952 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.158768792, 2024-09-17T22:42:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029341651, 2024-09-17T22:42:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202893 2024-09-17T22:42:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:42:34.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:42:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725063+0000", 2024-09-17T22:42:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6699999999999999e-07 2024-09-17T22:42:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725065+0000", 2024-09-17T22:42:34.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2120000000000002e-06 2024-09-17T22:42:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725071+0000", 2024-09-17T22:42:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5169999999999996e-06 2024-09-17T22:42:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725076+0000", 2024-09-17T22:42:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.78e-06 2024-09-17T22:42:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754233+0000", 2024-09-17T22:42:34.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029156277000000001 2024-09-17T22:42:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754298+0000", 2024-09-17T22:42:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5320000000000005e-05 2024-09-17T22:42:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754404+0000", 2024-09-17T22:42:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106078 2024-09-17T22:42:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202894 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:42:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.158748913, 2024-09-17T22:42:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029456572, 2024-09-17T22:42:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202894 2024-09-17T22:42:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.957 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:42:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:42:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725083+0000", 2024-09-17T22:42:34.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5900000000000002e-07 2024-09-17T22:42:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725085+0000", 2024-09-17T22:42:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.979e-06 2024-09-17T22:42:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725091+0000", 2024-09-17T22:42:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1480000000000003e-06 2024-09-17T22:42:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725095+0000", 2024-09-17T22:42:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1339999999999997e-06 2024-09-17T22:42:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754429+0000", 2024-09-17T22:42:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029333758000000001 2024-09-17T22:42:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754471+0000", 2024-09-17T22:42:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2274999999999998e-05 2024-09-17T22:42:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754539+0000", 2024-09-17T22:42:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7818999999999996e-05 2024-09-17T22:42:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202895 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.158730516999999, 2024-09-17T22:42:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029635578999999999, 2024-09-17T22:42:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.961 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202895 2024-09-17T22:42:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.08e-07 2024-09-17T22:42:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725107+0000", 2024-09-17T22:42:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9020000000000004e-06 2024-09-17T22:42:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725113+0000", 2024-09-17T22:42:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7400000000000001e-06 2024-09-17T22:42:34.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725117+0000", 2024-09-17T22:42:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:42:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754567+0000", 2024-09-17T22:42:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029449411000000002 2024-09-17T22:42:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754638+0000", 2024-09-17T22:42:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1432000000000003e-05 2024-09-17T22:42:34.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754736+0000", 2024-09-17T22:42:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8066000000000006e-05 2024-09-17T22:42:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202896 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:42:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.158708785999998, 2024-09-17T22:42:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029760534000000002, 2024-09-17T22:42:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.965 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202896 2024-09-17T22:42:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:42:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:42:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725123+0000", 2024-09-17T22:42:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:42:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725125+0000", 2024-09-17T22:42:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.863e-06 2024-09-17T22:42:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725130+0000", 2024-09-17T22:42:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7250000000000002e-06 2024-09-17T22:42:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725135+0000", 2024-09-17T22:42:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:42:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754763+0000", 2024-09-17T22:42:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029628202999999999 2024-09-17T22:42:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754805+0000", 2024-09-17T22:42:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2370000000000003e-05 2024-09-17T22:42:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754883+0000", 2024-09-17T22:42:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7565999999999996e-05 2024-09-17T22:42:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202897 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:34.969 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.158691426000001, 2024-09-17T22:42:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029947972999999999, 2024-09-17T22:42:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202897 2024-09-17T22:42:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:42:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725146+0000", 2024-09-17T22:42:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6030000000000004e-06 2024-09-17T22:42:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.971 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725152+0000", 2024-09-17T22:42:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9970000000000004e-06 2024-09-17T22:42:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725156+0000", 2024-09-17T22:42:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-06 2024-09-17T22:42:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754909+0000", 2024-09-17T22:42:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029752958999999999 2024-09-17T22:42:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.972 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754987+0000", 2024-09-17T22:42:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8208999999999996e-05 2024-09-17T22:42:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.755088+0000", 2024-09-17T22:42:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100348 2024-09-17T22:42:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:34.973 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:34.973 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:34.975 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:42:34.976 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:34.972+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:34.976 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:34.976 INFO:teuthology.orchestra.run.smithi115.stdout:5603 trunc from 0xe2b0b18 to 0xe55638f 2024-09-17T22:42:34.977 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:34.977 INFO:teuthology.orchestra.run.smithi115.stdout:5605 read 0xb5798a3 thru 0xb587670 (0xddce bytes) 2024-09-17T22:42:34.977 INFO:teuthology.orchestra.run.smithi115.stdout:5606 write 0xbb663f1 thru 0xbb7426f (0xde7f bytes) 2024-09-17T22:42:34.977 INFO:teuthology.orchestra.run.smithi115.stdout:5607 writesame 0x9926533 thru 0x9935438 data_size 0xa(0xef06 bytes) 2024-09-17T22:42:34.977 INFO:teuthology.orchestra.run.smithi115.stdout:5608 writesame 0x88f569d thru 0x8904921 data_size 0x5(0xf285 bytes) 2024-09-17T22:42:34.977 INFO:teuthology.orchestra.run.smithi115.stdout:5610 trunc from 0xe55638f to 0x16faa43 2024-09-17T22:42:34.977 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:34.977 INFO:teuthology.orchestra.run.smithi115.stdout:5613 write 0xc63886f thru 0xc6418e6 (0x9078 bytes) 2024-09-17T22:42:34.977 INFO:teuthology.orchestra.run.smithi115.stdout:5615 read 0x7eb8f5 thru 0x7eca23 (0x112f bytes) 2024-09-17T22:42:34.977 INFO:teuthology.orchestra.run.smithi115.stdout:5617 write 0xd36f017 thru 0xd372e1f (0x3e09 bytes) 2024-09-17T22:42:34.977 INFO:teuthology.orchestra.run.smithi115.stdout:5618 read 0xb4fb1bb thru 0xb5001f6 (0x503c bytes) 2024-09-17T22:42:34.977 INFO:teuthology.orchestra.run.smithi115.stdout:5619 write 0x56a2efb thru 0x56a5e7f (0x2f85 bytes) 2024-09-17T22:42:34.979 INFO:teuthology.orchestra.run.smithi115.stdout:5620 write 0x60f3e27 thru 0x60f782f (0x3a09 bytes) 2024-09-17T22:42:34.982 INFO:teuthology.orchestra.run.smithi115.stdout:5621 read 0x4c53f7a thru 0x4c57fed (0x4074 bytes) 2024-09-17T22:42:34.982 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:34.993 INFO:teuthology.orchestra.run.smithi115.stdout:5623 trunc from 0xd372e20 to 0x3b21904 2024-09-17T22:42:35.002 INFO:teuthology.orchestra.run.smithi115.stdout:5624 read 0xe59261 thru 0xe63429 (0xa1c9 bytes) 2024-09-17T22:42:35.013 INFO:teuthology.orchestra.run.smithi115.stdout:5626 writesame 0x725c063 thru 0x7266a1b data_size 0x7(0xa9b9 bytes) 2024-09-17T22:42:35.013 INFO:teuthology.orchestra.run.smithi115.stdout:5627 trunc from 0x7266a1c to 0x75d980a 2024-09-17T22:42:35.019 INFO:teuthology.orchestra.run.smithi115.stdout:5628 punch from 0x32d7205 to 0x32d8833, (0x162e bytes) 2024-09-17T22:42:35.033 INFO:teuthology.orchestra.run.smithi115.stdout:5629 write 0xea112a6 thru 0xea17d09 (0x6a64 bytes) 2024-09-17T22:42:35.037 INFO:teuthology.orchestra.run.smithi115.stdout:5630 writesame 0x24286d9 thru 0x242f929 data_size 0x5(0x7251 bytes) 2024-09-17T22:42:35.054 INFO:teuthology.orchestra.run.smithi115.stdout:5631 trunc from 0xea17d0a to 0x40dffbb 2024-09-17T22:42:35.064 INFO:teuthology.orchestra.run.smithi115.stdout:5632 read 0x7c4fb4 thru 0x7cad38 (0x5d85 bytes) 2024-09-17T22:42:35.075 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:35.072+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:35.080 INFO:teuthology.orchestra.run.smithi115.stdout:5634 write 0xca0c557 thru 0xca125c4 (0x606e bytes) 2024-09-17T22:42:35.083 INFO:teuthology.orchestra.run.smithi115.stdout:5635 punch from 0x48474a0 to 0x484f7ac, (0x830c bytes) 2024-09-17T22:42:35.083 INFO:teuthology.orchestra.run.smithi115.stdout:5636 write 0x2b2841a thru 0x2b33059 (0xac40 bytes) 2024-09-17T22:42:35.085 INFO:teuthology.orchestra.run.smithi115.stdout:5637 read 0x3900cbd thru 0x3905785 (0x4ac9 bytes) 2024-09-17T22:42:35.085 INFO:teuthology.orchestra.run.smithi115.stdout:5638 read 0xb438916 thru 0xb444be2 (0xc2cd bytes) 2024-09-17T22:42:35.085 INFO:teuthology.orchestra.run.smithi115.stdout:5640 writesame 0x2bb7f9f thru 0x2bc3248 data_size 0x4d(0xb2aa bytes) 2024-09-17T22:42:35.093 INFO:teuthology.orchestra.run.smithi115.stdout:5641 trunc from 0xca125c5 to 0x79cf1cb 2024-09-17T22:42:35.105 INFO:teuthology.orchestra.run.smithi115.stdout:5642 write 0x29937d9 thru 0x299c198 (0x89c0 bytes) 2024-09-17T22:42:35.115 INFO:teuthology.orchestra.run.smithi115.stdout:5643 writesame 0xb575426 thru 0xb5845be data_size 0x1(0xf199 bytes) 2024-09-17T22:42:35.124 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:35.124 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:35.124 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:35.124 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:35.125 INFO:teuthology.orchestra.run.smithi115.stdout:5644 trunc from 0xb5845bf to 0x2729e61 2024-09-17T22:42:35.132 INFO:teuthology.orchestra.run.smithi115.stdout:5645 read 0x1ac6c2c thru 0x1acac9b (0x4070 bytes) 2024-09-17T22:42:35.132 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:35.133 INFO:teuthology.orchestra.run.smithi115.stdout:5647 punch from 0x1bc07fb to 0x1bcddd2, (0xd5d7 bytes) 2024-09-17T22:42:35.134 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:42:35.145 INFO:teuthology.orchestra.run.smithi115.stdout:5648 writesame 0x7c6eb65 thru 0x7c7ba20 data_size 0x4(0xcebc bytes) 2024-09-17T22:42:35.145 INFO:teuthology.orchestra.run.smithi115.stdout:5649 read 0x612232d thru 0x6131d7e (0xfa52 bytes) 2024-09-17T22:42:35.145 INFO:teuthology.orchestra.run.smithi115.stdout:5651 trunc from 0x7c7ba21 to 0x7b39589 2024-09-17T22:42:35.149 INFO:teuthology.orchestra.run.smithi115.stdout:5653 read 0x1889d24 thru 0x1899667 (0xf944 bytes) 2024-09-17T22:42:35.157 INFO:teuthology.orchestra.run.smithi115.stdout:5654 trunc from 0x7b39589 to 0xe1d63c0 2024-09-17T22:42:35.162 INFO:teuthology.orchestra.run.smithi115.stdout:5655 write 0xde1de1f thru 0xde25b97 (0x7d79 bytes) 2024-09-17T22:42:35.167 INFO:teuthology.orchestra.run.smithi115.stdout:5656 trunc from 0xe1d63c0 to 0xcceebc8 2024-09-17T22:42:35.174 INFO:teuthology.orchestra.run.smithi115.stdout:5658 punch from 0x997d089 to 0x998413f, (0x70b6 bytes) 2024-09-17T22:42:35.174 INFO:teuthology.orchestra.run.smithi115.stdout:5659 trunc from 0xcceebc8 to 0xccaab81 2024-09-17T22:42:35.175 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:35.172+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:35.180 INFO:teuthology.orchestra.run.smithi115.stdout:5660 read 0xca42528 thru 0xca4fa81 (0xd55a bytes) 2024-09-17T22:42:35.180 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:35.181 INFO:teuthology.orchestra.run.smithi115.stdout:5665 write 0x10254a9 thru 0x1031724 (0xc27c bytes) 2024-09-17T22:42:35.182 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:35.182 INFO:teuthology.orchestra.run.smithi115.stdout:5667 read 0x8866aac thru 0x88676dd (0xc32 bytes) 2024-09-17T22:42:35.182 INFO:teuthology.orchestra.run.smithi115.stdout:5668 punch from 0x9f7d88c to 0x9f8bebb, (0xe62f bytes) 2024-09-17T22:42:35.183 INFO:teuthology.orchestra.run.smithi115.stdout:5669 read 0xc14745d thru 0xc1495bb (0x215f bytes) 2024-09-17T22:42:35.183 INFO:teuthology.orchestra.run.smithi115.stdout:5670 read 0x5265472 thru 0x526b018 (0x5ba7 bytes) 2024-09-17T22:42:35.183 INFO:teuthology.orchestra.run.smithi115.stdout:5671 write 0xf6fc1 thru 0xfe9fb (0x7a3b bytes) 2024-09-17T22:42:35.185 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:35.185 INFO:teuthology.orchestra.run.smithi115.stdout:5674 read 0x56a52b thru 0x572d9e (0x8874 bytes) 2024-09-17T22:42:35.185 INFO:teuthology.orchestra.run.smithi115.stdout:5675 read 0xca14a6c thru 0xca1c6df (0x7c74 bytes) 2024-09-17T22:42:35.195 INFO:teuthology.orchestra.run.smithi115.stdout:5677 write 0xebd17b9 thru 0xebd4d9b (0x35e3 bytes) 2024-09-17T22:42:35.198 INFO:teuthology.orchestra.run.smithi115.stdout:5679 read 0x9d7aa9e thru 0x9d80142 (0x56a5 bytes) 2024-09-17T22:42:35.198 INFO:teuthology.orchestra.run.smithi115.stdout:5680 punch from 0x11bcfee to 0x11c8e0c, (0xbe1e bytes) 2024-09-17T22:42:35.198 INFO:teuthology.orchestra.run.smithi115.stdout:5681 punch from 0x2406342 to 0x24090e7, (0x2da5 bytes) 2024-09-17T22:42:35.198 INFO:teuthology.orchestra.run.smithi115.stdout:5683 read 0x6944cd0 thru 0x6949f14 (0x5245 bytes) 2024-09-17T22:42:35.198 INFO:teuthology.orchestra.run.smithi115.stdout:5684 read 0x2aac3ec thru 0x2aae16f (0x1d84 bytes) 2024-09-17T22:42:35.208 INFO:teuthology.orchestra.run.smithi115.stdout:5685 trunc from 0xebd4d9c to 0x6127932 2024-09-17T22:42:35.220 INFO:teuthology.orchestra.run.smithi115.stdout:5686 punch from 0x4f156bc to 0x4f1c9a8, (0x72ec bytes) 2024-09-17T22:42:35.220 INFO:teuthology.orchestra.run.smithi115.stdout:5687 read 0x1f7c33f thru 0x1f7de88 (0x1b4a bytes) 2024-09-17T22:42:35.224 INFO:teuthology.orchestra.run.smithi115.stdout:5689 trunc from 0x6127932 to 0x893a07e 2024-09-17T22:42:35.233 INFO:teuthology.orchestra.run.smithi115.stdout:5690 trunc from 0x893a07e to 0x4ab5c02 2024-09-17T22:42:35.254 INFO:teuthology.orchestra.run.smithi115.stdout:5693 write 0xbc63b7c thru 0xbc707f8 (0xcc7d bytes) 2024-09-17T22:42:35.257 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:35.263 INFO:teuthology.orchestra.run.smithi115.stdout:5695 writesame 0xc7ebc89 thru 0xc7ed987 data_size 0x1(0x1cff bytes) 2024-09-17T22:42:35.276 INFO:teuthology.orchestra.run.smithi115.stdout:5696 trunc from 0xc7ed988 to 0x233cccb 2024-09-17T22:42:35.277 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:35.273+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:35.284 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:35.284 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:35.284 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:35.284 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:35.284 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:35.293 INFO:teuthology.orchestra.run.smithi115.stdout:5700 writesame 0x2f144bc thru 0x2f21501 data_size 0x2(0xd046 bytes) 2024-09-17T22:42:35.294 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:42:35.294 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:35.297 INFO:teuthology.orchestra.run.smithi115.stdout:5704 trunc from 0x2f21502 to 0x4fa479b 2024-09-17T22:42:35.302 INFO:teuthology.orchestra.run.smithi115.stdout:5705 writesame 0x3eb43a4 thru 0x3ec1d7e data_size 0x1(0xd9db bytes) 2024-09-17T22:42:35.311 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:42:35.312 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:42:35.312 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T22:42:35.312 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.000000 2024-09-17T22:42:35.312 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 0 2024-09-17T22:42:35.328 INFO:teuthology.orchestra.run.smithi115.stdout:5708 write 0x46298f2 thru 0x462d897 (0x3fa6 bytes) 2024-09-17T22:42:35.331 INFO:teuthology.orchestra.run.smithi115.stdout:5709 punch from 0x1e9437 to 0x1ea599, (0x1162 bytes) 2024-09-17T22:42:35.347 INFO:teuthology.orchestra.run.smithi115.stdout:5710 write 0xd14c884 thru 0xd151a64 (0x51e1 bytes) 2024-09-17T22:42:35.349 INFO:teuthology.orchestra.run.smithi115.stdout:5711 read 0x36d439d thru 0x36e0dd8 (0xca3c bytes) 2024-09-17T22:42:35.357 INFO:teuthology.orchestra.run.smithi115.stdout:5713 write 0xe5a40c1 thru 0xe5a4ee8 (0xe28 bytes) 2024-09-17T22:42:35.361 INFO:teuthology.orchestra.run.smithi115.stdout:5715 read 0x683d641 thru 0x683fe55 (0x2815 bytes) 2024-09-17T22:42:35.361 INFO:teuthology.orchestra.run.smithi115.stdout:5716 writesame 0xa94ccb6 thru 0xa957858 data_size 0x1(0xaba3 bytes) 2024-09-17T22:42:35.377 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:35.375+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:35.388 INFO:teuthology.orchestra.run.smithi115.stdout:5717 write 0xa358926 thru 0xa3595e4 (0xcbf bytes) 2024-09-17T22:42:35.389 INFO:teuthology.orchestra.run.smithi115.stdout:5719 read 0x8de6b79 thru 0x8df165b (0xaae3 bytes) 2024-09-17T22:42:35.389 INFO:teuthology.orchestra.run.smithi115.stdout:5720 writesame 0x90c8e4d thru 0x90d80a5 data_size 0x1(0xf259 bytes) 2024-09-17T22:42:35.417 INFO:teuthology.orchestra.run.smithi115.stdout:5721 read 0xcb33285 thru 0xcb41713 (0xe48f bytes) 2024-09-17T22:42:35.417 INFO:teuthology.orchestra.run.smithi115.stdout:5722 write 0xbea935d thru 0xbeb01f1 (0x6e95 bytes) 2024-09-17T22:42:35.421 INFO:teuthology.orchestra.run.smithi115.stdout:5725 writesame 0xa85a719 thru 0xa86608d data_size 0x1(0xb975 bytes) 2024-09-17T22:42:35.433 INFO:teuthology.orchestra.run.smithi115.stdout:5726 write 0xa1b7e4a thru 0xa1b9f04 (0x20bb bytes) 2024-09-17T22:42:35.434 INFO:teuthology.orchestra.run.smithi115.stdout:5727 punch from 0x21bfa80 to 0x21c32e0, (0x3860 bytes) 2024-09-17T22:42:35.435 INFO:teuthology.orchestra.run.smithi115.stdout:5728 write 0xb5d134a thru 0xb5e044f (0xf106 bytes) 2024-09-17T22:42:35.436 INFO:teuthology.orchestra.run.smithi115.stdout:5729 read 0x2446a29 thru 0x2454e75 (0xe44d bytes) 2024-09-17T22:42:35.437 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:35.437 INFO:teuthology.orchestra.run.smithi115.stdout:5731 writesame 0x3de4773 thru 0x3de6954 data_size 0x1(0x21e2 bytes) 2024-09-17T22:42:35.441 INFO:teuthology.orchestra.run.smithi115.stdout:5732 writesame 0x23398cb thru 0x2347756 data_size 0x2(0xde8c bytes) 2024-09-17T22:42:35.442 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:35.443 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:35.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:35.443 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:35.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.443 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202977 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:35.443 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:42:35.443 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.713457504000001, 2024-09-17T22:42:35.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023068750999999998, 2024-09-17T22:42:35.443 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.443 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.443 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:35.443 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:35.444 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:35.444 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.444 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202977 2024-09-17T22:42:35.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.444 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:42:35.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:42:35.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725980+0000", 2024-09-17T22:42:35.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:42:35.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725982+0000", 2024-09-17T22:42:35.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0669999999999999e-06 2024-09-17T22:42:35.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725988+0000", 2024-09-17T22:42:35.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6610000000000003e-06 2024-09-17T22:42:35.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725993+0000", 2024-09-17T22:42:35.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:42:35.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748888+0000", 2024-09-17T22:42:35.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022894930000000001 2024-09-17T22:42:35.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748951+0000", 2024-09-17T22:42:35.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3638000000000003e-05 2024-09-17T22:42:35.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.447 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.447 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749048+0000", 2024-09-17T22:42:35.447 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6343999999999998e-05 2024-09-17T22:42:35.447 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.447 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.447 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.447 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.447 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202978 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2211840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:35.447 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:35.447 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.713436991999998, 2024-09-17T22:42:35.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023238050999999999, 2024-09-17T22:42:35.448 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.448 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.448 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:35.448 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:35.448 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:35.448 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.448 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202978 2024-09-17T22:42:35.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.448 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:35.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:35.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:35.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.58e-07 2024-09-17T22:42:35.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.449 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.449 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726002+0000", 2024-09-17T22:42:35.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.193e-06 2024-09-17T22:42:35.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726009+0000", 2024-09-17T22:42:35.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8000000000000001e-06 2024-09-17T22:42:35.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726014+0000", 2024-09-17T22:42:35.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9840000000000001e-06 2024-09-17T22:42:35.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749090+0000", 2024-09-17T22:42:35.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023075917000000001 2024-09-17T22:42:35.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749163+0000", 2024-09-17T22:42:35.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3219999999999994e-05 2024-09-17T22:42:35.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749238+0000", 2024-09-17T22:42:35.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4479000000000003e-05 2024-09-17T22:42:35.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.451 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.452 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202979 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2342912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:35.452 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:42:35.452 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.713415541, 2024-09-17T22:42:35.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023393536999999999, 2024-09-17T22:42:35.452 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.452 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.452 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:35.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:35.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:35.452 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.452 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202979 2024-09-17T22:42:35.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.453 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:42:35.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:42:35.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726022+0000", 2024-09-17T22:42:35.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:42:35.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726024+0000", 2024-09-17T22:42:35.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1969999999999999e-06 2024-09-17T22:42:35.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726030+0000", 2024-09-17T22:42:35.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.691e-06 2024-09-17T22:42:35.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726035+0000", 2024-09-17T22:42:35.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8029999999999996e-06 2024-09-17T22:42:35.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749280+0000", 2024-09-17T22:42:35.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023244668 2024-09-17T22:42:35.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749344+0000", 2024-09-17T22:42:35.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4418000000000006e-05 2024-09-17T22:42:35.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749415+0000", 2024-09-17T22:42:35.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0272999999999996e-05 2024-09-17T22:42:35.456 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.456 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.456 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.456 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202980 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2473984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:35.456 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:35.456 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.713394772000001, 2024-09-17T22:42:35.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023562221000000001, 2024-09-17T22:42:35.456 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.456 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.457 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:35.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:35.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:35.457 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.457 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202980 2024-09-17T22:42:35.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.457 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:35.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:35.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:35.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:42:35.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726044+0000", 2024-09-17T22:42:35.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9489999999999999e-06 2024-09-17T22:42:35.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726066+0000", 2024-09-17T22:42:35.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1508000000000001e-05 2024-09-17T22:42:35.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726072+0000", 2024-09-17T22:42:35.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3450000000000002e-06 2024-09-17T22:42:35.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749455+0000", 2024-09-17T22:42:35.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023382824999999999 2024-09-17T22:42:35.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749527+0000", 2024-09-17T22:42:35.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1728000000000002e-05 2024-09-17T22:42:35.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749604+0000", 2024-09-17T22:42:35.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7428999999999993e-05 2024-09-17T22:42:35.460 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.460 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.460 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.461 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202981 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:35.461 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:42:35.461 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.713356496999999, 2024-09-17T22:42:35.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023677800999999998, 2024-09-17T22:42:35.461 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.461 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.461 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:35.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:35.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:35.461 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.461 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202981 2024-09-17T22:42:35.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.461 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:42:35.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:42:35.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.462 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.462 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.462 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726081+0000", 2024-09-17T22:42:35.462 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5499999999999998e-07 2024-09-17T22:42:35.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726083+0000", 2024-09-17T22:42:35.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1780000000000002e-06 2024-09-17T22:42:35.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726090+0000", 2024-09-17T22:42:35.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8510000000000001e-06 2024-09-17T22:42:35.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.463 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.463 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.463 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726101+0000", 2024-09-17T22:42:35.463 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.133e-05 2024-09-17T22:42:35.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749639+0000", 2024-09-17T22:42:35.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023537679999999998 2024-09-17T22:42:35.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.464 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.464 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.464 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749688+0000", 2024-09-17T22:42:35.464 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9548000000000001e-05 2024-09-17T22:42:35.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.465 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.465 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749758+0000", 2024-09-17T22:42:35.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9758999999999994e-05 2024-09-17T22:42:35.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.465 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.465 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.465 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.465 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.465 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202982 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:35.465 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:35.465 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.713328914000002, 2024-09-17T22:42:35.465 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023841469000000001, 2024-09-17T22:42:35.465 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.466 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.466 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:35.466 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:35.466 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:35.466 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.466 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202982 2024-09-17T22:42:35.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.466 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.466 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.466 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:35.466 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.466 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:35.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:35.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:42:35.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.467 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.467 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726110+0000", 2024-09-17T22:42:35.467 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.902e-06 2024-09-17T22:42:35.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.467 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726117+0000", 2024-09-17T22:42:35.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6189999999999996e-06 2024-09-17T22:42:35.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726122+0000", 2024-09-17T22:42:35.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.198e-06 2024-09-17T22:42:35.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.468 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.468 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.468 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749795+0000", 2024-09-17T22:42:35.468 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023673397999999998 2024-09-17T22:42:35.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749877+0000", 2024-09-17T22:42:35.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1910999999999996e-05 2024-09-17T22:42:35.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.469 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.469 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749949+0000", 2024-09-17T22:42:35.469 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2001000000000004e-05 2024-09-17T22:42:35.469 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.469 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.469 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.469 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.469 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202983 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:35.470 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:35.470 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.713307855, 2024-09-17T22:42:35.470 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024014081, 2024-09-17T22:42:35.470 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.470 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.470 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:35.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:35.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:35.470 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.470 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202983 2024-09-17T22:42:35.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.470 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.470 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.470 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.470 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:35.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:35.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.471 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:35.471 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:42:35.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.471 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.471 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726131+0000", 2024-09-17T22:42:35.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9429999999999999e-06 2024-09-17T22:42:35.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726138+0000", 2024-09-17T22:42:35.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T22:42:35.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.472 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726143+0000", 2024-09-17T22:42:35.472 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0629999999999996e-06 2024-09-17T22:42:35.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.472 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.472 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749991+0000", 2024-09-17T22:42:35.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023847671000000001 2024-09-17T22:42:35.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750065+0000", 2024-09-17T22:42:35.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4569999999999999e-05 2024-09-17T22:42:35.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.473 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.473 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.473 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750143+0000", 2024-09-17T22:42:35.473 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7649000000000001e-05 2024-09-17T22:42:35.473 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.473 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.474 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.474 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.474 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202984 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:35.474 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:42:35.474 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.713287154, 2024-09-17T22:42:35.474 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024177605000000001, 2024-09-17T22:42:35.474 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.474 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.474 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:35.474 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:35.474 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:35.474 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.474 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202984 2024-09-17T22:42:35.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.475 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:42:35.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.475 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:42:35.475 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.475 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.475 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726150+0000", 2024-09-17T22:42:35.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T22:42:35.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726152+0000", 2024-09-17T22:42:35.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:42:35.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.476 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.476 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.476 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726162+0000", 2024-09-17T22:42:35.476 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7839999999999998e-06 2024-09-17T22:42:35.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726167+0000", 2024-09-17T22:42:35.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5690000000000004e-06 2024-09-17T22:42:35.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.477 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.477 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.477 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750184+0000", 2024-09-17T22:42:35.477 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024016909999999999 2024-09-17T22:42:35.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750255+0000", 2024-09-17T22:42:35.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1137999999999996e-05 2024-09-17T22:42:35.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.478 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.478 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750327+0000", 2024-09-17T22:42:35.478 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1903000000000006e-05 2024-09-17T22:42:35.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.478 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.478 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.478 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.479 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202985 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:35.479 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:35.479 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.713261766999999, 2024-09-17T22:42:35.479 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024280500999999999, 2024-09-17T22:42:35.479 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.479 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.479 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:35.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:35.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:35.479 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.479 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202985 2024-09-17T22:42:35.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.479 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:35.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:35.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.480 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.480 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.480 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:35.480 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8100000000000003e-07 2024-09-17T22:42:35.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726182+0000", 2024-09-17T22:42:35.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:42:35.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726190+0000", 2024-09-17T22:42:35.481 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:42:35.481 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.481 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.481 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.481 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726195+0000", 2024-09-17T22:42:35.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2809999999999999e-06 2024-09-17T22:42:35.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750366+0000", 2024-09-17T22:42:35.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024171635 2024-09-17T22:42:35.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.482 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750410+0000", 2024-09-17T22:42:35.482 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3823999999999999e-05 2024-09-17T22:42:35.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.482 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.482 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.483 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750455+0000", 2024-09-17T22:42:35.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5039e-05 2024-09-17T22:42:35.483 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.483 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.483 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.483 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.483 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202986 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:35.483 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:42:35.483 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.713234187000001, 2024-09-17T22:42:35.483 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024453077, 2024-09-17T22:42:35.483 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.483 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.484 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:35.484 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:35.484 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:35.484 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.484 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202986 2024-09-17T22:42:35.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.484 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.484 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.484 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:42:35.484 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.484 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:42:35.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726203+0000", 2024-09-17T22:42:35.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T22:42:35.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.485 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.485 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.485 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726205+0000", 2024-09-17T22:42:35.485 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0250000000000001e-06 2024-09-17T22:42:35.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726211+0000", 2024-09-17T22:42:35.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1340000000000001e-06 2024-09-17T22:42:35.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726222+0000", 2024-09-17T22:42:35.486 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1105e-05 2024-09-17T22:42:35.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.486 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.486 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.486 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750520+0000", 2024-09-17T22:42:35.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024297462999999998 2024-09-17T22:42:35.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750585+0000", 2024-09-17T22:42:35.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5940999999999998e-05 2024-09-17T22:42:35.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.487 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.487 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.487 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750655+0000", 2024-09-17T22:42:35.487 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0012000000000007e-05 2024-09-17T22:42:35.487 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.487 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.487 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.488 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202987 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:35.488 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:35.488 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.713207807, 2024-09-17T22:42:35.488 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024622569, 2024-09-17T22:42:35.488 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.488 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.488 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:35.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:35.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:35.488 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.488 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202987 2024-09-17T22:42:35.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.488 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.488 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:35.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:35.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.489 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:35.489 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:42:35.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.489 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.489 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726238+0000", 2024-09-17T22:42:35.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7430000000000008e-06 2024-09-17T22:42:35.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726245+0000", 2024-09-17T22:42:35.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6640000000000001e-06 2024-09-17T22:42:35.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.490 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726249+0000", 2024-09-17T22:42:35.490 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9119999999999997e-06 2024-09-17T22:42:35.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.490 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.490 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750694+0000", 2024-09-17T22:42:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024444105000000001 2024-09-17T22:42:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750767+0000", 2024-09-17T22:42:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3577000000000004e-05 2024-09-17T22:42:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750851+0000", 2024-09-17T22:42:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4189000000000003e-05 2024-09-17T22:42:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.491 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202988 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.713180945000001, 2024-09-17T22:42:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024808883, 2024-09-17T22:42:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.492 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202988 2024-09-17T22:42:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.493 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:35.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5499999999999999e-07 2024-09-17T22:42:35.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726258+0000", 2024-09-17T22:42:35.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8020000000000001e-06 2024-09-17T22:42:35.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726272+0000", 2024-09-17T22:42:35.494 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3828e-05 2024-09-17T22:42:35.494 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.494 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.494 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.494 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726277+0000", 2024-09-17T22:42:35.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2379999999999997e-06 2024-09-17T22:42:35.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750895+0000", 2024-09-17T22:42:35.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024618543 2024-09-17T22:42:35.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750978+0000", 2024-09-17T22:42:35.495 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3030999999999998e-05 2024-09-17T22:42:35.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.495 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.495 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.495 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751065+0000", 2024-09-17T22:42:35.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6086000000000002e-05 2024-09-17T22:42:35.496 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.496 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.496 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.496 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.496 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202989 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:35.496 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:42:35.496 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.713152072, 2024-09-17T22:42:35.496 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024976788, 2024-09-17T22:42:35.496 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.496 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.496 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:35.496 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:35.497 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:35.497 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.497 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202989 2024-09-17T22:42:35.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.497 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:42:35.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.497 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.497 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.497 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:42:35.497 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726285+0000", 2024-09-17T22:42:35.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T22:42:35.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726287+0000", 2024-09-17T22:42:35.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9860000000000001e-06 2024-09-17T22:42:35.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.498 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.498 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726294+0000", 2024-09-17T22:42:35.498 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2849999999999997e-06 2024-09-17T22:42:35.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.498 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726299+0000", 2024-09-17T22:42:35.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0150000000000003e-06 2024-09-17T22:42:35.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751109+0000", 2024-09-17T22:42:35.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024810175 2024-09-17T22:42:35.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.499 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.499 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.499 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751184+0000", 2024-09-17T22:42:35.499 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4640000000000004e-05 2024-09-17T22:42:35.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.500 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.500 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751261+0000", 2024-09-17T22:42:35.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7317000000000005e-05 2024-09-17T22:42:35.500 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.500 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.500 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.500 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.500 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202990 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:35.500 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:35.500 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.713131857, 2024-09-17T22:42:35.500 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025156767999999999, 2024-09-17T22:42:35.500 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.500 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.501 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:35.501 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:35.501 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:35.501 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.501 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202990 2024-09-17T22:42:35.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.501 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:35.501 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.501 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.501 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:42:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726307+0000", 2024-09-17T22:42:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7090000000000001e-06 2024-09-17T22:42:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.502 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726312+0000", 2024-09-17T22:42:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4909999999999996e-06 2024-09-17T22:42:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726317+0000", 2024-09-17T22:42:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3800000000000004e-06 2024-09-17T22:42:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751308+0000", 2024-09-17T22:42:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024991143 2024-09-17T22:42:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.503 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751385+0000", 2024-09-17T22:42:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6793999999999997e-05 2024-09-17T22:42:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751461+0000", 2024-09-17T22:42:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6891999999999995e-05 2024-09-17T22:42:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202991 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:35.504 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.713113823, 2024-09-17T22:42:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025289477000000001, 2024-09-17T22:42:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202991 2024-09-17T22:42:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.27e-07 2024-09-17T22:42:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726325+0000", 2024-09-17T22:42:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1459999999999999e-06 2024-09-17T22:42:35.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726333+0000", 2024-09-17T22:42:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1289999999999996e-06 2024-09-17T22:42:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726339+0000", 2024-09-17T22:42:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2530000000000004e-06 2024-09-17T22:42:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751499+0000", 2024-09-17T22:42:35.507 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025160109 2024-09-17T22:42:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751543+0000", 2024-09-17T22:42:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4317000000000003e-05 2024-09-17T22:42:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751612+0000", 2024-09-17T22:42:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9096000000000004e-05 2024-09-17T22:42:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.508 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202992 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.713091759000001, 2024-09-17T22:42:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025420585999999998, 2024-09-17T22:42:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202992 2024-09-17T22:42:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.509 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5199999999999998e-07 2024-09-17T22:42:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.510 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726347+0000", 2024-09-17T22:42:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9939999999999999e-06 2024-09-17T22:42:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726358+0000", 2024-09-17T22:42:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0421999999999999e-05 2024-09-17T22:42:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726363+0000", 2024-09-17T22:42:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:42:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.511 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751644+0000", 2024-09-17T22:42:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025280727999999999 2024-09-17T22:42:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751712+0000", 2024-09-17T22:42:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8207999999999995e-05 2024-09-17T22:42:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751765+0000", 2024-09-17T22:42:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3533999999999999e-05 2024-09-17T22:42:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.512 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202993 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 17.713067767999998, 2024-09-17T22:42:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025516259999999999, 2024-09-17T22:42:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202993 2024-09-17T22:42:35.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:35.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:42:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726371+0000", 2024-09-17T22:42:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T22:42:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726380+0000", 2024-09-17T22:42:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9109999999999999e-06 2024-09-17T22:42:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726385+0000", 2024-09-17T22:42:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7380000000000002e-06 2024-09-17T22:42:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751785+0000", 2024-09-17T22:42:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025400526 2024-09-17T22:42:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751824+0000", 2024-09-17T22:42:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8989999999999998e-05 2024-09-17T22:42:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751885+0000", 2024-09-17T22:42:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0937000000000003e-05 2024-09-17T22:42:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:215737 3.b 3:d5ee24db:::rbd_data.115f2f67ca35.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-17T22:42:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:42:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.13829907299999999, 2024-09-17T22:42:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025642133000000001, 2024-09-17T22:42:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:35.517 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 215737 2024-09-17T22:42:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:42:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:42:35.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301138+0000", 2024-09-17T22:42:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8100000000000002e-07 2024-09-17T22:42:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301143+0000", 2024-09-17T22:42:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6999999999999999e-06 2024-09-17T22:42:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301152+0000", 2024-09-17T22:42:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4979999999999995e-06 2024-09-17T22:42:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301159+0000", 2024-09-17T22:42:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9430000000000003e-06 2024-09-17T22:42:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301201+0000", 2024-09-17T22:42:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1219e-05 2024-09-17T22:42:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301247+0000", 2024-09-17T22:42:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5924999999999998e-05 2024-09-17T22:42:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:42:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301870+0000", 2024-09-17T22:42:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00062316100000000001 2024-09-17T22:42:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326658+0000", 2024-09-17T22:42:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024788719000000001 2024-09-17T22:42:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:42:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326694+0000", 2024-09-17T22:42:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5858999999999997e-05 2024-09-17T22:42:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326727+0000", 2024-09-17T22:42:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3176999999999998e-05 2024-09-17T22:42:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326780+0000", 2024-09-17T22:42:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2151000000000003e-05 2024-09-17T22:42:35.522 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:215738 3.b 3:d5937cde:::rbd_data.115f2f67ca35.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-17T22:42:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:42:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.13826254099999999, 2024-09-17T22:42:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025650267000000001, 2024-09-17T22:42:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:35.523 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 215738 2024-09-17T22:42:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:42:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:42:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301175+0000", 2024-09-17T22:42:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2600000000000002e-07 2024-09-17T22:42:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301178+0000", 2024-09-17T22:42:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2049999999999998e-06 2024-09-17T22:42:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301185+0000", 2024-09-17T22:42:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8900000000000001e-06 2024-09-17T22:42:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301190+0000", 2024-09-17T22:42:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6910000000000002e-06 2024-09-17T22:42:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.304605+0000", 2024-09-17T22:42:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003414594 2024-09-17T22:42:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.304658+0000", 2024-09-17T22:42:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3334000000000001e-05 2024-09-17T22:42:35.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:42:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.308265+0000", 2024-09-17T22:42:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036068680000000001 2024-09-17T22:42:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326688+0000", 2024-09-17T22:42:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018422851000000001 2024-09-17T22:42:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:42:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326785+0000", 2024-09-17T22:42:35.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7050000000000001e-05 2024-09-17T22:42:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326806+0000", 2024-09-17T22:42:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1299999999999999e-05 2024-09-17T22:42:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326824+0000", 2024-09-17T22:42:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7958e-05 2024-09-17T22:42:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop_reply(client.4447.0:215737 3.b e87/23 ondisk, result = 0)", 2024-09-17T22:42:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:42:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.13580733, 2024-09-17T22:42:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023035287000000002, 2024-09-17T22:42:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:42:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:42:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303630+0000", 2024-09-17T22:42:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2e-07 2024-09-17T22:42:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303633+0000", 2024-09-17T22:42:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7470000000000001e-06 2024-09-17T22:42:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303641+0000", 2024-09-17T22:42:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7979999999999995e-06 2024-09-17T22:42:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303646+0000", 2024-09-17T22:42:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9910000000000002e-06 2024-09-17T22:42:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326647+0000", 2024-09-17T22:42:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023000936999999999 2024-09-17T22:42:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326654+0000", 2024-09-17T22:42:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2529999999999998e-06 2024-09-17T22:42:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326665+0000", 2024-09-17T22:42:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0040999999999999e-05 2024-09-17T22:42:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:35.532 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:35.533 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:35.535 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:35.531+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:35.535 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:42:35.536 INFO:teuthology.orchestra.run.smithi115.stdout:5734 write 0xc580b62 thru 0xc5829a0 (0x1e3f bytes) 2024-09-17T22:42:35.536 INFO:teuthology.orchestra.run.smithi115.stdout:5736 write 0xd5e7ab thru 0xd68d61 (0xa5b7 bytes) 2024-09-17T22:42:35.536 INFO:teuthology.orchestra.run.smithi115.stdout:5737 read 0x327c5ec thru 0x3285541 (0x8f56 bytes) 2024-09-17T22:42:35.536 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:35.536 INFO:teuthology.orchestra.run.smithi115.stdout:5741 read 0xc440ac5 thru 0xc44a361 (0x989d bytes) 2024-09-17T22:42:35.537 INFO:teuthology.orchestra.run.smithi115.stdout:5742 read 0x7aeaa7d thru 0x7af0511 (0x5a95 bytes) 2024-09-17T22:42:35.537 INFO:teuthology.orchestra.run.smithi115.stdout:5743 read 0xc7e64bd thru 0xc7f4a1e (0xe562 bytes) 2024-09-17T22:42:35.537 INFO:teuthology.orchestra.run.smithi115.stdout:5745 read 0x1469d16 thru 0x147237a (0x8665 bytes) 2024-09-17T22:42:35.537 INFO:teuthology.orchestra.run.smithi115.stdout:5746 write 0x9f817cb thru 0x9f83404 (0x1c3a bytes) 2024-09-17T22:42:35.537 INFO:teuthology.orchestra.run.smithi115.stdout:5749 read 0x7457c18 thru 0x7462dc9 (0xb1b2 bytes) 2024-09-17T22:42:35.537 INFO:teuthology.orchestra.run.smithi115.stdout:5751 writesame 0xc3f9ace thru 0xc407cf5 data_size 0x4(0xe228 bytes) 2024-09-17T22:42:35.537 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:35.537 INFO:teuthology.orchestra.run.smithi115.stdout:5753 writesame 0x1ef096f thru 0x1ef90a8 data_size 0x13(0x873a bytes) 2024-09-17T22:42:35.537 INFO:teuthology.orchestra.run.smithi115.stdout:5754 writesame 0xb8da5b4 thru 0xb8e3928 data_size 0x1(0x9375 bytes) 2024-09-17T22:42:35.537 INFO:teuthology.orchestra.run.smithi115.stdout:5756 trunc from 0xe5a4ee9 to 0x62a7b52 2024-09-17T22:42:35.537 INFO:teuthology.orchestra.run.smithi115.stdout:5759 write 0x419e075 thru 0x41a38d8 (0x5864 bytes) 2024-09-17T22:42:35.537 INFO:teuthology.orchestra.run.smithi115.stdout:5760 write 0xe2af505 thru 0xe2b6bac (0x76a8 bytes) 2024-09-17T22:42:35.537 INFO:teuthology.orchestra.run.smithi115.stdout:5761 writesame 0x740b5d2 thru 0x74145c4 data_size 0x1(0x8ff3 bytes) 2024-09-17T22:42:35.539 INFO:teuthology.orchestra.run.smithi115.stdout:5763 writesame 0xa2ff7df thru 0xa30dc15 data_size 0x1(0xe437 bytes) 2024-09-17T22:42:35.566 INFO:teuthology.orchestra.run.smithi115.stdout:5765 punch from 0xadde1de to 0xadea5fe, (0xc420 bytes) 2024-09-17T22:42:35.566 INFO:teuthology.orchestra.run.smithi115.stdout:5766 read 0x7da2c20 thru 0x7da97d3 (0x6bb4 bytes) 2024-09-17T22:42:35.566 INFO:teuthology.orchestra.run.smithi115.stdout:5767 read 0xd85f3da thru 0xd866693 (0x72ba bytes) 2024-09-17T22:42:35.566 INFO:teuthology.orchestra.run.smithi115.stdout:5768 read 0xc22e920 thru 0xc237363 (0x8a44 bytes) 2024-09-17T22:42:35.566 INFO:teuthology.orchestra.run.smithi115.stdout:5770 read 0x2b5b9c7 thru 0x2b5f6fb (0x3d35 bytes) 2024-09-17T22:42:35.567 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:35.567 INFO:teuthology.orchestra.run.smithi115.stdout:5772 read 0x60f9959 thru 0x6102f67 (0x960f bytes) 2024-09-17T22:42:35.574 INFO:teuthology.orchestra.run.smithi115.stdout:5773 trunc from 0xe2b6bad to 0x8711aa3 2024-09-17T22:42:35.585 INFO:teuthology.orchestra.run.smithi115.stdout:5775 write 0xabb15 thru 0xbaf3c (0xf428 bytes) 2024-09-17T22:42:35.587 INFO:teuthology.orchestra.run.smithi115.stdout:5778 write 0x1e78d29 thru 0x1e7fcaf (0x6f87 bytes) 2024-09-17T22:42:35.588 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:35.589 INFO:teuthology.orchestra.run.smithi115.stdout:5781 trunc from 0x8711aa3 to 0x8e83e59 2024-09-17T22:42:35.602 INFO:teuthology.orchestra.run.smithi115.stdout:5782 writesame 0x9e230db thru 0x9e25a98 data_size 0x27(0x29be bytes) 2024-09-17T22:42:35.602 INFO:teuthology.orchestra.run.smithi115.stdout:5783 punch from 0x361836f to 0x3628118, (0xfda9 bytes) 2024-09-17T22:42:35.602 INFO:teuthology.orchestra.run.smithi115.stdout:5784 read 0x9b588b7 thru 0x9b644b6 (0xbc00 bytes) 2024-09-17T22:42:35.602 INFO:teuthology.orchestra.run.smithi115.stdout:5785 punch from 0x2f83f64 to 0x2f85f61, (0x1ffd bytes) 2024-09-17T22:42:35.602 INFO:teuthology.orchestra.run.smithi115.stdout:5787 read 0x7a79a90 thru 0x7a88f9f (0xf510 bytes) 2024-09-17T22:42:35.602 INFO:teuthology.orchestra.run.smithi115.stdout:5788 write 0x8f99476 thru 0x8fa41a0 (0xad2b bytes) 2024-09-17T22:42:35.606 INFO:teuthology.orchestra.run.smithi115.stdout:5789 read 0x8eda12c thru 0x8ee6d99 (0xcc6e bytes) 2024-09-17T22:42:35.618 INFO:teuthology.orchestra.run.smithi115.stdout:5790 write 0xe88b73b thru 0xe89a389 (0xec4f bytes) 2024-09-17T22:42:35.621 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:35.634 INFO:teuthology.orchestra.run.smithi115.stdout:5793 trunc from 0xe89a38a to 0x47ad31b 2024-09-17T22:42:35.635 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:35.632+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:35.646 INFO:teuthology.orchestra.run.smithi115.stdout:5794 write 0x4b0972 thru 0x4be38a (0xda19 bytes) 2024-09-17T22:42:35.648 INFO:teuthology.orchestra.run.smithi115.stdout:5795 read 0x4567131 thru 0x457466d (0xd53d bytes) 2024-09-17T22:42:35.649 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:35.649 INFO:teuthology.orchestra.run.smithi115.stdout:5798 punch from 0x10a600a to 0x10b5a63, (0xfa59 bytes) 2024-09-17T22:42:35.649 INFO:teuthology.orchestra.run.smithi115.stdout:5799 punch from 0x142dcd9 to 0x143c183, (0xe4aa bytes) 2024-09-17T22:42:35.649 INFO:teuthology.orchestra.run.smithi115.stdout:5801 punch from 0xb88060 to 0xb91698, (0x9638 bytes) 2024-09-17T22:42:35.649 INFO:teuthology.orchestra.run.smithi115.stdout:5802 write 0x2510d4b thru 0x2517d55 (0x700b bytes) 2024-09-17T22:42:35.650 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:35.650 INFO:teuthology.orchestra.run.smithi115.stdout:5804 read 0x36df79a thru 0x36e4a0a (0x5271 bytes) 2024-09-17T22:42:35.653 INFO:teuthology.orchestra.run.smithi115.stdout:5806 trunc from 0x47ad31b to 0x684c764 2024-09-17T22:42:35.658 INFO:teuthology.orchestra.run.smithi115.stdout:5808 write 0xb5a62f thru 0xb64b57 (0xa529 bytes) 2024-09-17T22:42:35.660 INFO:teuthology.orchestra.run.smithi115.stdout:5810 read 0xbcdb6b thru 0xbcefa8 (0x143e bytes) 2024-09-17T22:42:35.660 INFO:teuthology.orchestra.run.smithi115.stdout:5811 clone 38 order 19 su 65536 sc 4 2024-09-17T22:42:35.667 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:35.667 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:35.667 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:35.668 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:35.676 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:42:35.736 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:35.733+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:35.819 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:35.819 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:35.820 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:35.820 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:35.828 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:42:35.837 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:35.834+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:35.932 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:35.932 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:35.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:35.932 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:35.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:42:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.570557797000006, 2024-09-17T22:42:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.249158453, 2024-09-17T22:42:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:42:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:42:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-17T22:42:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.933 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358680+0000", 2024-09-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.592e-06 2024-09-17T22:42:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358688+0000", 2024-09-17T22:42:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1610000000000003e-06 2024-09-17T22:42:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358697+0000", 2024-09-17T22:42:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2580000000000001e-06 2024-09-17T22:42:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358706+0000", 2024-09-17T22:42:35.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1640000000000007e-06 2024-09-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358766+0000", 2024-09-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9893999999999999e-05 2024-09-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358778+0000", 2024-09-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1148999999999999e-05 2024-09-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606347+0000", 2024-09-17T22:42:35.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.247569971 2024-09-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606533+0000", 2024-09-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018599400000000001 2024-09-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606752+0000", 2024-09-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000218373 2024-09-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607455+0000", 2024-09-17T22:42:35.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070311799999999997 2024-09-17T22:42:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607735+0000", 2024-09-17T22:42:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027972300000000001 2024-09-17T22:42:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:42:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607793+0000", 2024-09-17T22:42:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8529999999999997e-05 2024-09-17T22:42:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:35.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607837+0000", 2024-09-17T22:42:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3526000000000003e-05 2024-09-17T22:42:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186749 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 11960320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.505916474999999, 2024-09-17T22:42:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029226166000000001, 2024-09-17T22:42:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:35.939 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186749 2024-09-17T22:42:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:35.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-09-17T22:42:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423328+0000", 2024-09-17T22:42:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6790000000000005e-06 2024-09-17T22:42:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:35.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423334+0000", 2024-09-17T22:42:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0009999999999998e-06 2024-09-17T22:42:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423338+0000", 2024-09-17T22:42:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6419999999999998e-06 2024-09-17T22:42:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452464+0000", 2024-09-17T22:42:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029125389000000002 2024-09-17T22:42:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452501+0000", 2024-09-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7656e-05 2024-09-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452546+0000", 2024-09-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4468999999999997e-05 2024-09-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186750 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12050432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:35.943 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.505890685000001, 2024-09-17T22:42:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029307863999999999, 2024-09-17T22:42:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186750 2024-09-17T22:42:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:35.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423346+0000", 2024-09-17T22:42:35.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.84e-07 2024-09-17T22:42:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423348+0000", 2024-09-17T22:42:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7540000000000001e-06 2024-09-17T22:42:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423353+0000", 2024-09-17T22:42:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:42:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423362+0000", 2024-09-17T22:42:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9220000000000003e-06 2024-09-17T22:42:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452571+0000", 2024-09-17T22:42:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029209174000000001 2024-09-17T22:42:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452612+0000", 2024-09-17T22:42:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1363999999999999e-05 2024-09-17T22:42:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452653+0000", 2024-09-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0893999999999998e-05 2024-09-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186751 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12140544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.505868195000001, 2024-09-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029387488, 2024-09-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:35.948 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186751 2024-09-17T22:42:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1399999999999997e-07 2024-09-17T22:42:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423375+0000", 2024-09-17T22:42:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8889999999999996e-06 2024-09-17T22:42:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423381+0000", 2024-09-17T22:42:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5690000000000004e-06 2024-09-17T22:42:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423386+0000", 2024-09-17T22:42:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9559999999999996e-06 2024-09-17T22:42:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452679+0000", 2024-09-17T22:42:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029292820000000001 2024-09-17T22:42:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452718+0000", 2024-09-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8961000000000002e-05 2024-09-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452755+0000", 2024-09-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7879000000000001e-05 2024-09-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186752 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12230656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:35.952 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.505836943999999, 2024-09-17T22:42:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029463269, 2024-09-17T22:42:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186752 2024-09-17T22:42:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:35.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423400+0000", 2024-09-17T22:42:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:42:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423401+0000", 2024-09-17T22:42:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3599999999999999e-06 2024-09-17T22:42:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423404+0000", 2024-09-17T22:42:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5999999999999998e-06 2024-09-17T22:42:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423407+0000", 2024-09-17T22:42:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6869999999999999e-06 2024-09-17T22:42:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:35.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452782+0000", 2024-09-17T22:42:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029375107000000001 2024-09-17T22:42:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452821+0000", 2024-09-17T22:42:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9122999999999998e-05 2024-09-17T22:42:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452862+0000", 2024-09-17T22:42:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1019000000000001e-05 2024-09-17T22:42:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:35.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186753 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.505824921999999, 2024-09-17T22:42:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029553297999999999, 2024-09-17T22:42:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186753 2024-09-17T22:42:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.957 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:35.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.98e-07 2024-09-17T22:42:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423412+0000", 2024-09-17T22:42:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8700000000000004e-07 2024-09-17T22:42:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423415+0000", 2024-09-17T22:42:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:42:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423418+0000", 2024-09-17T22:42:35.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.283e-06 2024-09-17T22:42:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452893+0000", 2024-09-17T22:42:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029475881999999998 2024-09-17T22:42:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452928+0000", 2024-09-17T22:42:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4109000000000002e-05 2024-09-17T22:42:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452964+0000", 2024-09-17T22:42:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6937000000000002e-05 2024-09-17T22:42:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186754 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12410880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.505815784999999, 2024-09-17T22:42:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029656021000000001, 2024-09-17T22:42:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:35.961 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186754 2024-09-17T22:42:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:35.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423421+0000", 2024-09-17T22:42:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9999999999999999e-07 2024-09-17T22:42:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423425+0000", 2024-09-17T22:42:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6519999999999997e-06 2024-09-17T22:42:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:35.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423428+0000", 2024-09-17T22:42:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1370000000000002e-06 2024-09-17T22:42:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423431+0000", 2024-09-17T22:42:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4090000000000001e-06 2024-09-17T22:42:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452992+0000", 2024-09-17T22:42:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029561545000000002 2024-09-17T22:42:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453031+0000", 2024-09-17T22:42:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8834000000000001e-05 2024-09-17T22:42:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453076+0000", 2024-09-17T22:42:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5244000000000003e-05 2024-09-17T22:42:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.965 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186755 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12500992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.505802504000002, 2024-09-17T22:42:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029745649999999998, 2024-09-17T22:42:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186755 2024-09-17T22:42:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:35.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.17e-07 2024-09-17T22:42:35.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423435+0000", 2024-09-17T22:42:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:42:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423438+0000", 2024-09-17T22:42:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8210000000000002e-06 2024-09-17T22:42:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423443+0000", 2024-09-17T22:42:35.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1070000000000004e-06 2024-09-17T22:42:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453103+0000", 2024-09-17T22:42:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029660399 2024-09-17T22:42:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453141+0000", 2024-09-17T22:42:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.748e-05 2024-09-17T22:42:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:35.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453179+0000", 2024-09-17T22:42:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8671000000000003e-05 2024-09-17T22:42:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187439 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8437760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.388365406999998, 2024-09-17T22:42:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029278048000000001, 2024-09-17T22:42:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:35.970 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187439 2024-09-17T22:42:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6199999999999999e-07 2024-09-17T22:42:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540872+0000", 2024-09-17T22:42:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.09e-07 2024-09-17T22:42:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540875+0000", 2024-09-17T22:42:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8990000000000001e-06 2024-09-17T22:42:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540877+0000", 2024-09-17T22:42:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1789999999999998e-06 2024-09-17T22:42:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570074+0000", 2024-09-17T22:42:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029197358 2024-09-17T22:42:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570111+0000", 2024-09-17T22:42:35.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6170000000000001e-05 2024-09-17T22:42:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570149+0000", 2024-09-17T22:42:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8170999999999998e-05 2024-09-17T22:42:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187440 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8527872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.388349937000001, 2024-09-17T22:42:35.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029359758999999999, 2024-09-17T22:42:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187440 2024-09-17T22:42:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2700000000000001e-07 2024-09-17T22:42:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540888+0000", 2024-09-17T22:42:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-17T22:42:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540893+0000", 2024-09-17T22:42:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9110000000000001e-06 2024-09-17T22:42:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540895+0000", 2024-09-17T22:42:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.712e-06 2024-09-17T22:42:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:35.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570168+0000", 2024-09-17T22:42:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029273114999999999 2024-09-17T22:42:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570208+0000", 2024-09-17T22:42:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9227999999999998e-05 2024-09-17T22:42:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570246+0000", 2024-09-17T22:42:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8253000000000001e-05 2024-09-17T22:42:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.978 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187441 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8617984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.388336973000001, 2024-09-17T22:42:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029442211999999999, 2024-09-17T22:42:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.979 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187441 2024-09-17T22:42:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:35.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:42:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540900+0000", 2024-09-17T22:42:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7999999999999993e-07 2024-09-17T22:42:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540903+0000", 2024-09-17T22:42:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9129999999999999e-06 2024-09-17T22:42:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540909+0000", 2024-09-17T22:42:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4269999999999999e-06 2024-09-17T22:42:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570267+0000", 2024-09-17T22:42:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029358308 2024-09-17T22:42:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570304+0000", 2024-09-17T22:42:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6860999999999998e-05 2024-09-17T22:42:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570341+0000", 2024-09-17T22:42:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7509000000000003e-05 2024-09-17T22:42:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187442 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8708096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.388324266000001, 2024-09-17T22:42:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029522606999999999, 2024-09-17T22:42:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:35.983 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187442 2024-09-17T22:42:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.22e-07 2024-09-17T22:42:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540917+0000", 2024-09-17T22:42:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999998e-06 2024-09-17T22:42:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540920+0000", 2024-09-17T22:42:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4989999999999999e-06 2024-09-17T22:42:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540923+0000", 2024-09-17T22:42:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3870000000000002e-06 2024-09-17T22:42:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570361+0000", 2024-09-17T22:42:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029438345000000001 2024-09-17T22:42:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:35.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570397+0000", 2024-09-17T22:42:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5429000000000002e-05 2024-09-17T22:42:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570434+0000", 2024-09-17T22:42:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7824999999999998e-05 2024-09-17T22:42:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187443 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8798208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:35.987 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.388310339, 2024-09-17T22:42:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029603640000000001, 2024-09-17T22:42:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187443 2024-09-17T22:42:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.05e-07 2024-09-17T22:42:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540927+0000", 2024-09-17T22:42:35.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:42:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540930+0000", 2024-09-17T22:42:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9229999999999998e-06 2024-09-17T22:42:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540932+0000", 2024-09-17T22:42:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.294e-06 2024-09-17T22:42:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570454+0000", 2024-09-17T22:42:35.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029522230999999999 2024-09-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570491+0000", 2024-09-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6835999999999998e-05 2024-09-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570529+0000", 2024-09-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8167999999999998e-05 2024-09-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.991 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187444 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8888320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.388300865000001, 2024-09-17T22:42:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029687584999999999, 2024-09-17T22:42:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187444 2024-09-17T22:42:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-09-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540936+0000", 2024-09-17T22:42:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3900000000000003e-07 2024-09-17T22:42:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540939+0000", 2024-09-17T22:42:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.994e-06 2024-09-17T22:42:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540942+0000", 2024-09-17T22:42:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1739999999999999e-06 2024-09-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570549+0000", 2024-09-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029607652000000002 2024-09-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570585+0000", 2024-09-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5775000000000002e-05 2024-09-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570623+0000", 2024-09-17T22:42:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7839999999999997e-05 2024-09-17T22:42:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187445 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8978432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.388291977000002, 2024-09-17T22:42:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029772837999999999, 2024-09-17T22:42:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:35.996 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187445 2024-09-17T22:42:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:35.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540945+0000", 2024-09-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3699999999999999e-07 2024-09-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540948+0000", 2024-09-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.898e-06 2024-09-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540950+0000", 2024-09-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2249999999999999e-06 2024-09-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570643+0000", 2024-09-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029692463999999998 2024-09-17T22:42:35.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570680+0000", 2024-09-17T22:42:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6772e-05 2024-09-17T22:42:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570717+0000", 2024-09-17T22:42:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7438999999999998e-05 2024-09-17T22:42:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187446 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9068544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.388283239, 2024-09-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029856321000000002, 2024-09-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187446 2024-09-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540958+0000", 2024-09-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3449999999999996e-06 2024-09-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540962+0000", 2024-09-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3529999999999999e-06 2024-09-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540964+0000", 2024-09-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3810000000000002e-06 2024-09-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570737+0000", 2024-09-17T22:42:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029772824 2024-09-17T22:42:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570770+0000", 2024-09-17T22:42:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2835e-05 2024-09-17T22:42:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570809+0000", 2024-09-17T22:42:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9379999999999999e-05 2024-09-17T22:42:36.004 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187447 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9158656~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.388269270999999, 2024-09-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02993262, 2024-09-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.005 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187447 2024-09-17T22:42:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:36.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-09-17T22:42:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540968+0000", 2024-09-17T22:42:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6500000000000008e-07 2024-09-17T22:42:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540971+0000", 2024-09-17T22:42:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9869999999999999e-06 2024-09-17T22:42:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540973+0000", 2024-09-17T22:42:36.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3120000000000001e-06 2024-09-17T22:42:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570829+0000", 2024-09-17T22:42:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029855742000000001 2024-09-17T22:42:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570861+0000", 2024-09-17T22:42:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2152e-05 2024-09-17T22:42:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570899+0000", 2024-09-17T22:42:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8241999999999997e-05 2024-09-17T22:42:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187448 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9248768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.388260241000001, 2024-09-17T22:42:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030014865000000002, 2024-09-17T22:42:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.009 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187448 2024-09-17T22:42:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9600000000000001e-07 2024-09-17T22:42:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540977+0000", 2024-09-17T22:42:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.16e-07 2024-09-17T22:42:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540980+0000", 2024-09-17T22:42:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.892e-06 2024-09-17T22:42:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540982+0000", 2024-09-17T22:42:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3410000000000001e-06 2024-09-17T22:42:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570919+0000", 2024-09-17T22:42:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029936800999999999 2024-09-17T22:42:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570953+0000", 2024-09-17T22:42:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3939000000000002e-05 2024-09-17T22:42:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570991+0000", 2024-09-17T22:42:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7780000000000001e-05 2024-09-17T22:42:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187449 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9338880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.388251061999998, 2024-09-17T22:42:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030110238000000001, 2024-09-17T22:42:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.013 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187449 2024-09-17T22:42:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1500000000000001e-07 2024-09-17T22:42:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540986+0000", 2024-09-17T22:42:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9500000000000001e-07 2024-09-17T22:42:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540989+0000", 2024-09-17T22:42:36.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7949999999999999e-06 2024-09-17T22:42:36.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540994+0000", 2024-09-17T22:42:36.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0359999999999997e-06 2024-09-17T22:42:36.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571011+0000", 2024-09-17T22:42:36.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030016915000000002 2024-09-17T22:42:36.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571056+0000", 2024-09-17T22:42:36.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5457999999999997e-05 2024-09-17T22:42:36.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571095+0000", 2024-09-17T22:42:36.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8924000000000002e-05 2024-09-17T22:42:36.017 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.017 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.017 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.017 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187450 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9428992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:36.017 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:36.018 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.388239415000001, 2024-09-17T22:42:36.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030188204999999999, 2024-09-17T22:42:36.018 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.018 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.018 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.018 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.018 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.018 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.018 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187450 2024-09-17T22:42:36.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.018 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:36.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:36.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:36.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.16e-07 2024-09-17T22:42:36.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540998+0000", 2024-09-17T22:42:36.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9000000000000005e-07 2024-09-17T22:42:36.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.541001+0000", 2024-09-17T22:42:36.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0680000000000001e-06 2024-09-17T22:42:36.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.541006+0000", 2024-09-17T22:42:36.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4720000000000003e-06 2024-09-17T22:42:36.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571110+0000", 2024-09-17T22:42:36.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030103431 2024-09-17T22:42:36.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571147+0000", 2024-09-17T22:42:36.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6748999999999997e-05 2024-09-17T22:42:36.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571185+0000", 2024-09-17T22:42:36.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8278999999999997e-05 2024-09-17T22:42:36.021 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.021 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.021 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.021 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.021 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.021 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:36.023 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:42:36.023 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:36.020+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:36.123 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:36.121+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:36.166 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:36.166 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:36.166 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:36.166 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:36.174 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:42:36.260 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:36.257+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:36.318 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:36.318 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:36.318 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:36.318 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:36.318 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:36.325 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:42:36.325 INFO:teuthology.orchestra.run.smithi115.stderr:set osd.0 primary-affinity to 0 (802) 2024-09-17T22:42:36.326 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:36.323+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:36.329 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:42:36.405 INFO:teuthology.orchestra.run.smithi115.stdout:truncating image image_client.0-clone37 from 0x684c764 (overlap 0x16faa43) to 0xb7d521 2024-09-17T22:42:36.426 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:36.424+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:36.496 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:36.496 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:36.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:36.496 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:36.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.497 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145246 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3850240~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.497 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:36.497 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.162695968999998, 2024-09-17T22:42:36.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052644991000000002, 2024-09-17T22:42:36.497 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.497 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.497 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.498 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.498 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.498 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.498 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145246 2024-09-17T22:42:36.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.498 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:36.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:36.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329899+0000", 2024-09-17T22:42:36.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-09-17T22:42:36.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329901+0000", 2024-09-17T22:42:36.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1619999999999998e-06 2024-09-17T22:42:36.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329907+0000", 2024-09-17T22:42:36.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9059999999999999e-06 2024-09-17T22:42:36.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329911+0000", 2024-09-17T22:42:36.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7419999999999997e-06 2024-09-17T22:42:36.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382383+0000", 2024-09-17T22:42:36.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052471583000000002 2024-09-17T22:42:36.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382457+0000", 2024-09-17T22:42:36.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4309999999999998e-05 2024-09-17T22:42:36.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382543+0000", 2024-09-17T22:42:36.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5918e-05 2024-09-17T22:42:36.503 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.503 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.503 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.504 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145247 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3946496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.504 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:36.504 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.162677318999997, 2024-09-17T22:42:36.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052805863000000001, 2024-09-17T22:42:36.504 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.504 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.504 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.504 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.504 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.505 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.505 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145247 2024-09-17T22:42:36.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.505 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:36.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:36.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3200000000000001e-07 2024-09-17T22:42:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329919+0000", 2024-09-17T22:42:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.888e-06 2024-09-17T22:42:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329925+0000", 2024-09-17T22:42:36.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5369999999999997e-06 2024-09-17T22:42:36.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329929+0000", 2024-09-17T22:42:36.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.639e-06 2024-09-17T22:42:36.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382570+0000", 2024-09-17T22:42:36.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052641213999999999 2024-09-17T22:42:36.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382636+0000", 2024-09-17T22:42:36.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6008999999999998e-05 2024-09-17T22:42:36.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382723+0000", 2024-09-17T22:42:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6243999999999996e-05 2024-09-17T22:42:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145248 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3856384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:36.510 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.162659271000003, 2024-09-17T22:42:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053002858, 2024-09-17T22:42:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.511 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145248 2024-09-17T22:42:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:36.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.13e-07 2024-09-17T22:42:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329937+0000", 2024-09-17T22:42:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9300000000000002e-06 2024-09-17T22:42:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329943+0000", 2024-09-17T22:42:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-09-17T22:42:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329948+0000", 2024-09-17T22:42:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8579999999999999e-06 2024-09-17T22:42:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382763+0000", 2024-09-17T22:42:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052814537000000002 2024-09-17T22:42:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382840+0000", 2024-09-17T22:42:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6879999999999996e-05 2024-09-17T22:42:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382938+0000", 2024-09-17T22:42:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8159999999999995e-05 2024-09-17T22:42:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145249 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3952640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.162640369999998, 2024-09-17T22:42:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053188602000000001, 2024-09-17T22:42:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145249 2024-09-17T22:42:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:36.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3700000000000001e-07 2024-09-17T22:42:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329956+0000", 2024-09-17T22:42:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1430000000000001e-06 2024-09-17T22:42:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329962+0000", 2024-09-17T22:42:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8270000000000003e-06 2024-09-17T22:42:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329967+0000", 2024-09-17T22:42:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7249999999999997e-06 2024-09-17T22:42:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382969+0000", 2024-09-17T22:42:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053001952999999997 2024-09-17T22:42:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383040+0000", 2024-09-17T22:42:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1272999999999993e-05 2024-09-17T22:42:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383142+0000", 2024-09-17T22:42:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010234399999999999 2024-09-17T22:42:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145250 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3862528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.162621571000003, 2024-09-17T22:42:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053359537999999998, 2024-09-17T22:42:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.520 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145250 2024-09-17T22:42:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:42:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329975+0000", 2024-09-17T22:42:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2199999999999999e-06 2024-09-17T22:42:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329981+0000", 2024-09-17T22:42:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1020000000000002e-06 2024-09-17T22:42:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329991+0000", 2024-09-17T22:42:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6800000000000005e-06 2024-09-17T22:42:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383171+0000", 2024-09-17T22:42:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053180142 2024-09-17T22:42:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383242+0000", 2024-09-17T22:42:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1310000000000007e-05 2024-09-17T22:42:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383332+0000", 2024-09-17T22:42:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9703999999999994e-05 2024-09-17T22:42:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145251 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3958784~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.162595871000001, 2024-09-17T22:42:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053438537000000001, 2024-09-17T22:42:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145251 2024-09-17T22:42:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:36.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329999+0000", 2024-09-17T22:42:36.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-07 2024-09-17T22:42:36.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330001+0000", 2024-09-17T22:42:36.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:42:36.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330006+0000", 2024-09-17T22:42:36.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000004e-06 2024-09-17T22:42:36.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330011+0000", 2024-09-17T22:42:36.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4039999999999997e-06 2024-09-17T22:42:36.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383350+0000", 2024-09-17T22:42:36.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053339011999999998 2024-09-17T22:42:36.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383387+0000", 2024-09-17T22:42:36.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7085999999999997e-05 2024-09-17T22:42:36.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383437+0000", 2024-09-17T22:42:36.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9988000000000003e-05 2024-09-17T22:42:36.532 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.532 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.532 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.533 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145252 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3868672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.533 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:36.533 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.162577423999998, 2024-09-17T22:42:36.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053526724999999997, 2024-09-17T22:42:36.533 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.533 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.533 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.533 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.533 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.534 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.534 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145252 2024-09-17T22:42:36.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.534 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:36.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:36.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331054+0000", 2024-09-17T22:42:36.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010371670000000001 2024-09-17T22:42:36.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331059+0000", 2024-09-17T22:42:36.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1680000000000003e-06 2024-09-17T22:42:36.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331075+0000", 2024-09-17T22:42:36.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6028e-05 2024-09-17T22:42:36.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331084+0000", 2024-09-17T22:42:36.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3980000000000005e-06 2024-09-17T22:42:36.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383464+0000", 2024-09-17T22:42:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052379851999999998 2024-09-17T22:42:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383499+0000", 2024-09-17T22:42:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4610000000000002e-05 2024-09-17T22:42:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383543+0000", 2024-09-17T22:42:36.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4502000000000002e-05 2024-09-17T22:42:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145255 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3971072~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.161447332000002, 2024-09-17T22:42:36.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052677527000000002, 2024-09-17T22:42:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145255 2024-09-17T22:42:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.539 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8800000000000002e-07 2024-09-17T22:42:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331150+0000", 2024-09-17T22:42:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1160000000000002e-06 2024-09-17T22:42:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331156+0000", 2024-09-17T22:42:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7469999999999999e-06 2024-09-17T22:42:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331162+0000", 2024-09-17T22:42:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-17T22:42:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383750+0000", 2024-09-17T22:42:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052588441999999999 2024-09-17T22:42:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383782+0000", 2024-09-17T22:42:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1610999999999999e-05 2024-09-17T22:42:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383824+0000", 2024-09-17T22:42:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2512000000000002e-05 2024-09-17T22:42:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.543 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145256 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3880960~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.161425911000002, 2024-09-17T22:42:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052744775000000001, 2024-09-17T22:42:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.544 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145256 2024-09-17T22:42:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:36.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331169+0000", 2024-09-17T22:42:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:42:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331175+0000", 2024-09-17T22:42:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5660000000000003e-06 2024-09-17T22:42:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331182+0000", 2024-09-17T22:42:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0169999999999999e-06 2024-09-17T22:42:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331188+0000", 2024-09-17T22:42:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2750000000000003e-06 2024-09-17T22:42:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383839+0000", 2024-09-17T22:42:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052651728000000002 2024-09-17T22:42:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383872+0000", 2024-09-17T22:42:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2347e-05 2024-09-17T22:42:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383913+0000", 2024-09-17T22:42:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1362000000000001e-05 2024-09-17T22:42:36.548 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145257 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3977216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.161399863, 2024-09-17T22:42:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052809598999999999, 2024-09-17T22:42:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145257 2024-09-17T22:42:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333051+0000", 2024-09-17T22:42:36.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018567189999999999 2024-09-17T22:42:36.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333056+0000", 2024-09-17T22:42:36.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-17T22:42:36.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333067+0000", 2024-09-17T22:42:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1704e-05 2024-09-17T22:42:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333074+0000", 2024-09-17T22:42:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1280000000000002e-06 2024-09-17T22:42:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383927+0000", 2024-09-17T22:42:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050853094000000001 2024-09-17T22:42:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383960+0000", 2024-09-17T22:42:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2891000000000001e-05 2024-09-17T22:42:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.384004+0000", 2024-09-17T22:42:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3565e-05 2024-09-17T22:42:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146168 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2467840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.097911242000002, 2024-09-17T22:42:36.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052652501999999997, 2024-09-17T22:42:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146168 2024-09-17T22:42:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394989+0000", 2024-09-17T22:42:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030618300000000002 2024-09-17T22:42:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394994+0000", 2024-09-17T22:42:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.566e-06 2024-09-17T22:42:36.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395003+0000", 2024-09-17T22:42:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3619999999999995e-06 2024-09-17T22:42:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395009+0000", 2024-09-17T22:42:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3589999999999996e-06 2024-09-17T22:42:36.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447180+0000", 2024-09-17T22:42:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052170643000000003 2024-09-17T22:42:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447251+0000", 2024-09-17T22:42:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1382000000000002e-05 2024-09-17T22:42:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447335+0000", 2024-09-17T22:42:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4006999999999997e-05 2024-09-17T22:42:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.562 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146172 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2480128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.097556357000002, 2024-09-17T22:42:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052714629999999998, 2024-09-17T22:42:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.563 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146172 2024-09-17T22:42:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:36.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395486+0000", 2024-09-17T22:42:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044787000000000001 2024-09-17T22:42:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395491+0000", 2024-09-17T22:42:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8550000000000001e-06 2024-09-17T22:42:36.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395503+0000", 2024-09-17T22:42:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2271e-05 2024-09-17T22:42:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395510+0000", 2024-09-17T22:42:36.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6320000000000008e-06 2024-09-17T22:42:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447565+0000", 2024-09-17T22:42:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052055016000000003 2024-09-17T22:42:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447644+0000", 2024-09-17T22:42:36.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8306000000000005e-05 2024-09-17T22:42:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447752+0000", 2024-09-17T22:42:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010868 2024-09-17T22:42:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.569 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146181 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2504704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.096688966999999, 2024-09-17T22:42:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052850394000000002, 2024-09-17T22:42:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146181 2024-09-17T22:42:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396210+0000", 2024-09-17T22:42:36.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000305214 2024-09-17T22:42:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396214+0000", 2024-09-17T22:42:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9389999999999999e-06 2024-09-17T22:42:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396228+0000", 2024-09-17T22:42:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3365e-05 2024-09-17T22:42:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396235+0000", 2024-09-17T22:42:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2559999999999996e-06 2024-09-17T22:42:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448568+0000", 2024-09-17T22:42:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052333084000000002 2024-09-17T22:42:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448646+0000", 2024-09-17T22:42:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8287000000000004e-05 2024-09-17T22:42:36.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448756+0000", 2024-09-17T22:42:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109249 2024-09-17T22:42:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146183 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2510848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.096348419999998, 2024-09-17T22:42:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052725453999999998, 2024-09-17T22:42:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.576 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146183 2024-09-17T22:42:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396508+0000", 2024-09-17T22:42:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026271299999999998 2024-09-17T22:42:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396512+0000", 2024-09-17T22:42:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8039999999999999e-06 2024-09-17T22:42:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396522+0000", 2024-09-17T22:42:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9289999999999993e-06 2024-09-17T22:42:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396528+0000", 2024-09-17T22:42:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2990000000000002e-06 2024-09-17T22:42:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448798+0000", 2024-09-17T22:42:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052269985999999997 2024-09-17T22:42:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448871+0000", 2024-09-17T22:42:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2581000000000002e-05 2024-09-17T22:42:36.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448971+0000", 2024-09-17T22:42:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100142 2024-09-17T22:42:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.580 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146391 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.088620927999997, 2024-09-17T22:42:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052666719000000001, 2024-09-17T22:42:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.581 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146391 2024-09-17T22:42:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404098+0000", 2024-09-17T22:42:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000124509 2024-09-17T22:42:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404107+0000", 2024-09-17T22:42:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4220000000000006e-06 2024-09-17T22:42:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404117+0000", 2024-09-17T22:42:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011e-05 2024-09-17T22:42:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404123+0000", 2024-09-17T22:42:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1530000000000002e-06 2024-09-17T22:42:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456481+0000", 2024-09-17T22:42:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052357923000000001 2024-09-17T22:42:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456551+0000", 2024-09-17T22:42:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9869000000000005e-05 2024-09-17T22:42:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456640+0000", 2024-09-17T22:42:36.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8733000000000006e-05 2024-09-17T22:42:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146393 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2748416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.088460865000002, 2024-09-17T22:42:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052739239, 2024-09-17T22:42:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.586 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146393 2024-09-17T22:42:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404443+0000", 2024-09-17T22:42:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030977199999999999 2024-09-17T22:42:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404447+0000", 2024-09-17T22:42:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6370000000000001e-06 2024-09-17T22:42:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404469+0000", 2024-09-17T22:42:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2739999999999999e-05 2024-09-17T22:42:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404477+0000", 2024-09-17T22:42:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6559999999999992e-06 2024-09-17T22:42:36.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456734+0000", 2024-09-17T22:42:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052256889000000001 2024-09-17T22:42:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456796+0000", 2024-09-17T22:42:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2485999999999998e-05 2024-09-17T22:42:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456873+0000", 2024-09-17T22:42:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6058999999999998e-05 2024-09-17T22:42:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146397 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2760704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.088085100999997, 2024-09-17T22:42:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052788928999999998, 2024-09-17T22:42:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.591 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146397 2024-09-17T22:42:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404763+0000", 2024-09-17T22:42:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025351399999999999 2024-09-17T22:42:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404767+0000", 2024-09-17T22:42:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:42:36.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404777+0000", 2024-09-17T22:42:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9310000000000003e-06 2024-09-17T22:42:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404783+0000", 2024-09-17T22:42:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.336e-06 2024-09-17T22:42:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457126+0000", 2024-09-17T22:42:36.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052342975999999999 2024-09-17T22:42:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457200+0000", 2024-09-17T22:42:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4146000000000005e-05 2024-09-17T22:42:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457298+0000", 2024-09-17T22:42:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7807999999999996e-05 2024-09-17T22:42:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146399 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2766848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.087794959999997, 2024-09-17T22:42:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052829779, 2024-09-17T22:42:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.596 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146399 2024-09-17T22:42:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404800+0000", 2024-09-17T22:42:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8500000000000001e-07 2024-09-17T22:42:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404802+0000", 2024-09-17T22:42:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5349999999999999e-06 2024-09-17T22:42:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404811+0000", 2024-09-17T22:42:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9390000000000003e-06 2024-09-17T22:42:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404817+0000", 2024-09-17T22:42:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.524e-06 2024-09-17T22:42:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457461+0000", 2024-09-17T22:42:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052643784999999998 2024-09-17T22:42:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457539+0000", 2024-09-17T22:42:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8589000000000003e-05 2024-09-17T22:42:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457629+0000", 2024-09-17T22:42:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9821999999999995e-05 2024-09-17T22:42:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146401 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2772992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.087769713, 2024-09-17T22:42:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052910077, 2024-09-17T22:42:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.601 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146401 2024-09-17T22:42:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405106+0000", 2024-09-17T22:42:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000281286 2024-09-17T22:42:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405110+0000", 2024-09-17T22:42:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1459999999999998e-06 2024-09-17T22:42:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405120+0000", 2024-09-17T22:42:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0546e-05 2024-09-17T22:42:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405127+0000", 2024-09-17T22:42:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4300000000000003e-06 2024-09-17T22:42:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457647+0000", 2024-09-17T22:42:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052520229000000002 2024-09-17T22:42:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457684+0000", 2024-09-17T22:42:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6609999999999997e-05 2024-09-17T22:42:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457735+0000", 2024-09-17T22:42:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.083e-05 2024-09-17T22:42:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.606 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146402 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2779136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.087456301000003, 2024-09-17T22:42:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052689812000000003, 2024-09-17T22:42:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:36.607 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146402 2024-09-17T22:42:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405365+0000", 2024-09-17T22:42:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022744599999999999 2024-09-17T22:42:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405370+0000", 2024-09-17T22:42:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2769999999999999e-06 2024-09-17T22:42:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405380+0000", 2024-09-17T22:42:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0349e-05 2024-09-17T22:42:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405386+0000", 2024-09-17T22:42:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5169999999999996e-06 2024-09-17T22:42:36.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457750+0000", 2024-09-17T22:42:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052363448999999999 2024-09-17T22:42:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457784+0000", 2024-09-17T22:42:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4363999999999998e-05 2024-09-17T22:42:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:36.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457828+0000", 2024-09-17T22:42:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999999e-05 2024-09-17T22:42:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:36.612 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:36.612 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:36.613 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-17T22:42:36.614 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:42:36.616 INFO:teuthology.orchestra.run.smithi115.stdout:5812 trunc from 0x684c764 to 0x18b4e2d 2024-09-17T22:42:36.616 INFO:teuthology.orchestra.run.smithi115.stdout:5814 punch from 0x1460ea6 to 0x14635cf, (0x2729 bytes) 2024-09-17T22:42:36.616 INFO:teuthology.orchestra.run.smithi115.stdout:5815 write 0x6b742d2 thru 0x6b792f1 (0x5020 bytes) 2024-09-17T22:42:36.616 INFO:teuthology.orchestra.run.smithi115.stdout:5816 write 0x7df757b thru 0x7e03999 (0xc41f bytes) 2024-09-17T22:42:36.616 INFO:teuthology.orchestra.run.smithi115.stdout:5817 read 0x5353b8a thru 0x5356cb0 (0x3127 bytes) 2024-09-17T22:42:36.616 INFO:teuthology.orchestra.run.smithi115.stdout:5818 write 0xe4db48a thru 0xe4e0879 (0x53f0 bytes) 2024-09-17T22:42:36.616 INFO:teuthology.orchestra.run.smithi115.stdout:5819 read 0xb9b0be3 thru 0xb9b4c65 (0x4083 bytes) 2024-09-17T22:42:36.617 INFO:teuthology.orchestra.run.smithi115.stdout:5820 write 0xbce3ee7 thru 0xbcebb6e (0x7c88 bytes) 2024-09-17T22:42:36.617 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:36.617 INFO:teuthology.orchestra.run.smithi115.stdout:5822 read 0xbb9f5a7 thru 0xbba9989 (0xa3e3 bytes) 2024-09-17T22:42:36.617 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:36.617 INFO:teuthology.orchestra.run.smithi115.stdout:5824 write 0x83d1fdb thru 0x83dad06 (0x8d2c bytes) 2024-09-17T22:42:36.617 INFO:teuthology.orchestra.run.smithi115.stdout:5825 punch from 0x5f2b63a to 0x5f2d297, (0x1c5d bytes) 2024-09-17T22:42:36.617 INFO:teuthology.orchestra.run.smithi115.stdout:5826 read 0x42c49e1 thru 0x42ce5e2 (0x9c02 bytes) 2024-09-17T22:42:36.617 INFO:teuthology.orchestra.run.smithi115.stdout:5827 writesame 0xc0d7812 thru 0xc0dc451 data_size 0x28(0x4c40 bytes) 2024-09-17T22:42:36.617 INFO:teuthology.orchestra.run.smithi115.stdout:5828 read 0xb57c641 thru 0xb58b78d (0xf14d bytes) 2024-09-17T22:42:36.617 INFO:teuthology.orchestra.run.smithi115.stdout:5830 trunc from 0xe4e087a to 0xcd15fd 2024-09-17T22:42:36.617 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:36.611+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:36.625 INFO:teuthology.orchestra.run.smithi115.stdout:5832 trunc from 0xcd15fd to 0x39a2f34 2024-09-17T22:42:36.631 INFO:teuthology.orchestra.run.smithi115.stdout:5833 punch from 0x268bede to 0x268e0ea, (0x220c bytes) 2024-09-17T22:42:36.631 INFO:teuthology.orchestra.run.smithi115.stdout:5834 punch from 0x2153bb1 to 0x215a5dd, (0x6a2c bytes) 2024-09-17T22:42:36.631 INFO:teuthology.orchestra.run.smithi115.stdout:5835 write 0x27a485 thru 0x2877b9 (0xd335 bytes) 2024-09-17T22:42:36.651 INFO:teuthology.orchestra.run.smithi115.stdout:5836 trunc from 0x39a2f34 to 0xd7b7c2f 2024-09-17T22:42:36.661 INFO:teuthology.orchestra.run.smithi115.stdout:5838 trunc from 0xd7b7c2f to 0xc65a1b7 2024-09-17T22:42:36.663 INFO:teuthology.orchestra.run.smithi115.stdout:5840 read 0xa015705 thru 0xa017223 (0x1b1f bytes) 2024-09-17T22:42:36.663 INFO:teuthology.orchestra.run.smithi115.stdout:5843 trunc from 0xc65a1b7 to 0xcd831aa 2024-09-17T22:42:36.668 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:36.668 INFO:teuthology.orchestra.run.smithi115.stdout:5846 write 0x5e8926c thru 0x5e98afc (0xf891 bytes) 2024-09-17T22:42:36.675 INFO:teuthology.orchestra.run.smithi115.stdout:5847 trunc from 0xcd831aa to 0xabaa234 2024-09-17T22:42:36.680 INFO:teuthology.orchestra.run.smithi115.stdout:5849 trunc from 0xabaa234 to 0xa7e2e17 2024-09-17T22:42:36.685 INFO:teuthology.orchestra.run.smithi115.stdout:5850 read 0x1333f9a thru 0x1338534 (0x459b bytes) 2024-09-17T22:42:36.685 INFO:teuthology.orchestra.run.smithi115.stdout:5851 writesame 0x20491d4 thru 0x204ca39 data_size 0x1(0x3866 bytes) 2024-09-17T22:42:36.693 INFO:teuthology.orchestra.run.smithi115.stdout:5852 writesame 0x3a1a203 thru 0x3a25dda data_size 0x4(0xbbd8 bytes) 2024-09-17T22:42:36.697 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:42:36.701 INFO:teuthology.orchestra.run.smithi115.stdout:5854 trunc from 0xa7e2e17 to 0x8d17bb7 2024-09-17T22:42:36.708 INFO:teuthology.orchestra.run.smithi115.stdout:5855 write 0x70be022 thru 0x70be7e0 (0x7bf bytes) 2024-09-17T22:42:36.711 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:36.711 INFO:teuthology.orchestra.run.smithi115.stdout:5858 write 0x726b5e9 thru 0x72777d0 (0xc1e8 bytes) 2024-09-17T22:42:36.714 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:36.712+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:36.716 INFO:teuthology.orchestra.run.smithi115.stdout:5859 punch from 0x84d4044 to 0x84e3381, (0xf33d bytes) 2024-09-17T22:42:36.716 INFO:teuthology.orchestra.run.smithi115.stdout:5861 writesame 0xd113a4 thru 0xd1e377 data_size 0x5e(0xcfd4 bytes) 2024-09-17T22:42:36.725 INFO:teuthology.orchestra.run.smithi115.stdout:5862 write 0xa5b018e thru 0xa5be36e (0xe1e1 bytes) 2024-09-17T22:42:36.728 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:36.735 INFO:teuthology.orchestra.run.smithi115.stdout:5865 write 0xb682e12 thru 0xb68c906 (0x9af5 bytes) 2024-09-17T22:42:36.746 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:36.746 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:36.747 INFO:teuthology.orchestra.run.smithi115.stdout:5867 trunc from 0xb68c907 to 0x4017e28 2024-09-17T22:42:36.757 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:42:36.766 INFO:teuthology.orchestra.run.smithi115.stdout:5868 trunc from 0x4017e28 to 0xb7930e8 2024-09-17T22:42:36.772 INFO:teuthology.orchestra.run.smithi115.stdout:5872 punch from 0x570ec00 to 0x571d0d6, (0xe4d6 bytes) 2024-09-17T22:42:36.772 INFO:teuthology.orchestra.run.smithi115.stdout:5873 read 0x7eb33a9 thru 0x7ebd885 (0xa4dd bytes) 2024-09-17T22:42:36.772 INFO:teuthology.orchestra.run.smithi115.stdout:5874 read 0x2c43bcc thru 0x2c4bf3e (0x8373 bytes) 2024-09-17T22:42:36.772 INFO:teuthology.orchestra.run.smithi115.stdout:5877 write 0x9cd8511 thru 0x9ce6347 (0xde37 bytes) 2024-09-17T22:42:36.778 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:36.778 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:36.778 INFO:teuthology.orchestra.run.smithi115.stdout:5880 write 0x5b5350f thru 0x5b578f2 (0x43e4 bytes) 2024-09-17T22:42:36.781 INFO:teuthology.orchestra.run.smithi115.stdout:5881 punch from 0x5d4b8c3 to 0x5d55041, (0x977e bytes) 2024-09-17T22:42:36.781 INFO:teuthology.orchestra.run.smithi115.stdout:5882 read 0x404d302 thru 0x4055f78 (0x8c77 bytes) 2024-09-17T22:42:36.781 INFO:teuthology.orchestra.run.smithi115.stdout:5883 write 0x6b4387a thru 0x6b47530 (0x3cb7 bytes) 2024-09-17T22:42:36.784 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:36.784 INFO:teuthology.orchestra.run.smithi115.stdout:5887 writesame 0x40c79a thru 0x40cf30 data_size 0x1(0x797 bytes) 2024-09-17T22:42:36.792 INFO:teuthology.orchestra.run.smithi115.stdout:5889 writesame 0x551043b thru 0x551ec0a data_size 0x8(0xe7d0 bytes) 2024-09-17T22:42:36.799 INFO:teuthology.orchestra.run.smithi115.stdout:5890 read 0x2ee8c2a thru 0x2ef6bf3 (0xdfca bytes) 2024-09-17T22:42:36.799 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:36.799 INFO:teuthology.orchestra.run.smithi115.stdout:5892 trunc from 0xb7930e8 to 0xc4906f6 2024-09-17T22:42:36.805 INFO:teuthology.orchestra.run.smithi115.stdout:5894 write 0x93c02dd thru 0x93cc940 (0xc664 bytes) 2024-09-17T22:42:36.808 INFO:teuthology.orchestra.run.smithi115.stdout:5895 read 0x3b06da4 thru 0x3b14572 (0xd7cf bytes) 2024-09-17T22:42:36.809 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:36.809 INFO:teuthology.orchestra.run.smithi115.stdout:5898 read 0xae0d4a0 thru 0xae1ce1e (0xf97f bytes) 2024-09-17T22:42:36.809 INFO:teuthology.orchestra.run.smithi115.stdout:5899 write 0xb6756ff thru 0xb68407a (0xe97c bytes) 2024-09-17T22:42:36.813 INFO:teuthology.orchestra.run.smithi115.stdout:5900 read 0xbd859ba thru 0xbd91248 (0xb88f bytes) 2024-09-17T22:42:36.813 INFO:teuthology.orchestra.run.smithi115.stdout:5903 write 0x8724685 thru 0x87293f7 (0x4d73 bytes) 2024-09-17T22:42:36.816 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:36.813+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:36.817 INFO:teuthology.orchestra.run.smithi115.stdout:5906 read 0xb94dbae thru 0xb95899e (0xadf1 bytes) 2024-09-17T22:42:36.818 INFO:teuthology.orchestra.run.smithi115.stdout:5907 read 0x4cb8812 thru 0x4cc6255 (0xda44 bytes) 2024-09-17T22:42:36.818 INFO:teuthology.orchestra.run.smithi115.stdout:5908 punch from 0x40f3131 to 0x40f4848, (0x1717 bytes) 2024-09-17T22:42:36.818 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:36.818 INFO:teuthology.orchestra.run.smithi115.stdout:5912 punch from 0x4e3e0e3 to 0x4e49a0f, (0xb92c bytes) 2024-09-17T22:42:36.819 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:36.819 INFO:teuthology.orchestra.run.smithi115.stdout:5913 trunc from 0xc4906f6 to 0xa9c79e6 2024-09-17T22:42:36.829 INFO:teuthology.orchestra.run.smithi115.stdout:5915 punch from 0x165c17e to 0x166b095, (0xef17 bytes) 2024-09-17T22:42:36.830 INFO:teuthology.orchestra.run.smithi115.stdout:5919 punch from 0x656bfdf to 0x6570c2f, (0x4c50 bytes) 2024-09-17T22:42:36.834 INFO:teuthology.orchestra.run.smithi115.stdout:5920 trunc from 0xa9c79e6 to 0x7056993 2024-09-17T22:42:36.838 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:36.838 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:36.844 INFO:teuthology.orchestra.run.smithi115.stdout:5922 read 0x18c239a thru 0x18c6e48 (0x4aaf bytes) 2024-09-17T22:42:36.844 INFO:teuthology.orchestra.run.smithi115.stdout:5923 read 0x3e510c5 thru 0x3e54b4d (0x3a89 bytes) 2024-09-17T22:42:36.844 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:36.855 INFO:teuthology.orchestra.run.smithi115.stdout:5925 writesame 0xae25543 thru 0xae2f31b data_size 0x1(0x9dd9 bytes) 2024-09-17T22:42:36.855 INFO:teuthology.orchestra.run.smithi115.stdout:5926 read 0x1035264 thru 0x1037cd3 (0x2a70 bytes) 2024-09-17T22:42:36.855 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:36.855 INFO:teuthology.orchestra.run.smithi115.stdout:5928 writesame 0x2b4365f thru 0x2b4544a data_size 0xa(0x1dec bytes) 2024-09-17T22:42:36.857 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:36.857 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:36.864 INFO:teuthology.orchestra.run.smithi115.stdout:5929 trunc from 0xae2f31c to 0xe6d5a5f 2024-09-17T22:42:36.867 INFO:teuthology.orchestra.run.smithi115.stdout:5930 read 0x1dbf769 thru 0x1dc2b0d (0x33a5 bytes) 2024-09-17T22:42:36.870 INFO:teuthology.orchestra.run.smithi115.stdout:5931 trunc from 0xe6d5a5f to 0xbe47c38 2024-09-17T22:42:36.874 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:36.874 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:36.876 INFO:teuthology.orchestra.run.smithi115.stdout:5933 read 0x186dcc2 thru 0x186f128 (0x1467 bytes) 2024-09-17T22:42:36.876 INFO:teuthology.orchestra.run.smithi115.stdout:5934 read 0x705c2fa thru 0x706a03c (0xdd43 bytes) 2024-09-17T22:42:36.884 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:36.884 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:36.886 INFO:teuthology.orchestra.run.smithi115.stdout:5935 write 0xeb4e533 thru 0xeb589b1 (0xa47f bytes) 2024-09-17T22:42:36.889 INFO:teuthology.orchestra.run.smithi115.stdout:5937 write 0x1b4ea5a thru 0x1b58e96 (0xa43d bytes) 2024-09-17T22:42:36.893 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:36.893 INFO:teuthology.orchestra.run.smithi115.stdout:5939 read 0x775d552 thru 0x7767a33 (0xa4e2 bytes) 2024-09-17T22:42:36.894 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:42:36.897 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:36.897 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:36.906 INFO:teuthology.orchestra.run.smithi115.stdout:5940 trunc from 0xeb589b2 to 0x5012020 2024-09-17T22:42:36.916 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:36.914+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:36.918 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:36.918 INFO:teuthology.orchestra.run.smithi115.stdout:5943 read 0xfca9ed thru 0xfd927e (0xe892 bytes) 2024-09-17T22:42:36.918 INFO:teuthology.orchestra.run.smithi115.stdout:5945 punch from 0x323ee83 to 0x3247958, (0x8ad5 bytes) 2024-09-17T22:42:36.918 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:36.918 INFO:teuthology.orchestra.run.smithi115.stdout:5947 punch from 0x2d049c5 to 0x2d08d59, (0x4394 bytes) 2024-09-17T22:42:36.918 INFO:teuthology.orchestra.run.smithi115.stdout:5948 read 0x314a246 thru 0x315605b (0xbe16 bytes) 2024-09-17T22:42:36.918 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:36.923 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:36.923 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:36.929 INFO:teuthology.orchestra.run.smithi115.stdout:5950 trunc from 0x5012020 to 0xe9a61b1 2024-09-17T22:42:36.935 INFO:teuthology.orchestra.run.smithi115.stdout:5953 write 0xb7da06c thru 0xb7e5288 (0xb21d bytes) 2024-09-17T22:42:36.941 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:36.942 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:36.953 INFO:teuthology.orchestra.run.smithi115.stdout:5954 trunc from 0xe9a61b1 to 0x53406da 2024-09-17T22:42:36.967 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:36.967 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:36.971 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:36.971 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:36.975 INFO:teuthology.orchestra.run.smithi115.stdout:5956 write 0xb5ffbf6 thru 0xb60207b (0x2486 bytes) 2024-09-17T22:42:36.979 INFO:teuthology.orchestra.run.smithi115.stdout:5957 read 0x3a746fd thru 0x3a84271 (0xfb75 bytes) 2024-09-17T22:42:36.979 INFO:teuthology.orchestra.run.smithi115.stdout:5959 writesame 0xb5a6c1f thru 0xb5b32e6 data_size 0x4(0xc6c8 bytes) 2024-09-17T22:42:36.985 INFO:teuthology.orchestra.run.smithi115.stdout:5960 punch from 0xcbd147 to 0xcbf36b, (0x2224 bytes) 2024-09-17T22:42:36.985 INFO:teuthology.orchestra.run.smithi115.stdout:5961 read 0x1c18b5a thru 0x1c2542b (0xc8d2 bytes) 2024-09-17T22:42:36.993 INFO:teuthology.orchestra.run.smithi115.stdout:5962 trunc from 0xb60207c to 0x4381722 2024-09-17T22:42:37.003 INFO:teuthology.orchestra.run.smithi115.stdout:5963 punch from 0x2a22684 to 0x2a295e6, (0x6f62 bytes) 2024-09-17T22:42:37.003 INFO:teuthology.orchestra.run.smithi115.stdout:5964 write 0x2ad8f43 thru 0x2ade45a (0x5518 bytes) 2024-09-17T22:42:37.016 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:37.016 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:37.017 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:37.014+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:37.018 INFO:teuthology.orchestra.run.smithi115.stdout:5965 trunc from 0x4381722 to 0xed28091 2024-09-17T22:42:37.023 INFO:teuthology.orchestra.run.smithi115.stdout:5966 write 0x3aae93a thru 0x3abcb45 (0xe20c bytes) 2024-09-17T22:42:37.026 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:42:37.027 INFO:teuthology.orchestra.run.smithi115.stdout:5967 write 0xbd5797f thru 0xbd5b2e5 (0x3967 bytes) 2024-09-17T22:42:37.030 INFO:teuthology.orchestra.run.smithi115.stdout:5968 read 0x6f0e6ac thru 0x6f0e6eb (0x40 bytes) 2024-09-17T22:42:37.030 INFO:teuthology.orchestra.run.smithi115.stdout:5969 read 0x377b4da thru 0x3781097 (0x5bbe bytes) 2024-09-17T22:42:37.030 INFO:teuthology.orchestra.run.smithi115.stdout:5970 write 0x8388d72 thru 0x8392490 (0x971f bytes) 2024-09-17T22:42:37.035 INFO:teuthology.orchestra.run.smithi115.stdout:5971 read 0x74fe686 thru 0x750c370 (0xdceb bytes) 2024-09-17T22:42:37.036 INFO:teuthology.orchestra.run.smithi115.stdout:5973 write 0x8cd5e4a thru 0x8ce3eab (0xe062 bytes) 2024-09-17T22:42:37.039 INFO:teuthology.orchestra.run.smithi115.stdout:5976 read 0x24e4edb thru 0x24f4d6b (0xfe91 bytes) 2024-09-17T22:42:37.039 INFO:teuthology.orchestra.run.smithi115.stdout:5977 read 0x1c16d3e thru 0x1c238e8 (0xcbab bytes) 2024-09-17T22:42:37.040 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:37.040 INFO:teuthology.orchestra.run.smithi115.stdout:5979 write 0x4f7fc34 thru 0x4f85443 (0x5810 bytes) 2024-09-17T22:42:37.042 INFO:teuthology.orchestra.run.smithi115.stdout:5981 write 0x5cfec6 thru 0x5d213a (0x2275 bytes) 2024-09-17T22:42:37.047 INFO:teuthology.orchestra.run.smithi115.stdout:5983 read 0xaf10532 thru 0xaf1d5a1 (0xd070 bytes) 2024-09-17T22:42:37.048 INFO:teuthology.orchestra.run.smithi115.stdout:5984 write 0x965270f thru 0x966043a (0xdd2c bytes) 2024-09-17T22:42:37.051 INFO:teuthology.orchestra.run.smithi115.stdout:5986 writesame 0xe33ecc9 thru 0xe342db7 data_size 0x3(0x40ef bytes) 2024-09-17T22:42:37.056 INFO:teuthology.orchestra.run.smithi115.stdout:5987 write 0xbdd2262 thru 0xbdd8705 (0x64a4 bytes) 2024-09-17T22:42:37.069 INFO:teuthology.orchestra.run.smithi115.stdout:5988 trunc from 0xed28091 to 0x4decdee 2024-09-17T22:42:37.082 INFO:teuthology.orchestra.run.smithi115.stdout:5989 read 0x38ba315 thru 0x38c64b7 (0xc1a3 bytes) 2024-09-17T22:42:37.083 INFO:teuthology.orchestra.run.smithi115.stdout:5990 read 0x349e083 thru 0x34a7610 (0x958e bytes) 2024-09-17T22:42:37.083 INFO:teuthology.orchestra.run.smithi115.stdout:5993 read 0x44ee4af thru 0x44f4f06 (0x6a58 bytes) 2024-09-17T22:42:37.083 INFO:teuthology.orchestra.run.smithi115.stdout:5994 punch from 0x1948822 to 0x1949fcb, (0x17a9 bytes) 2024-09-17T22:42:37.083 INFO:teuthology.orchestra.run.smithi115.stdout:compare and write disabled 2024-09-17T22:42:37.088 INFO:teuthology.orchestra.run.smithi115.stdout:5997 write 0x5d7a0ba thru 0x5d7c37e (0x22c5 bytes) 2024-09-17T22:42:37.117 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:37.115+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:37.126 INFO:teuthology.orchestra.run.smithi115.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-17T22:42:37.141 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:37.141 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:37.150 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:42:37.234 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:37.232+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:37.299 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:37.299 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:37.308 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:42:37.331 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is unset 2024-09-17T22:42:37.348 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:37.346+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:37.420 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:37.417+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:37.450 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:37.450 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:37.459 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:42:37.521 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:37.518+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:37.564 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:37.564 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:37.573 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:42:37.622 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:37.620+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:37.676 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:37.677 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:37.685 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:42:37.723 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:37.720+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:37.789 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:37.790 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:37.798 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:42:37.842 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:37.840+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:37.903 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:37.903 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:37.912 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:42:37.953 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:37.951+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:38.025 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:38.023+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:38.055 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:38.056 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:38.064 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:42:38.126 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:38.123+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:38.166 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:38.167 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:38.175 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:42:38.227 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:38.225+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:38.319 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:38.319 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:38.328 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:42:38.328 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:38.325+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:38.428 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:38.426+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:38.477 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:38.477 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:38.486 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:42:38.529 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:38.527+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:38.629 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:38.629 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:38.630 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:38.627+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:38.638 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:42:38.731 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:38.729+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:38.747 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:38.747 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:38.756 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:42:38.831 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:38.829+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:38.858 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:38.858 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:38.866 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:42:38.932 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:38.930+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:38.969 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:38.969 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:38.972 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:42:38.977 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:42:39.061 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:39.059+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:39.077 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:39.094 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:39.094 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:39.111 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:39.111 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:39.124 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:39.125 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:39.127 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:39.127 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:39.136 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:39.131+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:39.137 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:42:39.144 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:39.144 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:39.161 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:39.161 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:39.178 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:39.179 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:39.195 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:39.196 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:39.199 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:39.199 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:39.240 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:39.238+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:39.248 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:39.248 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:39.248 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:39.257 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:42:39.341 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:39.339+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:39.346 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:42:39.417 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145246 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3850240~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.417 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.083000390000002, 2024-09-17T22:42:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052644991000000002, 2024-09-17T22:42:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145246 2024-09-17T22:42:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329899+0000", 2024-09-17T22:42:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-09-17T22:42:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329901+0000", 2024-09-17T22:42:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1619999999999998e-06 2024-09-17T22:42:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329907+0000", 2024-09-17T22:42:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9059999999999999e-06 2024-09-17T22:42:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329911+0000", 2024-09-17T22:42:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7419999999999997e-06 2024-09-17T22:42:39.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382383+0000", 2024-09-17T22:42:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052471583000000002 2024-09-17T22:42:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382457+0000", 2024-09-17T22:42:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4309999999999998e-05 2024-09-17T22:42:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382543+0000", 2024-09-17T22:42:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5918e-05 2024-09-17T22:42:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145247 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3946496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.082981740000001, 2024-09-17T22:42:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052805863000000001, 2024-09-17T22:42:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145247 2024-09-17T22:42:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3200000000000001e-07 2024-09-17T22:42:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329919+0000", 2024-09-17T22:42:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.888e-06 2024-09-17T22:42:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329925+0000", 2024-09-17T22:42:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5369999999999997e-06 2024-09-17T22:42:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329929+0000", 2024-09-17T22:42:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.639e-06 2024-09-17T22:42:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382570+0000", 2024-09-17T22:42:39.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052641213999999999 2024-09-17T22:42:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382636+0000", 2024-09-17T22:42:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6008999999999998e-05 2024-09-17T22:42:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382723+0000", 2024-09-17T22:42:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6243999999999996e-05 2024-09-17T22:42:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.426 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145248 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3856384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.082963692, 2024-09-17T22:42:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053002858, 2024-09-17T22:42:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145248 2024-09-17T22:42:39.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.13e-07 2024-09-17T22:42:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329937+0000", 2024-09-17T22:42:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9300000000000002e-06 2024-09-17T22:42:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329943+0000", 2024-09-17T22:42:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-09-17T22:42:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329948+0000", 2024-09-17T22:42:39.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8579999999999999e-06 2024-09-17T22:42:39.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382763+0000", 2024-09-17T22:42:39.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052814537000000002 2024-09-17T22:42:39.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382840+0000", 2024-09-17T22:42:39.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6879999999999996e-05 2024-09-17T22:42:39.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382938+0000", 2024-09-17T22:42:39.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8159999999999995e-05 2024-09-17T22:42:39.431 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.431 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.431 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.431 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145249 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3952640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.431 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:39.431 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.082944791000003, 2024-09-17T22:42:39.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053188602000000001, 2024-09-17T22:42:39.432 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.432 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.432 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.432 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145249 2024-09-17T22:42:39.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.432 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:39.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:39.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:39.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3700000000000001e-07 2024-09-17T22:42:39.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329956+0000", 2024-09-17T22:42:39.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1430000000000001e-06 2024-09-17T22:42:39.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329962+0000", 2024-09-17T22:42:39.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8270000000000003e-06 2024-09-17T22:42:39.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329967+0000", 2024-09-17T22:42:39.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7249999999999997e-06 2024-09-17T22:42:39.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382969+0000", 2024-09-17T22:42:39.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053001952999999997 2024-09-17T22:42:39.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383040+0000", 2024-09-17T22:42:39.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1272999999999993e-05 2024-09-17T22:42:39.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383142+0000", 2024-09-17T22:42:39.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010234399999999999 2024-09-17T22:42:39.435 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.436 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.436 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.436 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145250 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3862528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.436 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:39.436 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.082925992, 2024-09-17T22:42:39.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053359537999999998, 2024-09-17T22:42:39.436 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.436 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.436 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.436 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.436 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.436 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.437 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145250 2024-09-17T22:42:39.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.437 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:39.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:39.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:39.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:42:39.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329975+0000", 2024-09-17T22:42:39.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2199999999999999e-06 2024-09-17T22:42:39.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329981+0000", 2024-09-17T22:42:39.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1020000000000002e-06 2024-09-17T22:42:39.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329991+0000", 2024-09-17T22:42:39.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6800000000000005e-06 2024-09-17T22:42:39.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383171+0000", 2024-09-17T22:42:39.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053180142 2024-09-17T22:42:39.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383242+0000", 2024-09-17T22:42:39.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1310000000000007e-05 2024-09-17T22:42:39.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383332+0000", 2024-09-17T22:42:39.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9703999999999994e-05 2024-09-17T22:42:39.440 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.440 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.440 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.440 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145251 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3958784~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.440 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:39.440 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.082900291999998, 2024-09-17T22:42:39.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053438537000000001, 2024-09-17T22:42:39.441 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.441 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.441 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.441 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.441 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.441 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.441 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145251 2024-09-17T22:42:39.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.441 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:39.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:39.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329999+0000", 2024-09-17T22:42:39.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-07 2024-09-17T22:42:39.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330001+0000", 2024-09-17T22:42:39.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:42:39.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330006+0000", 2024-09-17T22:42:39.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000004e-06 2024-09-17T22:42:39.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330011+0000", 2024-09-17T22:42:39.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4039999999999997e-06 2024-09-17T22:42:39.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383350+0000", 2024-09-17T22:42:39.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053339011999999998 2024-09-17T22:42:39.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383387+0000", 2024-09-17T22:42:39.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7085999999999997e-05 2024-09-17T22:42:39.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383437+0000", 2024-09-17T22:42:39.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9988000000000003e-05 2024-09-17T22:42:39.444 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.445 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.445 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145252 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3868672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.445 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:39.445 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.082881845000003, 2024-09-17T22:42:39.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053526724999999997, 2024-09-17T22:42:39.445 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.445 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.445 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.445 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.445 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.446 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145252 2024-09-17T22:42:39.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.446 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:39.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:39.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331054+0000", 2024-09-17T22:42:39.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010371670000000001 2024-09-17T22:42:39.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331059+0000", 2024-09-17T22:42:39.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1680000000000003e-06 2024-09-17T22:42:39.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331075+0000", 2024-09-17T22:42:39.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6028e-05 2024-09-17T22:42:39.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331084+0000", 2024-09-17T22:42:39.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3980000000000005e-06 2024-09-17T22:42:39.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383464+0000", 2024-09-17T22:42:39.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052379851999999998 2024-09-17T22:42:39.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383499+0000", 2024-09-17T22:42:39.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4610000000000002e-05 2024-09-17T22:42:39.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383543+0000", 2024-09-17T22:42:39.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4502000000000002e-05 2024-09-17T22:42:39.449 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.449 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.449 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.450 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145255 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3971072~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.450 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:39.450 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.081751752999999, 2024-09-17T22:42:39.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052677527000000002, 2024-09-17T22:42:39.450 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.450 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.450 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.450 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.450 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.450 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.451 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145255 2024-09-17T22:42:39.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.451 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:39.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:39.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:39.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8800000000000002e-07 2024-09-17T22:42:39.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331150+0000", 2024-09-17T22:42:39.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1160000000000002e-06 2024-09-17T22:42:39.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331156+0000", 2024-09-17T22:42:39.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7469999999999999e-06 2024-09-17T22:42:39.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331162+0000", 2024-09-17T22:42:39.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-17T22:42:39.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383750+0000", 2024-09-17T22:42:39.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052588441999999999 2024-09-17T22:42:39.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383782+0000", 2024-09-17T22:42:39.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1610999999999999e-05 2024-09-17T22:42:39.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.454 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.454 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383824+0000", 2024-09-17T22:42:39.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2512000000000002e-05 2024-09-17T22:42:39.454 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.454 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.454 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.454 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145256 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3880960~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.454 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:39.455 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.081730331999999, 2024-09-17T22:42:39.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052744775000000001, 2024-09-17T22:42:39.455 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.455 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.455 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.455 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.455 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.455 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.455 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145256 2024-09-17T22:42:39.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.455 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:39.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:39.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331169+0000", 2024-09-17T22:42:39.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:42:39.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331175+0000", 2024-09-17T22:42:39.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5660000000000003e-06 2024-09-17T22:42:39.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331182+0000", 2024-09-17T22:42:39.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0169999999999999e-06 2024-09-17T22:42:39.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331188+0000", 2024-09-17T22:42:39.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2750000000000003e-06 2024-09-17T22:42:39.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383839+0000", 2024-09-17T22:42:39.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052651728000000002 2024-09-17T22:42:39.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383872+0000", 2024-09-17T22:42:39.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2347e-05 2024-09-17T22:42:39.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383913+0000", 2024-09-17T22:42:39.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1362000000000001e-05 2024-09-17T22:42:39.459 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.459 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.459 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.459 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145257 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3977216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.459 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:39.460 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.081704283999997, 2024-09-17T22:42:39.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052809598999999999, 2024-09-17T22:42:39.460 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.460 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.460 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.460 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.460 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.460 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.460 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145257 2024-09-17T22:42:39.460 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.460 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.460 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.460 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.460 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:39.460 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:39.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.461 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.461 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333051+0000", 2024-09-17T22:42:39.461 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018567189999999999 2024-09-17T22:42:39.461 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.461 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333056+0000", 2024-09-17T22:42:39.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-17T22:42:39.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333067+0000", 2024-09-17T22:42:39.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1704e-05 2024-09-17T22:42:39.462 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.462 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.462 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333074+0000", 2024-09-17T22:42:39.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1280000000000002e-06 2024-09-17T22:42:39.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383927+0000", 2024-09-17T22:42:39.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050853094000000001 2024-09-17T22:42:39.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383960+0000", 2024-09-17T22:42:39.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2891000000000001e-05 2024-09-17T22:42:39.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.384004+0000", 2024-09-17T22:42:39.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3565e-05 2024-09-17T22:42:39.464 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.464 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.464 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.464 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146168 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2467840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.465 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:39.465 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.018215662999999, 2024-09-17T22:42:39.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052652501999999997, 2024-09-17T22:42:39.465 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.465 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.465 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.465 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.465 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.465 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.465 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146168 2024-09-17T22:42:39.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.466 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:39.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:39.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394989+0000", 2024-09-17T22:42:39.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030618300000000002 2024-09-17T22:42:39.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.467 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394994+0000", 2024-09-17T22:42:39.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.566e-06 2024-09-17T22:42:39.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395003+0000", 2024-09-17T22:42:39.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3619999999999995e-06 2024-09-17T22:42:39.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395009+0000", 2024-09-17T22:42:39.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3589999999999996e-06 2024-09-17T22:42:39.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447180+0000", 2024-09-17T22:42:39.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052170643000000003 2024-09-17T22:42:39.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447251+0000", 2024-09-17T22:42:39.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1382000000000002e-05 2024-09-17T22:42:39.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447335+0000", 2024-09-17T22:42:39.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4006999999999997e-05 2024-09-17T22:42:39.470 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.470 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.470 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.470 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146172 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2480128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.470 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:39.470 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.017860777999999, 2024-09-17T22:42:39.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052714629999999998, 2024-09-17T22:42:39.470 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.471 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.471 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.471 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.471 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.471 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.471 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146172 2024-09-17T22:42:39.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.471 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.471 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.471 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:39.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:39.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395486+0000", 2024-09-17T22:42:39.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044787000000000001 2024-09-17T22:42:39.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395491+0000", 2024-09-17T22:42:39.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8550000000000001e-06 2024-09-17T22:42:39.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395503+0000", 2024-09-17T22:42:39.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2271e-05 2024-09-17T22:42:39.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395510+0000", 2024-09-17T22:42:39.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6320000000000008e-06 2024-09-17T22:42:39.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447565+0000", 2024-09-17T22:42:39.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052055016000000003 2024-09-17T22:42:39.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447644+0000", 2024-09-17T22:42:39.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8306000000000005e-05 2024-09-17T22:42:39.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447752+0000", 2024-09-17T22:42:39.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010868 2024-09-17T22:42:39.475 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.475 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.475 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.476 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146181 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2504704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.476 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:39.476 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.016993388000003, 2024-09-17T22:42:39.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052850394000000002, 2024-09-17T22:42:39.476 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.476 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.476 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.476 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.476 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.476 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.476 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146181 2024-09-17T22:42:39.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.477 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:39.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:39.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396210+0000", 2024-09-17T22:42:39.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000305214 2024-09-17T22:42:39.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396214+0000", 2024-09-17T22:42:39.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9389999999999999e-06 2024-09-17T22:42:39.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396228+0000", 2024-09-17T22:42:39.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3365e-05 2024-09-17T22:42:39.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396235+0000", 2024-09-17T22:42:39.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2559999999999996e-06 2024-09-17T22:42:39.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448568+0000", 2024-09-17T22:42:39.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052333084000000002 2024-09-17T22:42:39.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448646+0000", 2024-09-17T22:42:39.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8287000000000004e-05 2024-09-17T22:42:39.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448756+0000", 2024-09-17T22:42:39.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109249 2024-09-17T22:42:39.480 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.480 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.480 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.480 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146183 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2510848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.480 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:39.480 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.016652841000003, 2024-09-17T22:42:39.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052725453999999998, 2024-09-17T22:42:39.480 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.481 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.481 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.481 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.481 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.481 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.481 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146183 2024-09-17T22:42:39.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.481 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:39.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:39.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396508+0000", 2024-09-17T22:42:39.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026271299999999998 2024-09-17T22:42:39.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396512+0000", 2024-09-17T22:42:39.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8039999999999999e-06 2024-09-17T22:42:39.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396522+0000", 2024-09-17T22:42:39.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9289999999999993e-06 2024-09-17T22:42:39.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396528+0000", 2024-09-17T22:42:39.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2990000000000002e-06 2024-09-17T22:42:39.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448798+0000", 2024-09-17T22:42:39.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052269985999999997 2024-09-17T22:42:39.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448871+0000", 2024-09-17T22:42:39.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2581000000000002e-05 2024-09-17T22:42:39.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448971+0000", 2024-09-17T22:42:39.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100142 2024-09-17T22:42:39.484 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.485 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.485 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.485 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146391 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.485 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:39.485 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.008925349000002, 2024-09-17T22:42:39.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052666719000000001, 2024-09-17T22:42:39.485 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.485 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.485 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.485 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.485 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.486 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.486 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146391 2024-09-17T22:42:39.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.486 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:39.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:39.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404098+0000", 2024-09-17T22:42:39.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000124509 2024-09-17T22:42:39.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404107+0000", 2024-09-17T22:42:39.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4220000000000006e-06 2024-09-17T22:42:39.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404117+0000", 2024-09-17T22:42:39.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011e-05 2024-09-17T22:42:39.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404123+0000", 2024-09-17T22:42:39.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1530000000000002e-06 2024-09-17T22:42:39.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456481+0000", 2024-09-17T22:42:39.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052357923000000001 2024-09-17T22:42:39.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456551+0000", 2024-09-17T22:42:39.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9869000000000005e-05 2024-09-17T22:42:39.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456640+0000", 2024-09-17T22:42:39.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8733000000000006e-05 2024-09-17T22:42:39.489 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.489 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.489 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.490 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146393 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2748416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.490 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:39.490 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.008765285999999, 2024-09-17T22:42:39.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052739239, 2024-09-17T22:42:39.490 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.490 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.490 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.490 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.490 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.490 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.490 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146393 2024-09-17T22:42:39.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.490 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:39.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:39.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404443+0000", 2024-09-17T22:42:39.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030977199999999999 2024-09-17T22:42:39.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404447+0000", 2024-09-17T22:42:39.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6370000000000001e-06 2024-09-17T22:42:39.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404469+0000", 2024-09-17T22:42:39.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2739999999999999e-05 2024-09-17T22:42:39.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404477+0000", 2024-09-17T22:42:39.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6559999999999992e-06 2024-09-17T22:42:39.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456734+0000", 2024-09-17T22:42:39.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052256889000000001 2024-09-17T22:42:39.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456796+0000", 2024-09-17T22:42:39.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2485999999999998e-05 2024-09-17T22:42:39.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456873+0000", 2024-09-17T22:42:39.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6058999999999998e-05 2024-09-17T22:42:39.494 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.494 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.494 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.494 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146397 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2760704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.494 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:39.495 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.008389522000002, 2024-09-17T22:42:39.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052788928999999998, 2024-09-17T22:42:39.495 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.495 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.495 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.495 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.495 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.495 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.495 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146397 2024-09-17T22:42:39.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.495 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:39.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:39.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404763+0000", 2024-09-17T22:42:39.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025351399999999999 2024-09-17T22:42:39.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404767+0000", 2024-09-17T22:42:39.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:42:39.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404777+0000", 2024-09-17T22:42:39.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9310000000000003e-06 2024-09-17T22:42:39.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404783+0000", 2024-09-17T22:42:39.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.336e-06 2024-09-17T22:42:39.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457126+0000", 2024-09-17T22:42:39.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052342975999999999 2024-09-17T22:42:39.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457200+0000", 2024-09-17T22:42:39.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4146000000000005e-05 2024-09-17T22:42:39.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457298+0000", 2024-09-17T22:42:39.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7807999999999996e-05 2024-09-17T22:42:39.499 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.499 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.499 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.499 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146399 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2766848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.499 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:39.499 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.008099381000001, 2024-09-17T22:42:39.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052829779, 2024-09-17T22:42:39.499 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.499 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.499 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.500 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.500 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.500 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.500 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146399 2024-09-17T22:42:39.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.500 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:39.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:39.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404800+0000", 2024-09-17T22:42:39.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8500000000000001e-07 2024-09-17T22:42:39.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404802+0000", 2024-09-17T22:42:39.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5349999999999999e-06 2024-09-17T22:42:39.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404811+0000", 2024-09-17T22:42:39.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9390000000000003e-06 2024-09-17T22:42:39.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404817+0000", 2024-09-17T22:42:39.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.524e-06 2024-09-17T22:42:39.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457461+0000", 2024-09-17T22:42:39.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052643784999999998 2024-09-17T22:42:39.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457539+0000", 2024-09-17T22:42:39.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8589000000000003e-05 2024-09-17T22:42:39.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457629+0000", 2024-09-17T22:42:39.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9821999999999995e-05 2024-09-17T22:42:39.503 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.503 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.503 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.504 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146401 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2772992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.504 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:39.504 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.008074133999997, 2024-09-17T22:42:39.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052910077, 2024-09-17T22:42:39.504 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.504 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.504 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.504 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.504 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.504 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.504 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146401 2024-09-17T22:42:39.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.505 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:39.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:39.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405106+0000", 2024-09-17T22:42:39.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000281286 2024-09-17T22:42:39.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405110+0000", 2024-09-17T22:42:39.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1459999999999998e-06 2024-09-17T22:42:39.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405120+0000", 2024-09-17T22:42:39.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0546e-05 2024-09-17T22:42:39.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405127+0000", 2024-09-17T22:42:39.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4300000000000003e-06 2024-09-17T22:42:39.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457647+0000", 2024-09-17T22:42:39.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052520229000000002 2024-09-17T22:42:39.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457684+0000", 2024-09-17T22:42:39.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6609999999999997e-05 2024-09-17T22:42:39.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457735+0000", 2024-09-17T22:42:39.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.083e-05 2024-09-17T22:42:39.508 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.508 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.508 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.509 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146402 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2779136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.509 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:39.509 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.007760722, 2024-09-17T22:42:39.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052689812000000003, 2024-09-17T22:42:39.509 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.509 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.509 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.509 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.510 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.510 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.510 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146402 2024-09-17T22:42:39.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.510 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:39.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:39.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405365+0000", 2024-09-17T22:42:39.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022744599999999999 2024-09-17T22:42:39.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405370+0000", 2024-09-17T22:42:39.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2769999999999999e-06 2024-09-17T22:42:39.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405380+0000", 2024-09-17T22:42:39.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0349e-05 2024-09-17T22:42:39.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405386+0000", 2024-09-17T22:42:39.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5169999999999996e-06 2024-09-17T22:42:39.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457750+0000", 2024-09-17T22:42:39.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052363448999999999 2024-09-17T22:42:39.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457784+0000", 2024-09-17T22:42:39.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4363999999999998e-05 2024-09-17T22:42:39.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457828+0000", 2024-09-17T22:42:39.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999999e-05 2024-09-17T22:42:39.513 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.513 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.513 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.514 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.514 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.514 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:39.515 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:42:39.516 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:39.512+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:39.616 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:39.613+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:39.620 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:39.620 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:39.629 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:42:39.718 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:39.715+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:39.771 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:39.771 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:39.771 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:39.771 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:39.771 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:39.779 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:42:39.823 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:39.821+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:39.883 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:39.883 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:39.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:39.884 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:39.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.884 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4229.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-17T22:42:39.884 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:39.884 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.485709994999993, 2024-09-17T22:42:39.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25708877200000002, 2024-09-17T22:42:39.884 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.884 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.884 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-17T22:42:39.884 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.884 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4229", 2024-09-17T22:42:39.884 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/3080427038", 2024-09-17T22:42:39.885 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2 2024-09-17T22:42:39.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.885 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:39.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:39.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394611+0000", 2024-09-17T22:42:39.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3149999999999999e-06 2024-09-17T22:42:39.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394619+0000", 2024-09-17T22:42:39.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240000000000003e-06 2024-09-17T22:42:39.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394632+0000", 2024-09-17T22:42:39.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2921e-05 2024-09-17T22:42:39.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394639+0000", 2024-09-17T22:42:39.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-09-17T22:42:39.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394703+0000", 2024-09-17T22:42:39.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4407999999999997e-05 2024-09-17T22:42:39.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:42:39.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394711+0000", 2024-09-17T22:42:39.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5939999999999996e-06 2024-09-17T22:42:39.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651545+0000", 2024-09-17T22:42:39.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25683409200000001 2024-09-17T22:42:39.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651698+0000", 2024-09-17T22:42:39.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000153678 2024-09-17T22:42:39.889 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.889 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.889 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.889 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38736 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:39.889 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:39.889 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.103113989999997, 2024-09-17T22:42:39.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051395418999999998, 2024-09-17T22:42:39.889 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.889 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.889 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.890 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38736 2024-09-17T22:42:39.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.890 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:39.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:39.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:39.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6899999999999998e-07 2024-09-17T22:42:39.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777212+0000", 2024-09-17T22:42:39.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1240000000000002e-06 2024-09-17T22:42:39.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777219+0000", 2024-09-17T22:42:39.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5409999999999997e-06 2024-09-17T22:42:39.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777224+0000", 2024-09-17T22:42:39.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:42:39.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.826131+0000", 2024-09-17T22:42:39.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048907369999999999 2024-09-17T22:42:39.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828457+0000", 2024-09-17T22:42:39.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023256209999999999 2024-09-17T22:42:39.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828601+0000", 2024-09-17T22:42:39.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014438799999999999 2024-09-17T22:42:39.893 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.893 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.893 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.894 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38752 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 673792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:39.894 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:39.894 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.103089918999999, 2024-09-17T22:42:39.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051582301999999997, 2024-09-17T22:42:39.894 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.894 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.894 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.894 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.894 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.894 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.894 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38752 2024-09-17T22:42:39.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.895 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:39.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:39.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:39.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3000000000000001e-07 2024-09-17T22:42:39.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777232+0000", 2024-09-17T22:42:39.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9709999999999998e-06 2024-09-17T22:42:39.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777238+0000", 2024-09-17T22:42:39.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7209999999999999e-06 2024-09-17T22:42:39.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777243+0000", 2024-09-17T22:42:39.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5210000000000004e-06 2024-09-17T22:42:39.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828633+0000", 2024-09-17T22:42:39.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051390915000000002 2024-09-17T22:42:39.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828711+0000", 2024-09-17T22:42:39.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7474999999999999e-05 2024-09-17T22:42:39.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828812+0000", 2024-09-17T22:42:39.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010116900000000001 2024-09-17T22:42:39.898 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.898 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.898 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.898 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38753 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:39.898 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:39.899 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.103070646000006, 2024-09-17T22:42:39.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051770444999999998, 2024-09-17T22:42:39.899 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.899 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.899 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.899 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.899 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.899 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.899 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38753 2024-09-17T22:42:39.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.899 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:39.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:39.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:39.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:42:39.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777251+0000", 2024-09-17T22:42:39.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8059999999999999e-06 2024-09-17T22:42:39.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777257+0000", 2024-09-17T22:42:39.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2480000000000001e-06 2024-09-17T22:42:39.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777262+0000", 2024-09-17T22:42:39.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000004e-06 2024-09-17T22:42:39.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828842+0000", 2024-09-17T22:42:39.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051580203999999998 2024-09-17T22:42:39.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828917+0000", 2024-09-17T22:42:39.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4980000000000004e-05 2024-09-17T22:42:39.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829019+0000", 2024-09-17T22:42:39.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102248 2024-09-17T22:42:39.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.902 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.903 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38766 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:39.903 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:39.903 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.103051660999995, 2024-09-17T22:42:39.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051958839999999999, 2024-09-17T22:42:39.903 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.903 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.903 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.903 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38766 2024-09-17T22:42:39.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.903 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:39.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:39.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777269+0000", 2024-09-17T22:42:39.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:42:39.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777271+0000", 2024-09-17T22:42:39.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-17T22:42:39.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777276+0000", 2024-09-17T22:42:39.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:42:39.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777281+0000", 2024-09-17T22:42:39.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:42:39.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829055+0000", 2024-09-17T22:42:39.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051773570999999997 2024-09-17T22:42:39.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829128+0000", 2024-09-17T22:42:39.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3131000000000002e-05 2024-09-17T22:42:39.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829227+0000", 2024-09-17T22:42:39.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8800000000000003e-05 2024-09-17T22:42:39.907 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.907 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.907 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.907 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38767 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:39.907 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:39.907 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.103031677999994, 2024-09-17T22:42:39.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137018, 2024-09-17T22:42:39.907 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.907 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.908 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.908 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38767 2024-09-17T22:42:39.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.908 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:39.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:39.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:39.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:42:39.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777290+0000", 2024-09-17T22:42:39.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.88e-06 2024-09-17T22:42:39.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777296+0000", 2024-09-17T22:42:39.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-09-17T22:42:39.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777300+0000", 2024-09-17T22:42:39.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4510000000000002e-06 2024-09-17T22:42:39.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829257+0000", 2024-09-17T22:42:39.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051956975000000002 2024-09-17T22:42:39.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829328+0000", 2024-09-17T22:42:39.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1026999999999996e-05 2024-09-17T22:42:39.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829425+0000", 2024-09-17T22:42:39.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.679e-05 2024-09-17T22:42:39.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.911 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.911 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38784 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 675840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:39.911 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:39.912 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.103013223999994, 2024-09-17T22:42:39.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052344522999999997, 2024-09-17T22:42:39.912 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.912 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.912 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.912 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38784 2024-09-17T22:42:39.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.912 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:39.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:39.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777307+0000", 2024-09-17T22:42:39.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:42:39.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777309+0000", 2024-09-17T22:42:39.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.063e-06 2024-09-17T22:42:39.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777314+0000", 2024-09-17T22:42:39.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-17T22:42:39.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777319+0000", 2024-09-17T22:42:39.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999996e-06 2024-09-17T22:42:39.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829456+0000", 2024-09-17T22:42:39.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137151 2024-09-17T22:42:39.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829527+0000", 2024-09-17T22:42:39.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1588000000000006e-05 2024-09-17T22:42:39.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829651+0000", 2024-09-17T22:42:39.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012365 2024-09-17T22:42:39.916 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.916 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.916 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.916 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38785 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 749568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:39.916 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:39.916 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.102995109000005, 2024-09-17T22:42:39.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052546572999999999, 2024-09-17T22:42:39.916 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.916 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.916 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.916 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.916 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.917 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.917 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38785 2024-09-17T22:42:39.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.917 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:39.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:39.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:39.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:42:39.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777327+0000", 2024-09-17T22:42:39.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:42:39.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777333+0000", 2024-09-17T22:42:39.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7309999999999998e-06 2024-09-17T22:42:39.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777337+0000", 2024-09-17T22:42:39.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:42:39.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829686+0000", 2024-09-17T22:42:39.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052349159999999999 2024-09-17T22:42:39.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829766+0000", 2024-09-17T22:42:39.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9326999999999995e-05 2024-09-17T22:42:39.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829871+0000", 2024-09-17T22:42:39.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000105466 2024-09-17T22:42:39.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.920 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.920 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38798 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:39.920 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:39.920 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.102361501000004, 2024-09-17T22:42:39.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052117564999999998, 2024-09-17T22:42:39.921 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.921 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.921 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.921 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38798 2024-09-17T22:42:39.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.921 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:39.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:39.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777959+0000", 2024-09-17T22:42:39.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9599999999999999e-07 2024-09-17T22:42:39.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777963+0000", 2024-09-17T22:42:39.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2660000000000003e-06 2024-09-17T22:42:39.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777971+0000", 2024-09-17T22:42:39.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3489999999999992e-06 2024-09-17T22:42:39.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777978+0000", 2024-09-17T22:42:39.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:42:39.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829906+0000", 2024-09-17T22:42:39.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051927635999999999 2024-09-17T22:42:39.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829986+0000", 2024-09-17T22:42:39.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0444000000000004e-05 2024-09-17T22:42:39.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830076+0000", 2024-09-17T22:42:39.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9473000000000002e-05 2024-09-17T22:42:39.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.925 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.925 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.925 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38799 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 882688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:39.925 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:39.925 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.102330061999993, 2024-09-17T22:42:39.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052359606000000003, 2024-09-17T22:42:39.925 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.925 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.925 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.926 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.926 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38799 2024-09-17T22:42:39.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.926 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:39.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:39.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:39.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8299999999999997e-07 2024-09-17T22:42:39.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777992+0000", 2024-09-17T22:42:39.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.176e-06 2024-09-17T22:42:39.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777998+0000", 2024-09-17T22:42:39.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.173e-06 2024-09-17T22:42:39.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778003+0000", 2024-09-17T22:42:39.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6349999999999997e-06 2024-09-17T22:42:39.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830112+0000", 2024-09-17T22:42:39.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052109366999999997 2024-09-17T22:42:39.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830192+0000", 2024-09-17T22:42:39.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9814000000000005e-05 2024-09-17T22:42:39.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830349+0000", 2024-09-17T22:42:39.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015695800000000001 2024-09-17T22:42:39.929 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.929 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.929 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.930 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38814 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:39.930 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:39.930 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.102046658000006, 2024-09-17T22:42:39.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052274339000000003, 2024-09-17T22:42:39.930 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.930 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.930 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.930 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.930 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.930 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.930 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38814 2024-09-17T22:42:39.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.930 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:39.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:39.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:39.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:42:39.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778276+0000", 2024-09-17T22:42:39.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2029999999999999e-06 2024-09-17T22:42:39.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778282+0000", 2024-09-17T22:42:39.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0059999999999998e-06 2024-09-17T22:42:39.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778287+0000", 2024-09-17T22:42:39.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0039999999999999e-06 2024-09-17T22:42:39.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830382+0000", 2024-09-17T22:42:39.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052095379999999997 2024-09-17T22:42:39.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830455+0000", 2024-09-17T22:42:39.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3127000000000006e-05 2024-09-17T22:42:39.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830547+0000", 2024-09-17T22:42:39.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2207999999999995e-05 2024-09-17T22:42:39.934 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.934 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.934 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.934 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38828 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 163840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:39.934 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:39.934 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.101864250000006, 2024-09-17T22:42:39.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052466039999999999, 2024-09-17T22:42:39.934 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.934 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.934 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.935 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.935 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.935 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.935 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38828 2024-09-17T22:42:39.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.935 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:39.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:39.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778456+0000", 2024-09-17T22:42:39.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7599999999999996e-07 2024-09-17T22:42:39.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778460+0000", 2024-09-17T22:42:39.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8229999999999997e-06 2024-09-17T22:42:39.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778470+0000", 2024-09-17T22:42:39.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8199999999999992e-06 2024-09-17T22:42:39.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778476+0000", 2024-09-17T22:42:39.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2589999999999997e-06 2024-09-17T22:42:39.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830578+0000", 2024-09-17T22:42:39.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052102273999999997 2024-09-17T22:42:39.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830667+0000", 2024-09-17T22:42:39.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8701000000000003e-05 2024-09-17T22:42:39.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830921+0000", 2024-09-17T22:42:39.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025438700000000002 2024-09-17T22:42:39.938 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.938 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.938 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.938 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38829 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:39.939 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:39.939 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.101832853000005, 2024-09-17T22:42:39.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052648882000000001, 2024-09-17T22:42:39.939 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.939 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.939 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.939 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.939 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.939 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.939 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38829 2024-09-17T22:42:39.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.939 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:39.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:39.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:39.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7e-07 2024-09-17T22:42:39.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778490+0000", 2024-09-17T22:42:39.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:42:39.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778495+0000", 2024-09-17T22:42:39.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9329999999999998e-06 2024-09-17T22:42:39.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778500+0000", 2024-09-17T22:42:39.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9509999999999997e-06 2024-09-17T22:42:39.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830958+0000", 2024-09-17T22:42:39.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05245785 2024-09-17T22:42:39.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831051+0000", 2024-09-17T22:42:39.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2989e-05 2024-09-17T22:42:39.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831136+0000", 2024-09-17T22:42:39.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4448000000000002e-05 2024-09-17T22:42:39.943 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.943 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.943 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.943 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38891 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:39.943 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:39.943 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.098520794999999, 2024-09-17T22:42:39.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050847934999999997, 2024-09-17T22:42:39.943 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.943 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.943 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.944 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.944 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.944 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.944 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38891 2024-09-17T22:42:39.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.944 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:39.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:39.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:39.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-09-17T22:42:39.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781802+0000", 2024-09-17T22:42:39.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:42:39.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781810+0000", 2024-09-17T22:42:39.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8019999999999992e-06 2024-09-17T22:42:39.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781816+0000", 2024-09-17T22:42:39.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.285e-06 2024-09-17T22:42:39.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832485+0000", 2024-09-17T22:42:39.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050668462999999997 2024-09-17T22:42:39.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832565+0000", 2024-09-17T22:42:39.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9692999999999997e-05 2024-09-17T22:42:39.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832647+0000", 2024-09-17T22:42:39.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2224000000000004e-05 2024-09-17T22:42:39.947 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.947 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.947 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.948 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38892 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:39.948 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:39.948 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.098492546000003, 2024-09-17T22:42:39.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050946120999999997, 2024-09-17T22:42:39.948 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.948 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.948 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.948 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38892 2024-09-17T22:42:39.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.949 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:39.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:39.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781828+0000", 2024-09-17T22:42:39.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5400000000000002e-07 2024-09-17T22:42:39.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781830+0000", 2024-09-17T22:42:39.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1959999999999998e-06 2024-09-17T22:42:39.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781837+0000", 2024-09-17T22:42:39.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8310000000000003e-06 2024-09-17T22:42:39.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781843+0000", 2024-09-17T22:42:39.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1e-06 2024-09-17T22:42:39.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832674+0000", 2024-09-17T22:42:39.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050831182000000003 2024-09-17T22:42:39.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832725+0000", 2024-09-17T22:42:39.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1588000000000001e-05 2024-09-17T22:42:39.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832773+0000", 2024-09-17T22:42:39.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7769999999999998e-05 2024-09-17T22:42:39.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.952 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.952 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38909 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 36864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:39.952 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:39.952 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.098469776000002, 2024-09-17T22:42:39.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051046518999999999, 2024-09-17T22:42:39.952 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.952 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.953 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.953 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38909 2024-09-17T22:42:39.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.953 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:39.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:39.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:39.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:42:39.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781852+0000", 2024-09-17T22:42:39.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0590000000000001e-06 2024-09-17T22:42:39.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781859+0000", 2024-09-17T22:42:39.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3380000000000001e-06 2024-09-17T22:42:39.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781864+0000", 2024-09-17T22:42:39.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:42:39.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832804+0000", 2024-09-17T22:42:39.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050939614000000001 2024-09-17T22:42:39.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832845+0000", 2024-09-17T22:42:39.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1014999999999999e-05 2024-09-17T22:42:39.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832896+0000", 2024-09-17T22:42:39.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1836999999999999e-05 2024-09-17T22:42:39.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.956 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.956 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38910 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:39.956 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:39.957 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.098449466000005, 2024-09-17T22:42:39.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051216818999999997, 2024-09-17T22:42:39.957 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.957 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.957 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.957 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38910 2024-09-17T22:42:39.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.957 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:39.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:39.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781871+0000", 2024-09-17T22:42:39.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:42:39.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781873+0000", 2024-09-17T22:42:39.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:42:39.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781879+0000", 2024-09-17T22:42:39.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0700000000000003e-06 2024-09-17T22:42:39.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781884+0000", 2024-09-17T22:42:39.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-09-17T22:42:39.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832932+0000", 2024-09-17T22:42:39.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051048219999999998 2024-09-17T22:42:39.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833001+0000", 2024-09-17T22:42:39.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8504999999999995e-05 2024-09-17T22:42:39.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833087+0000", 2024-09-17T22:42:39.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6464000000000004e-05 2024-09-17T22:42:39.960 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.961 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.961 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38923 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:39.961 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:39.961 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.098429146000001, 2024-09-17T22:42:39.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051379573999999997, 2024-09-17T22:42:39.961 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.961 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.961 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.961 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.961 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.962 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38923 2024-09-17T22:42:39.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.962 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:39.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:39.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:39.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:42:39.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781893+0000", 2024-09-17T22:42:39.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-09-17T22:42:39.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781900+0000", 2024-09-17T22:42:39.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3860000000000004e-06 2024-09-17T22:42:39.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781904+0000", 2024-09-17T22:42:39.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9300000000000002e-06 2024-09-17T22:42:39.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833115+0000", 2024-09-17T22:42:39.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051210845999999997 2024-09-17T22:42:39.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833188+0000", 2024-09-17T22:42:39.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2276999999999999e-05 2024-09-17T22:42:39.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833270+0000", 2024-09-17T22:42:39.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2501000000000002e-05 2024-09-17T22:42:39.965 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.965 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.965 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.965 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38941 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:39.965 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:39.965 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.097636610999999, 2024-09-17T22:42:39.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050969459000000002, 2024-09-17T22:42:39.965 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.966 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.966 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.966 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.966 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.966 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.966 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38941 2024-09-17T22:42:39.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.966 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:39.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:39.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782684+0000", 2024-09-17T22:42:39.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:42:39.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782686+0000", 2024-09-17T22:42:39.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.396e-06 2024-09-17T22:42:39.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782693+0000", 2024-09-17T22:42:39.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1999999999999997e-06 2024-09-17T22:42:39.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782699+0000", 2024-09-17T22:42:39.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7910000000000001e-06 2024-09-17T22:42:39.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833471+0000", 2024-09-17T22:42:39.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050772333000000003 2024-09-17T22:42:39.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833536+0000", 2024-09-17T22:42:39.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4380000000000004e-05 2024-09-17T22:42:39.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833653+0000", 2024-09-17T22:42:39.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011691799999999999 2024-09-17T22:42:39.969 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.969 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.969 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.970 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:151189 3.14 3:2b696b13:::rbd_data.115f14f2d1d5.0000000000000009:e [read 3127296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:39.970 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:39.970 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.167415865999999, 2024-09-17T22:42:39.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.054254700000000003, 2024-09-17T22:42:39.970 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:39.970 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:39.970 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:39.970 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:39.970 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:39.970 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:39.970 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 151189 2024-09-17T22:42:39.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.971 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:39.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:39.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:39.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:39.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:39.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:39.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:39.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713051+0000", 2024-09-17T22:42:39.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014722100000000001 2024-09-17T22:42:39.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:39.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713055+0000", 2024-09-17T22:42:39.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.16e-06 2024-09-17T22:42:39.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:39.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713065+0000", 2024-09-17T22:42:39.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8390000000000002e-06 2024-09-17T22:42:39.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:39.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713073+0000", 2024-09-17T22:42:39.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0390000000000005e-06 2024-09-17T22:42:39.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:39.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762644+0000", 2024-09-17T22:42:39.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049571160000000003 2024-09-17T22:42:39.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:39.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762739+0000", 2024-09-17T22:42:39.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4565e-05 2024-09-17T22:42:39.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:39.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:39.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:39.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.767159+0000", 2024-09-17T22:42:39.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044197159999999997 2024-09-17T22:42:39.974 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.974 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.974 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.974 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:39.974 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:39.974 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:39.975 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:42:39.976 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:39.973+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:40.076 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:40.074+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:40.083 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:40.083 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:40.084 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:40.084 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:40.093 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:42:40.177 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:40.175+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:40.237 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:40.237 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:40.238 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:40.238 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:40.238 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:40.246 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:42:40.289 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:40.287+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:40.348 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is unset 2024-09-17T22:42:40.352 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:40.352 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:40.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:40.353 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:40.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.353 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158673 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.353 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:40.353 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.107155339999998, 2024-09-17T22:42:40.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039400614, 2024-09-17T22:42:40.353 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.353 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.353 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.353 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.353 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.353 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.353 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158673 2024-09-17T22:42:40.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.354 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:40.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:40.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:40.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.04e-07 2024-09-17T22:42:40.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241563+0000", 2024-09-17T22:42:40.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5799999999999998e-07 2024-09-17T22:42:40.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241566+0000", 2024-09-17T22:42:40.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8279999999999999e-06 2024-09-17T22:42:40.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241568+0000", 2024-09-17T22:42:40.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2709999999999999e-06 2024-09-17T22:42:40.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280954+0000", 2024-09-17T22:42:40.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039386051999999998 2024-09-17T22:42:40.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280962+0000", 2024-09-17T22:42:40.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3010000000000007e-06 2024-09-17T22:42:40.356 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.356 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.357 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.357 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158675 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.357 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:40.357 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.107146049000001, 2024-09-17T22:42:40.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039408033000000002, 2024-09-17T22:42:40.357 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.357 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.357 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.357 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.357 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.357 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.357 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158675 2024-09-17T22:42:40.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.358 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:40.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:40.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:40.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.36e-07 2024-09-17T22:42:40.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241572+0000", 2024-09-17T22:42:40.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6599999999999994e-07 2024-09-17T22:42:40.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241575+0000", 2024-09-17T22:42:40.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8389999999999998e-06 2024-09-17T22:42:40.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241577+0000", 2024-09-17T22:42:40.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4119999999999999e-06 2024-09-17T22:42:40.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280975+0000", 2024-09-17T22:42:40.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039397205999999997 2024-09-17T22:42:40.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280979+0000", 2024-09-17T22:42:40.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.374e-06 2024-09-17T22:42:40.361 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.361 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.361 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.361 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158677 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.361 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:40.361 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.107123606000002, 2024-09-17T22:42:40.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039409307999999997, 2024-09-17T22:42:40.361 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.361 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.361 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.361 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.361 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.362 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.362 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158677 2024-09-17T22:42:40.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.362 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:40.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:40.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241594+0000", 2024-09-17T22:42:40.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:42:40.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241595+0000", 2024-09-17T22:42:40.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2729999999999999e-06 2024-09-17T22:42:40.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241602+0000", 2024-09-17T22:42:40.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:42:40.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241606+0000", 2024-09-17T22:42:40.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3019999999999999e-06 2024-09-17T22:42:40.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280994+0000", 2024-09-17T22:42:40.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039388484000000001 2024-09-17T22:42:40.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281003+0000", 2024-09-17T22:42:40.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5459999999999992e-06 2024-09-17T22:42:40.365 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.365 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.365 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.365 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158679 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.365 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:40.365 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.107107927999998, 2024-09-17T22:42:40.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039413403999999999, 2024-09-17T22:42:40.365 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.365 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.366 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.366 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.366 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.366 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.366 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158679 2024-09-17T22:42:40.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.366 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:40.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:40.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:40.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4999999999999999e-07 2024-09-17T22:42:40.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241613+0000", 2024-09-17T22:42:40.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.207e-06 2024-09-17T22:42:40.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241616+0000", 2024-09-17T22:42:40.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.275e-06 2024-09-17T22:42:40.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241618+0000", 2024-09-17T22:42:40.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:42:40.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281017+0000", 2024-09-17T22:42:40.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039398385000000001 2024-09-17T22:42:40.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281022+0000", 2024-09-17T22:42:40.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.896e-06 2024-09-17T22:42:40.369 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.369 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.369 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.369 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158681 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.369 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:40.369 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.107095749999999, 2024-09-17T22:42:40.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039425292000000001, 2024-09-17T22:42:40.370 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.370 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.370 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.370 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.370 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.370 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.370 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158681 2024-09-17T22:42:40.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.370 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:40.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:40.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:40.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:42:40.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241622+0000", 2024-09-17T22:42:40.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:42:40.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241625+0000", 2024-09-17T22:42:40.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9629999999999998e-06 2024-09-17T22:42:40.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241628+0000", 2024-09-17T22:42:40.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4339999999999999e-06 2024-09-17T22:42:40.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281039+0000", 2024-09-17T22:42:40.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039410672000000001 2024-09-17T22:42:40.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281047+0000", 2024-09-17T22:42:40.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9969999999999998e-06 2024-09-17T22:42:40.373 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.373 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.373 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.374 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158682 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.374 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:40.374 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.107086465999998, 2024-09-17T22:42:40.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039436455000000002, 2024-09-17T22:42:40.374 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.374 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.374 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.374 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.374 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.374 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.374 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158682 2024-09-17T22:42:40.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.374 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:40.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:40.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:40.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:42:40.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241632+0000", 2024-09-17T22:42:40.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:42:40.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241635+0000", 2024-09-17T22:42:40.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8940000000000002e-06 2024-09-17T22:42:40.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241637+0000", 2024-09-17T22:42:40.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3659999999999999e-06 2024-09-17T22:42:40.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281061+0000", 2024-09-17T22:42:40.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039423664999999997 2024-09-17T22:42:40.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281067+0000", 2024-09-17T22:42:40.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.285e-06 2024-09-17T22:42:40.377 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.378 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.378 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.378 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158684 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.378 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:40.378 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.10707712, 2024-09-17T22:42:40.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462980000000002, 2024-09-17T22:42:40.378 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.378 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.378 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.379 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158684 2024-09-17T22:42:40.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.379 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:40.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:40.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:40.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:40.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241641+0000", 2024-09-17T22:42:40.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:42:40.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241644+0000", 2024-09-17T22:42:40.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8870000000000001e-06 2024-09-17T22:42:40.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241646+0000", 2024-09-17T22:42:40.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3530000000000002e-06 2024-09-17T22:42:40.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281093+0000", 2024-09-17T22:42:40.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039446947000000003 2024-09-17T22:42:40.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281103+0000", 2024-09-17T22:42:40.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6299999999999993e-06 2024-09-17T22:42:40.382 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.382 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.382 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.382 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158686 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.382 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:40.382 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.107067950000001, 2024-09-17T22:42:40.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039474214000000001, 2024-09-17T22:42:40.382 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.382 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.383 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.383 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158686 2024-09-17T22:42:40.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.383 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:40.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:40.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:40.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:42:40.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241650+0000", 2024-09-17T22:42:40.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2999999999999999e-07 2024-09-17T22:42:40.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241653+0000", 2024-09-17T22:42:40.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9689999999999999e-06 2024-09-17T22:42:40.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241655+0000", 2024-09-17T22:42:40.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3460000000000001e-06 2024-09-17T22:42:40.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281118+0000", 2024-09-17T22:42:40.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462876000000001 2024-09-17T22:42:40.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281123+0000", 2024-09-17T22:42:40.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8840000000000002e-06 2024-09-17T22:42:40.386 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.386 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.386 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.386 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158688 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.386 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:40.386 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.107058719000001, 2024-09-17T22:42:40.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039490458999999999, 2024-09-17T22:42:40.386 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.387 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.387 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.387 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.387 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.387 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.387 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158688 2024-09-17T22:42:40.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.387 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:40.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:40.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:40.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:42:40.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241659+0000", 2024-09-17T22:42:40.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3699999999999999e-07 2024-09-17T22:42:40.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241662+0000", 2024-09-17T22:42:40.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8499999999999998e-06 2024-09-17T22:42:40.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241665+0000", 2024-09-17T22:42:40.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.356e-06 2024-09-17T22:42:40.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281141+0000", 2024-09-17T22:42:40.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039476031000000002 2024-09-17T22:42:40.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281149+0000", 2024-09-17T22:42:40.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0709999999999995e-06 2024-09-17T22:42:40.390 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.390 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.390 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.390 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158690 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.390 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:40.390 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.107049660000001, 2024-09-17T22:42:40.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039502378999999997, 2024-09-17T22:42:40.391 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.391 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.391 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.391 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.391 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.391 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.391 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158690 2024-09-17T22:42:40.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.391 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:40.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:40.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241668+0000", 2024-09-17T22:42:40.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:40.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241669+0000", 2024-09-17T22:42:40.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1009999999999999e-06 2024-09-17T22:42:40.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241672+0000", 2024-09-17T22:42:40.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9059999999999998e-06 2024-09-17T22:42:40.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241674+0000", 2024-09-17T22:42:40.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4439999999999998e-06 2024-09-17T22:42:40.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281162+0000", 2024-09-17T22:42:40.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039488189 2024-09-17T22:42:40.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281170+0000", 2024-09-17T22:42:40.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5310000000000004e-06 2024-09-17T22:42:40.394 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.394 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.395 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.395 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158692 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.395 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:40.395 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.107040226999999, 2024-09-17T22:42:40.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039531064999999997, 2024-09-17T22:42:40.395 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.395 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.395 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.395 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.395 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158692 2024-09-17T22:42:40.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.396 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:40.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:40.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:40.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:40.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241678+0000", 2024-09-17T22:42:40.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0800000000000003e-07 2024-09-17T22:42:40.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241681+0000", 2024-09-17T22:42:40.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8729999999999999e-06 2024-09-17T22:42:40.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241683+0000", 2024-09-17T22:42:40.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.447e-06 2024-09-17T22:42:40.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281199+0000", 2024-09-17T22:42:40.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039515388999999998 2024-09-17T22:42:40.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281208+0000", 2024-09-17T22:42:40.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2399999999999996e-06 2024-09-17T22:42:40.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.399 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.399 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158694 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.399 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:40.399 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.107031055999997, 2024-09-17T22:42:40.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039543986000000003, 2024-09-17T22:42:40.399 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.399 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.399 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.400 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158694 2024-09-17T22:42:40.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.400 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:40.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:40.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:40.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:42:40.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241687+0000", 2024-09-17T22:42:40.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:42:40.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241691+0000", 2024-09-17T22:42:40.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.811e-06 2024-09-17T22:42:40.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241693+0000", 2024-09-17T22:42:40.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.385e-06 2024-09-17T22:42:40.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281223+0000", 2024-09-17T22:42:40.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039529332 2024-09-17T22:42:40.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281230+0000", 2024-09-17T22:42:40.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2139999999999999e-06 2024-09-17T22:42:40.403 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.403 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.403 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.403 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158696 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.403 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:40.403 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.107020667999997, 2024-09-17T22:42:40.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039570458000000003, 2024-09-17T22:42:40.403 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.403 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.403 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.404 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.404 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.404 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.404 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158696 2024-09-17T22:42:40.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.404 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:40.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:40.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:42:40.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:42:40.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.405 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:42:40.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.33e-07 2024-09-17T22:42:40.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241700+0000", 2024-09-17T22:42:40.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8949999999999998e-06 2024-09-17T22:42:40.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241703+0000", 2024-09-17T22:42:40.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3159999999999999e-06 2024-09-17T22:42:40.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281257+0000", 2024-09-17T22:42:40.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039553862000000002 2024-09-17T22:42:40.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281267+0000", 2024-09-17T22:42:40.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.024e-05 2024-09-17T22:42:40.407 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.407 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.407 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.407 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158698 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.407 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:40.407 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.107010590000002, 2024-09-17T22:42:40.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039576493999999997, 2024-09-17T22:42:40.407 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.408 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.408 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.408 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.408 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.408 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.408 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158698 2024-09-17T22:42:40.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.408 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:40.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:40.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241707+0000", 2024-09-17T22:42:40.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:42:40.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241708+0000", 2024-09-17T22:42:40.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5000000000000001e-07 2024-09-17T22:42:40.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241711+0000", 2024-09-17T22:42:40.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9670000000000001e-06 2024-09-17T22:42:40.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.410 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241713+0000", 2024-09-17T22:42:40.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2689999999999998e-06 2024-09-17T22:42:40.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.410 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281278+0000", 2024-09-17T22:42:40.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039565206999999998 2024-09-17T22:42:40.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281283+0000", 2024-09-17T22:42:40.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:42:40.411 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.411 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.411 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.411 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158700 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.412 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:40.412 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.107001379000003, 2024-09-17T22:42:40.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039604292999999999, 2024-09-17T22:42:40.412 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.412 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.412 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.412 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.412 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.412 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.412 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158700 2024-09-17T22:42:40.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.412 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:40.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:40.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:40.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:40.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241717+0000", 2024-09-17T22:42:40.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.78e-07 2024-09-17T22:42:40.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241720+0000", 2024-09-17T22:42:40.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8119999999999999e-06 2024-09-17T22:42:40.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241722+0000", 2024-09-17T22:42:40.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-17T22:42:40.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281311+0000", 2024-09-17T22:42:40.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039588894999999999 2024-09-17T22:42:40.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281320+0000", 2024-09-17T22:42:40.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9989999999999997e-06 2024-09-17T22:42:40.415 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.415 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.415 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.416 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158702 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.416 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:40.416 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.106990056999997, 2024-09-17T22:42:40.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039616142, 2024-09-17T22:42:40.416 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.416 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.416 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.416 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.416 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.416 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.416 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158702 2024-09-17T22:42:40.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.416 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:40.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:40.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:40.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4400000000000001e-07 2024-09-17T22:42:40.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241728+0000", 2024-09-17T22:42:40.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0839999999999999e-06 2024-09-17T22:42:40.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241731+0000", 2024-09-17T22:42:40.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0690000000000002e-06 2024-09-17T22:42:40.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241734+0000", 2024-09-17T22:42:40.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4159999999999998e-06 2024-09-17T22:42:40.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281334+0000", 2024-09-17T22:42:40.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039600110000000001 2024-09-17T22:42:40.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281343+0000", 2024-09-17T22:42:40.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2189999999999993e-06 2024-09-17T22:42:40.419 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.419 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.419 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.420 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158704 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.420 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:40.420 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.106980192999998, 2024-09-17T22:42:40.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039641485999999997, 2024-09-17T22:42:40.420 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.420 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.420 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.420 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.420 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.420 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.420 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158704 2024-09-17T22:42:40.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.420 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:40.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:40.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.421 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.421 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.421 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:40.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:42:40.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241738+0000", 2024-09-17T22:42:40.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0589999999999999e-06 2024-09-17T22:42:40.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241741+0000", 2024-09-17T22:42:40.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.943e-06 2024-09-17T22:42:40.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241744+0000", 2024-09-17T22:42:40.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4819999999999999e-06 2024-09-17T22:42:40.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281370+0000", 2024-09-17T22:42:40.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039625661 2024-09-17T22:42:40.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281378+0000", 2024-09-17T22:42:40.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1310000000000006e-06 2024-09-17T22:42:40.423 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.423 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.423 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.424 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158705 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.424 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:40.424 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.106969642999999, 2024-09-17T22:42:40.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039652637999999997, 2024-09-17T22:42:40.424 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.424 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.424 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.424 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.424 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.424 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.424 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158705 2024-09-17T22:42:40.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.424 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:40.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:40.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:42:40.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-07 2024-09-17T22:42:40.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:42:40.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0599999999999999e-07 2024-09-17T22:42:40.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241751+0000", 2024-09-17T22:42:40.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.926e-06 2024-09-17T22:42:40.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241754+0000", 2024-09-17T22:42:40.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4360000000000001e-06 2024-09-17T22:42:40.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281392+0000", 2024-09-17T22:42:40.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039638566 2024-09-17T22:42:40.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.427 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.427 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281400+0000", 2024-09-17T22:42:40.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.571e-06 2024-09-17T22:42:40.427 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.427 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.427 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.428 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158707 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.428 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:40.428 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.106948541999998, 2024-09-17T22:42:40.428 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039654809999999999, 2024-09-17T22:42:40.428 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.428 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.428 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.428 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158707 2024-09-17T22:42:40.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.429 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:40.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:40.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241769+0000", 2024-09-17T22:42:40.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4299999999999999e-07 2024-09-17T22:42:40.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241770+0000", 2024-09-17T22:42:40.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3769999999999999e-06 2024-09-17T22:42:40.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241773+0000", 2024-09-17T22:42:40.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2109999999999998e-06 2024-09-17T22:42:40.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241776+0000", 2024-09-17T22:42:40.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5069999999999999e-06 2024-09-17T22:42:40.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281415+0000", 2024-09-17T22:42:40.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039639499000000002 2024-09-17T22:42:40.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281423+0000", 2024-09-17T22:42:40.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8730000000000007e-06 2024-09-17T22:42:40.431 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.432 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.432 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.432 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158709 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.432 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:40.432 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.106937932999998, 2024-09-17T22:42:40.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039661937000000001, 2024-09-17T22:42:40.432 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.432 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.432 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:40.432 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.432 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.432 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.433 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158709 2024-09-17T22:42:40.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.433 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:40.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:40.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242843+0000", 2024-09-17T22:42:40.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010634780000000001 2024-09-17T22:42:40.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242845+0000", 2024-09-17T22:42:40.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7209999999999999e-06 2024-09-17T22:42:40.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.434 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.434 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242851+0000", 2024-09-17T22:42:40.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6949999999999997e-06 2024-09-17T22:42:40.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242855+0000", 2024-09-17T22:42:40.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7960000000000002e-06 2024-09-17T22:42:40.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281434+0000", 2024-09-17T22:42:40.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038579519 2024-09-17T22:42:40.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281441+0000", 2024-09-17T22:42:40.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7279999999999998e-06 2024-09-17T22:42:40.435 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.435 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.435 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.436 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.436 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.436 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:40.438 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:42:40.438 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:40.435+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:40.538 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:40.536+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:40.580 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:40.580 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:40.580 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:40.580 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:40.589 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:42:40.638 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:40.636+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:40.692 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:40.692 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:40.692 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:40.692 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:40.692 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:40.701 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:42:40.742 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:40.740+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:40.840 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:40.838+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:40.845 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:40.845 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:40.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:40.845 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:40.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.845 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157229 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4073472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.845 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:40.846 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.696048222000002, 2024-09-17T22:42:40.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041957959000000003, 2024-09-17T22:42:40.846 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.846 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.846 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.846 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.846 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.846 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.846 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157229 2024-09-17T22:42:40.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.846 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:40.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:40.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146207+0000", 2024-09-17T22:42:40.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0299999999999999e-07 2024-09-17T22:42:40.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146215+0000", 2024-09-17T22:42:40.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3059999999999998e-06 2024-09-17T22:42:40.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146222+0000", 2024-09-17T22:42:40.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1180000000000003e-06 2024-09-17T22:42:40.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146227+0000", 2024-09-17T22:42:40.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1749999999999998e-06 2024-09-17T22:42:40.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.187985+0000", 2024-09-17T22:42:40.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041758391999999998 2024-09-17T22:42:40.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188065+0000", 2024-09-17T22:42:40.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0290000000000005e-05 2024-09-17T22:42:40.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188164+0000", 2024-09-17T22:42:40.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9074999999999996e-05 2024-09-17T22:42:40.850 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.850 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.850 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.850 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157231 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4079616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.850 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:40.850 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.695910597999998, 2024-09-17T22:42:40.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042034581000000001, 2024-09-17T22:42:40.851 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.851 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.851 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.851 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.851 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.851 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.851 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157231 2024-09-17T22:42:40.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.851 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:40.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:40.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146345+0000", 2024-09-17T22:42:40.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:42:40.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146347+0000", 2024-09-17T22:42:40.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.357e-06 2024-09-17T22:42:40.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146352+0000", 2024-09-17T22:42:40.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T22:42:40.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146361+0000", 2024-09-17T22:42:40.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0219999999999993e-06 2024-09-17T22:42:40.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188199+0000", 2024-09-17T22:42:40.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041838130000000001 2024-09-17T22:42:40.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188285+0000", 2024-09-17T22:42:40.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5160000000000005e-05 2024-09-17T22:42:40.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188379+0000", 2024-09-17T22:42:40.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3949000000000005e-05 2024-09-17T22:42:40.854 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.855 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.855 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.855 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157233 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4085760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.855 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:40.855 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.695694449000001, 2024-09-17T22:42:40.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042008960999999997, 2024-09-17T22:42:40.855 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.855 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.855 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.855 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.855 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.855 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.856 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157233 2024-09-17T22:42:40.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.856 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:40.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:40.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146561+0000", 2024-09-17T22:42:40.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6300000000000005e-07 2024-09-17T22:42:40.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146564+0000", 2024-09-17T22:42:40.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-17T22:42:40.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146575+0000", 2024-09-17T22:42:40.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0891e-05 2024-09-17T22:42:40.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146579+0000", 2024-09-17T22:42:40.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.284e-06 2024-09-17T22:42:40.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188407+0000", 2024-09-17T22:42:40.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041828390999999999 2024-09-17T22:42:40.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188476+0000", 2024-09-17T22:42:40.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9129999999999997e-05 2024-09-17T22:42:40.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188569+0000", 2024-09-17T22:42:40.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2669000000000003e-05 2024-09-17T22:42:40.859 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.859 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.859 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.859 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157235 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4091904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.860 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:40.860 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.695500451999997, 2024-09-17T22:42:40.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042047517, 2024-09-17T22:42:40.860 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.860 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.860 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.860 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.860 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.860 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.860 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157235 2024-09-17T22:42:40.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.860 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:40.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:40.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146755+0000", 2024-09-17T22:42:40.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0900000000000001e-07 2024-09-17T22:42:40.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146758+0000", 2024-09-17T22:42:40.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.399e-06 2024-09-17T22:42:40.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146768+0000", 2024-09-17T22:42:40.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4730000000000006e-06 2024-09-17T22:42:40.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146774+0000", 2024-09-17T22:42:40.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6939999999999998e-06 2024-09-17T22:42:40.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188616+0000", 2024-09-17T22:42:40.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041841481999999999 2024-09-17T22:42:40.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188700+0000", 2024-09-17T22:42:40.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4227999999999994e-05 2024-09-17T22:42:40.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188802+0000", 2024-09-17T22:42:40.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101532 2024-09-17T22:42:40.864 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.864 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.864 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.864 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157237 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4098048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.864 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:40.864 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.695470180999997, 2024-09-17T22:42:40.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165452999999999, 2024-09-17T22:42:40.864 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.864 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.865 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.865 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.865 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.865 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.865 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157237 2024-09-17T22:42:40.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.865 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:40.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:40.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146785+0000", 2024-09-17T22:42:40.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6800000000000001e-07 2024-09-17T22:42:40.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146787+0000", 2024-09-17T22:42:40.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.046e-06 2024-09-17T22:42:40.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146794+0000", 2024-09-17T22:42:40.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9249999999999998e-06 2024-09-17T22:42:40.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146799+0000", 2024-09-17T22:42:40.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9289999999999997e-06 2024-09-17T22:42:40.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188844+0000", 2024-09-17T22:42:40.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042045386999999997 2024-09-17T22:42:40.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188888+0000", 2024-09-17T22:42:40.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3760000000000001e-05 2024-09-17T22:42:40.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188950+0000", 2024-09-17T22:42:40.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1938000000000002e-05 2024-09-17T22:42:40.868 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.868 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.868 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.869 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157239 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4104192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.869 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:40.869 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.695441795000001, 2024-09-17T22:42:40.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042347193999999998, 2024-09-17T22:42:40.869 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.869 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.869 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.869 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.869 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.869 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.869 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157239 2024-09-17T22:42:40.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.869 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:40.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:40.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:40.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:42:40.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146816+0000", 2024-09-17T22:42:40.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4700000000000001e-06 2024-09-17T22:42:40.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146822+0000", 2024-09-17T22:42:40.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.579e-06 2024-09-17T22:42:40.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146826+0000", 2024-09-17T22:42:40.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3780000000000001e-06 2024-09-17T22:42:40.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188991+0000", 2024-09-17T22:42:40.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165553000000001 2024-09-17T22:42:40.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189068+0000", 2024-09-17T22:42:40.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6548e-05 2024-09-17T22:42:40.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189160+0000", 2024-09-17T22:42:40.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2163000000000005e-05 2024-09-17T22:42:40.873 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.873 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.873 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.873 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157242 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4110336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.873 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:40.873 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.695404009000001, 2024-09-17T22:42:40.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042499783999999999, 2024-09-17T22:42:40.873 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.874 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.874 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.874 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.874 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.874 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.874 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157242 2024-09-17T22:42:40.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.874 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:40.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:40.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:40.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:42:40.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146852+0000", 2024-09-17T22:42:40.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.37e-06 2024-09-17T22:42:40.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146856+0000", 2024-09-17T22:42:40.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1690000000000002e-06 2024-09-17T22:42:40.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146863+0000", 2024-09-17T22:42:40.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9920000000000001e-06 2024-09-17T22:42:40.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189193+0000", 2024-09-17T22:42:40.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042329355999999999 2024-09-17T22:42:40.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189246+0000", 2024-09-17T22:42:40.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3643000000000001e-05 2024-09-17T22:42:40.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189350+0000", 2024-09-17T22:42:40.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010394 2024-09-17T22:42:40.878 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.878 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.878 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.878 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157244 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4116480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.878 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:40.878 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.695384797999999, 2024-09-17T22:42:40.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042675684999999998, 2024-09-17T22:42:40.878 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.878 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.878 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.878 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.878 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.879 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.879 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157244 2024-09-17T22:42:40.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.879 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:40.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:40.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:40.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1e-07 2024-09-17T22:42:40.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146871+0000", 2024-09-17T22:42:40.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.189e-06 2024-09-17T22:42:40.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146877+0000", 2024-09-17T22:42:40.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8100000000000003e-06 2024-09-17T22:42:40.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146880+0000", 2024-09-17T22:42:40.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.83e-06 2024-09-17T22:42:40.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189388+0000", 2024-09-17T22:42:40.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042507637000000001 2024-09-17T22:42:40.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189443+0000", 2024-09-17T22:42:40.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5742999999999998e-05 2024-09-17T22:42:40.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189545+0000", 2024-09-17T22:42:40.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102166 2024-09-17T22:42:40.882 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.882 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.882 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.882 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157246 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4122624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.882 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:40.882 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.695355745999997, 2024-09-17T22:42:40.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042847083000000001, 2024-09-17T22:42:40.883 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.883 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.883 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.883 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.883 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.883 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.883 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157246 2024-09-17T22:42:40.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.883 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:40.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:40.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:40.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0100000000000001e-07 2024-09-17T22:42:40.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146901+0000", 2024-09-17T22:42:40.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.531e-06 2024-09-17T22:42:40.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146905+0000", 2024-09-17T22:42:40.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0759999999999996e-06 2024-09-17T22:42:40.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146908+0000", 2024-09-17T22:42:40.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.083e-06 2024-09-17T22:42:40.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189585+0000", 2024-09-17T22:42:40.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042677009000000002 2024-09-17T22:42:40.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189656+0000", 2024-09-17T22:42:40.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0712999999999999e-05 2024-09-17T22:42:40.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189746+0000", 2024-09-17T22:42:40.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0370000000000004e-05 2024-09-17T22:42:40.887 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.887 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.887 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.887 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157247 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4128768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.887 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:40.887 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.695283910000001, 2024-09-17T22:42:40.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043016916000000002, 2024-09-17T22:42:40.887 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.887 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.887 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.888 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157247 2024-09-17T22:42:40.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.888 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:40.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:40.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147071+0000", 2024-09-17T22:42:40.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100769 2024-09-17T22:42:40.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147081+0000", 2024-09-17T22:42:40.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9229999999999997e-06 2024-09-17T22:42:40.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147090+0000", 2024-09-17T22:42:40.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.935e-06 2024-09-17T22:42:40.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147097+0000", 2024-09-17T22:42:40.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T22:42:40.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189787+0000", 2024-09-17T22:42:40.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042690525 2024-09-17T22:42:40.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189884+0000", 2024-09-17T22:42:40.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6858e-05 2024-09-17T22:42:40.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189988+0000", 2024-09-17T22:42:40.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103336 2024-09-17T22:42:40.891 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.891 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.891 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.891 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157250 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4134912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.892 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:40.892 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.695140449, 2024-09-17T22:42:40.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043099446, 2024-09-17T22:42:40.892 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.892 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.892 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.892 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157250 2024-09-17T22:42:40.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.892 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:40.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:40.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147235+0000", 2024-09-17T22:42:40.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120975 2024-09-17T22:42:40.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147238+0000", 2024-09-17T22:42:40.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5289999999999998e-06 2024-09-17T22:42:40.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147244+0000", 2024-09-17T22:42:40.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6050000000000003e-06 2024-09-17T22:42:40.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147249+0000", 2024-09-17T22:42:40.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4939999999999997e-06 2024-09-17T22:42:40.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190020+0000", 2024-09-17T22:42:40.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042771300999999998 2024-09-17T22:42:40.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190102+0000", 2024-09-17T22:42:40.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1600000000000005e-05 2024-09-17T22:42:40.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190214+0000", 2024-09-17T22:42:40.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011194200000000001 2024-09-17T22:42:40.896 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.896 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.896 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.896 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157252 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4141056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.896 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:40.896 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.694997780000001, 2024-09-17T22:42:40.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043165508999999998, 2024-09-17T22:42:40.896 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.896 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.897 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.897 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.897 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.897 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.897 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157252 2024-09-17T22:42:40.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.897 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:40.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:40.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147402+0000", 2024-09-17T22:42:40.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014475600000000001 2024-09-17T22:42:40.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147411+0000", 2024-09-17T22:42:40.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4350000000000003e-06 2024-09-17T22:42:40.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147420+0000", 2024-09-17T22:42:40.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4790000000000002e-06 2024-09-17T22:42:40.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147427+0000", 2024-09-17T22:42:40.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5400000000000001e-06 2024-09-17T22:42:40.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190254+0000", 2024-09-17T22:42:40.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042826731 2024-09-17T22:42:40.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190326+0000", 2024-09-17T22:42:40.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1798999999999995e-05 2024-09-17T22:42:40.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190422+0000", 2024-09-17T22:42:40.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6768999999999995e-05 2024-09-17T22:42:40.900 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.900 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.900 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.901 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157253 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4147200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.901 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:40.901 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.694811414, 2024-09-17T22:42:40.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043157752000000001, 2024-09-17T22:42:40.901 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.901 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.901 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.901 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157253 2024-09-17T22:42:40.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.902 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:40.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:40.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147590+0000", 2024-09-17T22:42:40.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014709 2024-09-17T22:42:40.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147593+0000", 2024-09-17T22:42:40.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5840000000000002e-06 2024-09-17T22:42:40.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147598+0000", 2024-09-17T22:42:40.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2469999999999999e-06 2024-09-17T22:42:40.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147602+0000", 2024-09-17T22:42:40.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0049999999999998e-06 2024-09-17T22:42:40.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190475+0000", 2024-09-17T22:42:40.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872767999999999 2024-09-17T22:42:40.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190528+0000", 2024-09-17T22:42:40.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2889999999999997e-05 2024-09-17T22:42:40.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190601+0000", 2024-09-17T22:42:40.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3168000000000001e-05 2024-09-17T22:42:40.905 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.905 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.905 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.905 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157255 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4153344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.905 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:40.905 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.694645432999998, 2024-09-17T22:42:40.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043132251000000003, 2024-09-17T22:42:40.905 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.905 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.906 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.906 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.906 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.906 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.906 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157255 2024-09-17T22:42:40.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.906 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:40.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:40.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147820+0000", 2024-09-17T22:42:40.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021123600000000001 2024-09-17T22:42:40.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147824+0000", 2024-09-17T22:42:40.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6780000000000002e-06 2024-09-17T22:42:40.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147833+0000", 2024-09-17T22:42:40.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4250000000000004e-06 2024-09-17T22:42:40.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147842+0000", 2024-09-17T22:42:40.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0679999999999997e-06 2024-09-17T22:42:40.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190632+0000", 2024-09-17T22:42:40.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042790863999999998 2024-09-17T22:42:40.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190674+0000", 2024-09-17T22:42:40.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1804000000000002e-05 2024-09-17T22:42:40.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.909 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190741+0000", 2024-09-17T22:42:40.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7175999999999995e-05 2024-09-17T22:42:40.909 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.909 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.909 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.909 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157257 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4159488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.909 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:40.910 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.694404943000002, 2024-09-17T22:42:40.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.04307035, 2024-09-17T22:42:40.910 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.910 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.910 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.910 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157257 2024-09-17T22:42:40.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.910 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:40.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:40.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148844+0000", 2024-09-17T22:42:40.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00099455599999999996 2024-09-17T22:42:40.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148848+0000", 2024-09-17T22:42:40.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0180000000000003e-06 2024-09-17T22:42:40.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148858+0000", 2024-09-17T22:42:40.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.825e-06 2024-09-17T22:42:40.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148865+0000", 2024-09-17T22:42:40.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8290000000000002e-06 2024-09-17T22:42:40.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190789+0000", 2024-09-17T22:42:40.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041924630999999997 2024-09-17T22:42:40.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190860+0000", 2024-09-17T22:42:40.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0557999999999998e-05 2024-09-17T22:42:40.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.914 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.914 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190920+0000", 2024-09-17T22:42:40.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9932999999999997e-05 2024-09-17T22:42:40.914 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.914 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.914 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.915 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157259 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4165632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.915 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:40.915 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.693378924000001, 2024-09-17T22:42:40.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042261411999999998, 2024-09-17T22:42:40.915 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.915 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.915 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.915 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.915 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.915 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.916 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157259 2024-09-17T22:42:40.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.916 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:40.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:40.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:40.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:42:40.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148878+0000", 2024-09-17T22:42:40.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:42:40.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148885+0000", 2024-09-17T22:42:40.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7290000000000003e-06 2024-09-17T22:42:40.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148893+0000", 2024-09-17T22:42:40.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8110000000000003e-06 2024-09-17T22:42:40.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190967+0000", 2024-09-17T22:42:40.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042073948999999999 2024-09-17T22:42:40.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191069+0000", 2024-09-17T22:42:40.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102075 2024-09-17T22:42:40.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191137+0000", 2024-09-17T22:42:40.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8281000000000006e-05 2024-09-17T22:42:40.920 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.920 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.920 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.920 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157261 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4171776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.920 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:40.920 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.693329317, 2024-09-17T22:42:40.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042437255, 2024-09-17T22:42:40.920 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.921 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.921 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.921 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.921 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.921 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.921 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157261 2024-09-17T22:42:40.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.921 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:40.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:40.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148926+0000", 2024-09-17T22:42:40.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T22:42:40.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148928+0000", 2024-09-17T22:42:40.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4329999999999998e-06 2024-09-17T22:42:40.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148936+0000", 2024-09-17T22:42:40.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6969999999999993e-06 2024-09-17T22:42:40.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.923 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148942+0000", 2024-09-17T22:42:40.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9479999999999996e-06 2024-09-17T22:42:40.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191185+0000", 2024-09-17T22:42:40.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042243147000000002 2024-09-17T22:42:40.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191266+0000", 2024-09-17T22:42:40.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1186000000000005e-05 2024-09-17T22:42:40.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191362+0000", 2024-09-17T22:42:40.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6384000000000004e-05 2024-09-17T22:42:40.925 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.925 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.925 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.925 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157263 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4177920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.926 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:40.926 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.693304918000003, 2024-09-17T22:42:40.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042636395000000001, 2024-09-17T22:42:40.926 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.926 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.926 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.926 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.926 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.926 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.926 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157263 2024-09-17T22:42:40.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.926 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:40.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:40.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:40.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.37e-07 2024-09-17T22:42:40.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148952+0000", 2024-09-17T22:42:40.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.88e-06 2024-09-17T22:42:40.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.928 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.928 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148959+0000", 2024-09-17T22:42:40.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5649999999999998e-06 2024-09-17T22:42:40.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148963+0000", 2024-09-17T22:42:40.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8570000000000003e-06 2024-09-17T22:42:40.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191408+0000", 2024-09-17T22:42:40.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042444707999999998 2024-09-17T22:42:40.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191489+0000", 2024-09-17T22:42:40.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0612999999999995e-05 2024-09-17T22:42:40.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191586+0000", 2024-09-17T22:42:40.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7235e-05 2024-09-17T22:42:40.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.930 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.930 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.931 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157265 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4184064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.931 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:40.931 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.693283369, 2024-09-17T22:42:40.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042791830000000003, 2024-09-17T22:42:40.931 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.931 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.931 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.931 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.932 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157265 2024-09-17T22:42:40.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.932 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:40.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:40.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148972+0000", 2024-09-17T22:42:40.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:42:40.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.933 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.933 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.933 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148973+0000", 2024-09-17T22:42:40.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.894e-06 2024-09-17T22:42:40.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148985+0000", 2024-09-17T22:42:40.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1995e-05 2024-09-17T22:42:40.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148990+0000", 2024-09-17T22:42:40.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8269999999999997e-06 2024-09-17T22:42:40.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191630+0000", 2024-09-17T22:42:40.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042639639 2024-09-17T22:42:40.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191686+0000", 2024-09-17T22:42:40.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6066000000000002e-05 2024-09-17T22:42:40.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191763+0000", 2024-09-17T22:42:40.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7035999999999999e-05 2024-09-17T22:42:40.936 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.936 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.936 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.936 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157267 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4190208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:40.936 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:40.936 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 50.693257555999999, 2024-09-17T22:42:40.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043028813999999999, 2024-09-17T22:42:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157267 2024-09-17T22:42:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:42:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148999+0000", 2024-09-17T22:42:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.432e-06 2024-09-17T22:42:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149002+0000", 2024-09-17T22:42:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7850000000000002e-06 2024-09-17T22:42:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149005+0000", 2024-09-17T22:42:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:42:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191878+0000", 2024-09-17T22:42:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872312000000003 2024-09-17T22:42:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191961+0000", 2024-09-17T22:42:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3418000000000007e-05 2024-09-17T22:42:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.192026+0000", 2024-09-17T22:42:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4689000000000004e-05 2024-09-17T22:42:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:40.941 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:40.943 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:42:40.944 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:40.941+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:41.044 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:41.042+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:41.047 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:41.047 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:41.047 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:41.047 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:41.056 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:42:41.145 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:41.142+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:41.200 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:42:41.200 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:41.201 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:41.201 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:41.201 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:41.201 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:41.209 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:42:41.252 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:41.250+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:41.310 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:41.314 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:41.314 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:41.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:41.314 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:41.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.315 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188693 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5881856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:41.315 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:41.315 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.484317486999998, 2024-09-17T22:42:41.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028193223, 2024-09-17T22:42:41.315 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.315 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.315 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.315 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.315 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.315 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.315 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188693 2024-09-17T22:42:41.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.316 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:41.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:41.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:41.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:42:41.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827090+0000", 2024-09-17T22:42:41.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2170000000000001e-06 2024-09-17T22:42:41.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827098+0000", 2024-09-17T22:42:41.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1510000000000004e-06 2024-09-17T22:42:41.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827103+0000", 2024-09-17T22:42:41.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0839999999999999e-06 2024-09-17T22:42:41.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855158+0000", 2024-09-17T22:42:41.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028055217 2024-09-17T22:42:41.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855201+0000", 2024-09-17T22:42:41.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3272000000000002e-05 2024-09-17T22:42:41.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855280+0000", 2024-09-17T22:42:41.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8845999999999997e-05 2024-09-17T22:42:41.319 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.319 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.319 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.319 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188694 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5971968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:41.319 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:41.319 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.484293600000001, 2024-09-17T22:42:41.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028312278, 2024-09-17T22:42:41.320 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.320 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.320 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.320 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.320 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.320 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.320 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188694 2024-09-17T22:42:41.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.320 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:41.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:41.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:41.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:42:41.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827114+0000", 2024-09-17T22:42:41.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3029999999999998e-06 2024-09-17T22:42:41.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827121+0000", 2024-09-17T22:42:41.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3699999999999997e-06 2024-09-17T22:42:41.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827126+0000", 2024-09-17T22:42:41.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7350000000000004e-06 2024-09-17T22:42:41.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855307+0000", 2024-09-17T22:42:41.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028180851999999999 2024-09-17T22:42:41.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855346+0000", 2024-09-17T22:42:41.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9274999999999999e-05 2024-09-17T22:42:41.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855423+0000", 2024-09-17T22:42:41.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7294999999999998e-05 2024-09-17T22:42:41.323 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.323 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.323 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.324 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188695 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:41.324 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:41.324 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.484272822000001, 2024-09-17T22:42:41.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028435876999999998, 2024-09-17T22:42:41.324 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.324 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.324 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.324 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.324 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.324 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.324 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188695 2024-09-17T22:42:41.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.325 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:41.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:41.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:41.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:42:41.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827138+0000", 2024-09-17T22:42:41.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2580000000000001e-06 2024-09-17T22:42:41.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827145+0000", 2024-09-17T22:42:41.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0129999999999996e-06 2024-09-17T22:42:41.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827150+0000", 2024-09-17T22:42:41.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3749999999999996e-06 2024-09-17T22:42:41.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855447+0000", 2024-09-17T22:42:41.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028297224999999999 2024-09-17T22:42:41.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855487+0000", 2024-09-17T22:42:41.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0454000000000003e-05 2024-09-17T22:42:41.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855567+0000", 2024-09-17T22:42:41.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0149999999999997e-05 2024-09-17T22:42:41.328 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.328 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.328 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.328 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188696 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:41.328 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:41.328 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.484249038000002, 2024-09-17T22:42:41.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028543570000000001, 2024-09-17T22:42:41.328 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.328 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.328 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.328 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.329 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.329 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.329 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188696 2024-09-17T22:42:41.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.329 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:41.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:41.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827156+0000", 2024-09-17T22:42:41.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T22:42:41.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827158+0000", 2024-09-17T22:42:41.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.875e-06 2024-09-17T22:42:41.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827164+0000", 2024-09-17T22:42:41.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7950000000000002e-06 2024-09-17T22:42:41.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827169+0000", 2024-09-17T22:42:41.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2880000000000003e-06 2024-09-17T22:42:41.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855582+0000", 2024-09-17T22:42:41.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028412929 2024-09-17T22:42:41.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855624+0000", 2024-09-17T22:42:41.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2484000000000002e-05 2024-09-17T22:42:41.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855699+0000", 2024-09-17T22:42:41.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4869000000000004e-05 2024-09-17T22:42:41.332 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.332 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.332 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.332 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188697 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6242304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:41.332 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:41.332 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 32.484217508999997, 2024-09-17T22:42:41.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028658598, 2024-09-17T22:42:41.332 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.333 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.333 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.333 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.333 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.333 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.333 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188697 2024-09-17T22:42:41.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.333 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:41.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:41.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:41.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8800000000000003e-07 2024-09-17T22:42:41.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827190+0000", 2024-09-17T22:42:41.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.351e-06 2024-09-17T22:42:41.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827198+0000", 2024-09-17T22:42:41.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.909e-06 2024-09-17T22:42:41.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827203+0000", 2024-09-17T22:42:41.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4020000000000001e-06 2024-09-17T22:42:41.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855720+0000", 2024-09-17T22:42:41.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028516909999999999 2024-09-17T22:42:41.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855762+0000", 2024-09-17T22:42:41.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2184999999999997e-05 2024-09-17T22:42:41.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855845+0000", 2024-09-17T22:42:41.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3353e-05 2024-09-17T22:42:41.336 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.336 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.336 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.337 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202883 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2342912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:41.337 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:42:41.337 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.586581127999999, 2024-09-17T22:42:41.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028098636, 2024-09-17T22:42:41.337 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.337 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.337 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.337 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.337 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.337 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.337 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202883 2024-09-17T22:42:41.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.337 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:42:41.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:42:41.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724824+0000", 2024-09-17T22:42:41.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:42:41.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724826+0000", 2024-09-17T22:42:41.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.086e-06 2024-09-17T22:42:41.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724832+0000", 2024-09-17T22:42:41.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9100000000000002e-06 2024-09-17T22:42:41.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724837+0000", 2024-09-17T22:42:41.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9439999999999996e-06 2024-09-17T22:42:41.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752790+0000", 2024-09-17T22:42:41.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027953657999999999 2024-09-17T22:42:41.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752848+0000", 2024-09-17T22:42:41.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7677999999999999e-05 2024-09-17T22:42:41.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752922+0000", 2024-09-17T22:42:41.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3957999999999999e-05 2024-09-17T22:42:41.340 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.341 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.341 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.341 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202884 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2473984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:41.341 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:42:41.341 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.586552903000001, 2024-09-17T22:42:41.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028206208999999999, 2024-09-17T22:42:41.341 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.341 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.341 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.341 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.341 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.341 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.341 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202884 2024-09-17T22:42:41.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.342 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:42:41.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:42:41.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724852+0000", 2024-09-17T22:42:41.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2599999999999998e-07 2024-09-17T22:42:41.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724854+0000", 2024-09-17T22:42:41.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.26e-06 2024-09-17T22:42:41.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724871+0000", 2024-09-17T22:42:41.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6589999999999999e-05 2024-09-17T22:42:41.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724877+0000", 2024-09-17T22:42:41.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.054e-06 2024-09-17T22:42:41.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752951+0000", 2024-09-17T22:42:41.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028073747 2024-09-17T22:42:41.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752993+0000", 2024-09-17T22:42:41.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2082000000000001e-05 2024-09-17T22:42:41.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753058+0000", 2024-09-17T22:42:41.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5049999999999996e-05 2024-09-17T22:42:41.345 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.345 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.345 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.345 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202885 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:41.345 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:41.345 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.586519599999999, 2024-09-17T22:42:41.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028333028, 2024-09-17T22:42:41.345 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.345 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.346 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.346 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202885 2024-09-17T22:42:41.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.346 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:41.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:41.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:41.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3300000000000002e-07 2024-09-17T22:42:41.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724892+0000", 2024-09-17T22:42:41.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2779999999999999e-06 2024-09-17T22:42:41.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724899+0000", 2024-09-17T22:42:41.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9110000000000004e-06 2024-09-17T22:42:41.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724904+0000", 2024-09-17T22:42:41.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2360000000000004e-06 2024-09-17T22:42:41.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753094+0000", 2024-09-17T22:42:41.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02819033 2024-09-17T22:42:41.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753142+0000", 2024-09-17T22:42:41.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.829e-05 2024-09-17T22:42:41.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753218+0000", 2024-09-17T22:42:41.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5450000000000004e-05 2024-09-17T22:42:41.349 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.349 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.349 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.349 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202886 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:41.349 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:41.349 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.586494402, 2024-09-17T22:42:41.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028442512, 2024-09-17T22:42:41.350 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.350 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.350 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.350 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202886 2024-09-17T22:42:41.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.350 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:41.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:41.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:41.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:42:41.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724912+0000", 2024-09-17T22:42:41.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.967e-06 2024-09-17T22:42:41.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724918+0000", 2024-09-17T22:42:41.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8069999999999996e-06 2024-09-17T22:42:41.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724923+0000", 2024-09-17T22:42:41.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.386e-06 2024-09-17T22:42:41.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753244+0000", 2024-09-17T22:42:41.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028321567999999998 2024-09-17T22:42:41.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753289+0000", 2024-09-17T22:42:41.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4620000000000003e-05 2024-09-17T22:42:41.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753353+0000", 2024-09-17T22:42:41.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3795999999999997e-05 2024-09-17T22:42:41.353 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.353 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.354 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202887 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:41.354 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:41.354 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.586476488999999, 2024-09-17T22:42:41.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028565838999999999, 2024-09-17T22:42:41.354 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.354 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.354 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.354 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.354 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.354 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.354 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202887 2024-09-17T22:42:41.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.355 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:41.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:41.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:41.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:42:41.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724930+0000", 2024-09-17T22:42:41.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8980000000000001e-06 2024-09-17T22:42:41.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724936+0000", 2024-09-17T22:42:41.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7849999999999997e-06 2024-09-17T22:42:41.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724942+0000", 2024-09-17T22:42:41.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9649999999999996e-06 2024-09-17T22:42:41.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753380+0000", 2024-09-17T22:42:41.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028438049 2024-09-17T22:42:41.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753425+0000", 2024-09-17T22:42:41.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5163000000000001e-05 2024-09-17T22:42:41.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753494+0000", 2024-09-17T22:42:41.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8588000000000001e-05 2024-09-17T22:42:41.358 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.358 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.358 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.358 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202888 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:41.358 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:42:41.360 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.586456990999999, 2024-09-17T22:42:41.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028703314000000001, 2024-09-17T22:42:41.360 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.360 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.360 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.360 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.360 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.360 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.360 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202888 2024-09-17T22:42:41.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.360 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:42:41.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:42:41.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724948+0000", 2024-09-17T22:42:41.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:42:41.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724950+0000", 2024-09-17T22:42:41.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8029999999999999e-06 2024-09-17T22:42:41.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724955+0000", 2024-09-17T22:42:41.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8760000000000002e-06 2024-09-17T22:42:41.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724960+0000", 2024-09-17T22:42:41.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5979999999999999e-06 2024-09-17T22:42:41.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753527+0000", 2024-09-17T22:42:41.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028566864000000001 2024-09-17T22:42:41.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753568+0000", 2024-09-17T22:42:41.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0775000000000002e-05 2024-09-17T22:42:41.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753651+0000", 2024-09-17T22:42:41.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3039000000000002e-05 2024-09-17T22:42:41.363 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.364 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.364 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.364 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202889 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:41.364 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:41.364 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.586439419000001, 2024-09-17T22:42:41.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028820964000000001, 2024-09-17T22:42:41.364 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.364 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.364 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.364 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.364 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.365 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.365 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202889 2024-09-17T22:42:41.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.365 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:41.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:41.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:41.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2599999999999998e-07 2024-09-17T22:42:41.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724967+0000", 2024-09-17T22:42:41.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T22:42:41.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724973+0000", 2024-09-17T22:42:41.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6060000000000002e-06 2024-09-17T22:42:41.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724978+0000", 2024-09-17T22:42:41.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8230000000000002e-06 2024-09-17T22:42:41.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753687+0000", 2024-09-17T22:42:41.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028709495000000002 2024-09-17T22:42:41.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753727+0000", 2024-09-17T22:42:41.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9953999999999997e-05 2024-09-17T22:42:41.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753786+0000", 2024-09-17T22:42:41.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8941999999999999e-05 2024-09-17T22:42:41.368 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.368 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.368 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.368 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202890 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:41.368 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:41.368 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.586421851000001, 2024-09-17T22:42:41.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028933838, 2024-09-17T22:42:41.369 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.369 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.369 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.369 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.369 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.369 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.369 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202890 2024-09-17T22:42:41.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.369 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:41.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:41.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:41.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5999999999999999e-07 2024-09-17T22:42:41.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724985+0000", 2024-09-17T22:42:41.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8169999999999999e-06 2024-09-17T22:42:41.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724991+0000", 2024-09-17T22:42:41.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:42:41.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724995+0000", 2024-09-17T22:42:41.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4530000000000004e-06 2024-09-17T22:42:41.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753811+0000", 2024-09-17T22:42:41.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028815584000000002 2024-09-17T22:42:41.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753854+0000", 2024-09-17T22:42:41.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3356999999999998e-05 2024-09-17T22:42:41.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753916+0000", 2024-09-17T22:42:41.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2488000000000002e-05 2024-09-17T22:42:41.372 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.373 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.373 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.373 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202891 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:41.373 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:41.373 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.586404393999999, 2024-09-17T22:42:41.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029067554999999998, 2024-09-17T22:42:41.373 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.373 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.373 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.373 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.373 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.373 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.374 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202891 2024-09-17T22:42:41.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.374 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:41.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:41.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:41.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:42:41.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725002+0000", 2024-09-17T22:42:41.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:42:41.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725008+0000", 2024-09-17T22:42:41.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6629999999999998e-06 2024-09-17T22:42:41.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725016+0000", 2024-09-17T22:42:41.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8269999999999998e-06 2024-09-17T22:42:41.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753949+0000", 2024-09-17T22:42:41.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028932625999999999 2024-09-17T22:42:41.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753994+0000", 2024-09-17T22:42:41.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5717000000000003e-05 2024-09-17T22:42:41.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754068+0000", 2024-09-17T22:42:41.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3312000000000006e-05 2024-09-17T22:42:41.377 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.377 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.377 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.377 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202892 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:41.377 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:41.378 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.586361687, 2024-09-17T22:42:41.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029159939999999999, 2024-09-17T22:42:41.378 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.378 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.378 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.378 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202892 2024-09-17T22:42:41.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.378 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:41.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:41.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:41.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7999999999999995e-07 2024-09-17T22:42:41.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725045+0000", 2024-09-17T22:42:41.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-17T22:42:41.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725052+0000", 2024-09-17T22:42:41.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3629999999999999e-06 2024-09-17T22:42:41.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725056+0000", 2024-09-17T22:42:41.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3250000000000001e-06 2024-09-17T22:42:41.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754094+0000", 2024-09-17T22:42:41.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029038270000000001 2024-09-17T22:42:41.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754134+0000", 2024-09-17T22:42:41.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9722999999999999e-05 2024-09-17T22:42:41.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754203+0000", 2024-09-17T22:42:41.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8524999999999999e-05 2024-09-17T22:42:41.382 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.382 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.382 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.382 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202893 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:41.382 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:42:41.382 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.58634211, 2024-09-17T22:42:41.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029341651, 2024-09-17T22:42:41.382 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.382 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.382 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.382 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.382 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.383 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202893 2024-09-17T22:42:41.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.383 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:42:41.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:42:41.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725063+0000", 2024-09-17T22:42:41.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6699999999999999e-07 2024-09-17T22:42:41.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725065+0000", 2024-09-17T22:42:41.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2120000000000002e-06 2024-09-17T22:42:41.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725071+0000", 2024-09-17T22:42:41.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5169999999999996e-06 2024-09-17T22:42:41.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725076+0000", 2024-09-17T22:42:41.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.78e-06 2024-09-17T22:42:41.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754233+0000", 2024-09-17T22:42:41.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029156277000000001 2024-09-17T22:42:41.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754298+0000", 2024-09-17T22:42:41.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5320000000000005e-05 2024-09-17T22:42:41.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754404+0000", 2024-09-17T22:42:41.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106078 2024-09-17T22:42:41.386 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.386 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.386 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.386 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202894 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:41.386 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:42:41.386 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.586322231, 2024-09-17T22:42:41.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029456572, 2024-09-17T22:42:41.387 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.387 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.387 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.387 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.387 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.387 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.387 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202894 2024-09-17T22:42:41.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.387 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:42:41.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:42:41.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725083+0000", 2024-09-17T22:42:41.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5900000000000002e-07 2024-09-17T22:42:41.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725085+0000", 2024-09-17T22:42:41.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.979e-06 2024-09-17T22:42:41.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725091+0000", 2024-09-17T22:42:41.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1480000000000003e-06 2024-09-17T22:42:41.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725095+0000", 2024-09-17T22:42:41.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1339999999999997e-06 2024-09-17T22:42:41.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754429+0000", 2024-09-17T22:42:41.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029333758000000001 2024-09-17T22:42:41.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754471+0000", 2024-09-17T22:42:41.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2274999999999998e-05 2024-09-17T22:42:41.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754539+0000", 2024-09-17T22:42:41.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7818999999999996e-05 2024-09-17T22:42:41.390 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.390 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.391 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.391 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202895 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:41.391 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:41.391 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.586303834999999, 2024-09-17T22:42:41.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029635578999999999, 2024-09-17T22:42:41.391 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.391 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.391 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.391 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.391 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.391 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.391 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202895 2024-09-17T22:42:41.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.392 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:41.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:41.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:41.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.08e-07 2024-09-17T22:42:41.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725107+0000", 2024-09-17T22:42:41.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9020000000000004e-06 2024-09-17T22:42:41.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725113+0000", 2024-09-17T22:42:41.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7400000000000001e-06 2024-09-17T22:42:41.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725117+0000", 2024-09-17T22:42:41.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:42:41.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754567+0000", 2024-09-17T22:42:41.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029449411000000002 2024-09-17T22:42:41.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754638+0000", 2024-09-17T22:42:41.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1432000000000003e-05 2024-09-17T22:42:41.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754736+0000", 2024-09-17T22:42:41.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8066000000000006e-05 2024-09-17T22:42:41.395 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.395 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.395 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.395 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202896 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:41.395 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:42:41.395 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.586282103999999, 2024-09-17T22:42:41.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029760534000000002, 2024-09-17T22:42:41.395 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.395 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.395 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.396 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.396 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.396 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.396 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202896 2024-09-17T22:42:41.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.396 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:42:41.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:42:41.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725123+0000", 2024-09-17T22:42:41.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:42:41.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725125+0000", 2024-09-17T22:42:41.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.863e-06 2024-09-17T22:42:41.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725130+0000", 2024-09-17T22:42:41.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7250000000000002e-06 2024-09-17T22:42:41.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725135+0000", 2024-09-17T22:42:41.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:42:41.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754763+0000", 2024-09-17T22:42:41.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029628202999999999 2024-09-17T22:42:41.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754805+0000", 2024-09-17T22:42:41.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2370000000000003e-05 2024-09-17T22:42:41.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754883+0000", 2024-09-17T22:42:41.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7565999999999996e-05 2024-09-17T22:42:41.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.399 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.400 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202897 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:41.400 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:41.400 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 23.586264744000001, 2024-09-17T22:42:41.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029947972999999999, 2024-09-17T22:42:41.400 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:41.400 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:41.400 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:41.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:41.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:41.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:41.400 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202897 2024-09-17T22:42:41.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.401 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:41.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:41.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:41.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:41.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:41.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:41.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:41.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:41.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:42:41.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:41.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725146+0000", 2024-09-17T22:42:41.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6030000000000004e-06 2024-09-17T22:42:41.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:41.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725152+0000", 2024-09-17T22:42:41.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9970000000000004e-06 2024-09-17T22:42:41.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:41.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725156+0000", 2024-09-17T22:42:41.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-06 2024-09-17T22:42:41.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:41.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754909+0000", 2024-09-17T22:42:41.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029752958999999999 2024-09-17T22:42:41.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:41.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754987+0000", 2024-09-17T22:42:41.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8208999999999996e-05 2024-09-17T22:42:41.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:41.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:41.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:41.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.755088+0000", 2024-09-17T22:42:41.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100348 2024-09-17T22:42:41.404 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.404 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.404 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.404 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:41.404 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:41.404 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:41.404 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:42:41.404 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:42:41.405 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-17T22:42:41.405 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T22:42:41.405 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T22:42:41.405 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-17T22:42:41.406 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:41.406 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:41.407 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:41.407 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:41.408 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:42:41.408 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:41.408 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:41.409 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:41.409 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:41.409 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:41.409 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:41.409 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:41.403+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:41.420 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:41.420 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:41.439 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:41.440 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:41.443 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:41.444 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:41.507 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:41.505+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:41.555 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:41.556 INFO:teuthology.orchestra.run.smithi129.stdout: "success": "" 2024-09-17T22:42:41.556 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:41.556 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:41.556 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:41.566 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:42:41.609 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:41.607+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:41.668 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:41.669 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:41.677 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:42:41.718 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:41.716+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:41.810 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:41.808+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:41.820 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:41.820 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:41.829 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:42:41.911 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:41.908+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:41.934 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:41.934 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:41.943 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:42:42.011 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:42.009+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:42.045 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:42.045 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:42.053 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:42:42.112 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:42.110+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:42.157 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:42.157 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:42.166 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:42:42.213 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:42.211+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:42.269 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:42.269 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:42.278 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:42:42.315 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:42.312+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:42.384 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:42.384 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:42.394 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:42:42.416 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:42.413+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:42.438 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:42:42.514 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:42.514 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:42.516 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:42.514+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:42.523 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:42:42.617 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:42.614+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:42.631 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:42.631 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:42.641 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:42:42.724 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:42.722+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:42.783 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:42.783 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:42.793 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:42:42.835 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:42.832+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:42.920 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:42.917+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:42.936 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:42.937 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:42.946 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:42:43.021 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:43.019+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:43.052 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:43.052 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:43.061 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:42:43.144 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:43.142+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:43.204 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:43.204 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:43.213 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:42:43.254 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:43.252+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:43.323 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:43.321+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:43.357 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:43.358 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:43.366 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:42:43.367 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is set 2024-09-17T22:42:43.424 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:43.422+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:43.445 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:42:43.518 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:43.518 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:43.525 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:43.523+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:43.526 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:42:43.550 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:43.567 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:43.567 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:43.584 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:43.584 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:43.599 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:43.600 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:43.616 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:43.616 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:43.626 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:43.624+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:43.632 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:43.632 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:43.649 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:43.649 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:43.666 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:43.666 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:43.669 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:43.669 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:43.672 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:43.672 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:43.672 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:43.672 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:43.672 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:43.681 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:42:43.765 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:43.763+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:43.826 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:43.826 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:43.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:43.826 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:43.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.826 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157229 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4073472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.826 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:43.827 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.676539062000003, 2024-09-17T22:42:43.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041957959000000003, 2024-09-17T22:42:43.827 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.827 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.827 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.827 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.827 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.827 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.827 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157229 2024-09-17T22:42:43.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.827 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:43.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:43.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146207+0000", 2024-09-17T22:42:43.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0299999999999999e-07 2024-09-17T22:42:43.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146215+0000", 2024-09-17T22:42:43.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3059999999999998e-06 2024-09-17T22:42:43.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146222+0000", 2024-09-17T22:42:43.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1180000000000003e-06 2024-09-17T22:42:43.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146227+0000", 2024-09-17T22:42:43.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1749999999999998e-06 2024-09-17T22:42:43.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.187985+0000", 2024-09-17T22:42:43.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041758391999999998 2024-09-17T22:42:43.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188065+0000", 2024-09-17T22:42:43.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0290000000000005e-05 2024-09-17T22:42:43.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188164+0000", 2024-09-17T22:42:43.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9074999999999996e-05 2024-09-17T22:42:43.831 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.831 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.831 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.831 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157231 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4079616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.831 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:43.831 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.676401437999999, 2024-09-17T22:42:43.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042034581000000001, 2024-09-17T22:42:43.832 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.832 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.832 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.832 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.832 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.832 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.832 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157231 2024-09-17T22:42:43.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.832 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:43.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:43.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146345+0000", 2024-09-17T22:42:43.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:42:43.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146347+0000", 2024-09-17T22:42:43.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.357e-06 2024-09-17T22:42:43.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146352+0000", 2024-09-17T22:42:43.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T22:42:43.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146361+0000", 2024-09-17T22:42:43.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0219999999999993e-06 2024-09-17T22:42:43.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188199+0000", 2024-09-17T22:42:43.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041838130000000001 2024-09-17T22:42:43.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188285+0000", 2024-09-17T22:42:43.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5160000000000005e-05 2024-09-17T22:42:43.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188379+0000", 2024-09-17T22:42:43.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3949000000000005e-05 2024-09-17T22:42:43.836 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.836 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.836 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.836 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157233 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4085760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.836 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:43.836 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.676185289000003, 2024-09-17T22:42:43.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042008960999999997, 2024-09-17T22:42:43.836 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.836 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.836 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.836 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.837 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.837 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.837 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157233 2024-09-17T22:42:43.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.837 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:43.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:43.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146561+0000", 2024-09-17T22:42:43.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6300000000000005e-07 2024-09-17T22:42:43.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146564+0000", 2024-09-17T22:42:43.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-17T22:42:43.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146575+0000", 2024-09-17T22:42:43.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0891e-05 2024-09-17T22:42:43.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146579+0000", 2024-09-17T22:42:43.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.284e-06 2024-09-17T22:42:43.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188407+0000", 2024-09-17T22:42:43.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041828390999999999 2024-09-17T22:42:43.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188476+0000", 2024-09-17T22:42:43.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9129999999999997e-05 2024-09-17T22:42:43.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188569+0000", 2024-09-17T22:42:43.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2669000000000003e-05 2024-09-17T22:42:43.840 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.840 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.840 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.841 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157235 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4091904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.841 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:43.841 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.675991291999999, 2024-09-17T22:42:43.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042047517, 2024-09-17T22:42:43.841 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.841 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.841 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.842 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.842 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.842 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.842 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157235 2024-09-17T22:42:43.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.842 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:43.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:43.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146755+0000", 2024-09-17T22:42:43.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0900000000000001e-07 2024-09-17T22:42:43.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146758+0000", 2024-09-17T22:42:43.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.399e-06 2024-09-17T22:42:43.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146768+0000", 2024-09-17T22:42:43.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4730000000000006e-06 2024-09-17T22:42:43.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146774+0000", 2024-09-17T22:42:43.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6939999999999998e-06 2024-09-17T22:42:43.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188616+0000", 2024-09-17T22:42:43.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041841481999999999 2024-09-17T22:42:43.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188700+0000", 2024-09-17T22:42:43.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4227999999999994e-05 2024-09-17T22:42:43.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188802+0000", 2024-09-17T22:42:43.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101532 2024-09-17T22:42:43.846 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.846 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.846 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.846 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157237 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4098048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.846 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:43.846 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.675961020999999, 2024-09-17T22:42:43.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165452999999999, 2024-09-17T22:42:43.847 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.847 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.847 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.847 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.847 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.847 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.847 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157237 2024-09-17T22:42:43.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.847 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:43.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:43.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146785+0000", 2024-09-17T22:42:43.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6800000000000001e-07 2024-09-17T22:42:43.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146787+0000", 2024-09-17T22:42:43.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.046e-06 2024-09-17T22:42:43.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146794+0000", 2024-09-17T22:42:43.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9249999999999998e-06 2024-09-17T22:42:43.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146799+0000", 2024-09-17T22:42:43.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9289999999999997e-06 2024-09-17T22:42:43.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188844+0000", 2024-09-17T22:42:43.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042045386999999997 2024-09-17T22:42:43.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188888+0000", 2024-09-17T22:42:43.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3760000000000001e-05 2024-09-17T22:42:43.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188950+0000", 2024-09-17T22:42:43.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1938000000000002e-05 2024-09-17T22:42:43.851 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.851 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.851 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.851 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157239 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4104192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.851 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:43.851 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.675932635000002, 2024-09-17T22:42:43.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042347193999999998, 2024-09-17T22:42:43.851 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.851 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.851 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.851 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.852 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.852 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.852 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157239 2024-09-17T22:42:43.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.852 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:43.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:43.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:43.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:42:43.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146816+0000", 2024-09-17T22:42:43.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4700000000000001e-06 2024-09-17T22:42:43.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146822+0000", 2024-09-17T22:42:43.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.579e-06 2024-09-17T22:42:43.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146826+0000", 2024-09-17T22:42:43.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3780000000000001e-06 2024-09-17T22:42:43.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188991+0000", 2024-09-17T22:42:43.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165553000000001 2024-09-17T22:42:43.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189068+0000", 2024-09-17T22:42:43.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6548e-05 2024-09-17T22:42:43.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189160+0000", 2024-09-17T22:42:43.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2163000000000005e-05 2024-09-17T22:42:43.856 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.856 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.856 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.856 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157242 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4110336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.856 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:43.856 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.675894849000002, 2024-09-17T22:42:43.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042499783999999999, 2024-09-17T22:42:43.856 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.857 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.857 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.857 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.857 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.857 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.857 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157242 2024-09-17T22:42:43.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.857 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:43.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:43.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:43.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:42:43.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146852+0000", 2024-09-17T22:42:43.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.37e-06 2024-09-17T22:42:43.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146856+0000", 2024-09-17T22:42:43.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1690000000000002e-06 2024-09-17T22:42:43.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146863+0000", 2024-09-17T22:42:43.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9920000000000001e-06 2024-09-17T22:42:43.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189193+0000", 2024-09-17T22:42:43.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042329355999999999 2024-09-17T22:42:43.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189246+0000", 2024-09-17T22:42:43.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3643000000000001e-05 2024-09-17T22:42:43.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189350+0000", 2024-09-17T22:42:43.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010394 2024-09-17T22:42:43.861 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.861 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.861 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.861 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157244 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4116480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.862 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:43.862 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.675875638000001, 2024-09-17T22:42:43.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042675684999999998, 2024-09-17T22:42:43.862 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.862 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.862 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.862 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.862 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.862 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.862 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157244 2024-09-17T22:42:43.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.863 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:43.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:43.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:43.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1e-07 2024-09-17T22:42:43.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146871+0000", 2024-09-17T22:42:43.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.189e-06 2024-09-17T22:42:43.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146877+0000", 2024-09-17T22:42:43.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8100000000000003e-06 2024-09-17T22:42:43.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146880+0000", 2024-09-17T22:42:43.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.83e-06 2024-09-17T22:42:43.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189388+0000", 2024-09-17T22:42:43.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042507637000000001 2024-09-17T22:42:43.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189443+0000", 2024-09-17T22:42:43.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5742999999999998e-05 2024-09-17T22:42:43.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189545+0000", 2024-09-17T22:42:43.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102166 2024-09-17T22:42:43.867 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.867 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.867 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.867 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157246 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4122624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.867 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:43.867 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.675846585999999, 2024-09-17T22:42:43.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042847083000000001, 2024-09-17T22:42:43.868 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.868 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.868 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.868 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157246 2024-09-17T22:42:43.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.868 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:43.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:43.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0100000000000001e-07 2024-09-17T22:42:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146901+0000", 2024-09-17T22:42:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.531e-06 2024-09-17T22:42:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146905+0000", 2024-09-17T22:42:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0759999999999996e-06 2024-09-17T22:42:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146908+0000", 2024-09-17T22:42:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.083e-06 2024-09-17T22:42:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189585+0000", 2024-09-17T22:42:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042677009000000002 2024-09-17T22:42:43.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189656+0000", 2024-09-17T22:42:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0712999999999999e-05 2024-09-17T22:42:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189746+0000", 2024-09-17T22:42:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0370000000000004e-05 2024-09-17T22:42:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157247 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4128768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.675774750000002, 2024-09-17T22:42:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043016916000000002, 2024-09-17T22:42:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157247 2024-09-17T22:42:43.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147071+0000", 2024-09-17T22:42:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100769 2024-09-17T22:42:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147081+0000", 2024-09-17T22:42:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9229999999999997e-06 2024-09-17T22:42:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147090+0000", 2024-09-17T22:42:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.935e-06 2024-09-17T22:42:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147097+0000", 2024-09-17T22:42:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T22:42:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189787+0000", 2024-09-17T22:42:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042690525 2024-09-17T22:42:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189884+0000", 2024-09-17T22:42:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6858e-05 2024-09-17T22:42:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189988+0000", 2024-09-17T22:42:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103336 2024-09-17T22:42:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157250 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4134912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.675631289000002, 2024-09-17T22:42:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043099446, 2024-09-17T22:42:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.877 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157250 2024-09-17T22:42:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147235+0000", 2024-09-17T22:42:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120975 2024-09-17T22:42:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147238+0000", 2024-09-17T22:42:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5289999999999998e-06 2024-09-17T22:42:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147244+0000", 2024-09-17T22:42:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6050000000000003e-06 2024-09-17T22:42:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147249+0000", 2024-09-17T22:42:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4939999999999997e-06 2024-09-17T22:42:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190020+0000", 2024-09-17T22:42:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042771300999999998 2024-09-17T22:42:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190102+0000", 2024-09-17T22:42:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1600000000000005e-05 2024-09-17T22:42:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190214+0000", 2024-09-17T22:42:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011194200000000001 2024-09-17T22:42:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157252 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4141056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.881 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.675488620000003, 2024-09-17T22:42:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043165508999999998, 2024-09-17T22:42:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157252 2024-09-17T22:42:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147402+0000", 2024-09-17T22:42:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014475600000000001 2024-09-17T22:42:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147411+0000", 2024-09-17T22:42:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4350000000000003e-06 2024-09-17T22:42:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147420+0000", 2024-09-17T22:42:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4790000000000002e-06 2024-09-17T22:42:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147427+0000", 2024-09-17T22:42:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5400000000000001e-06 2024-09-17T22:42:43.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190254+0000", 2024-09-17T22:42:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042826731 2024-09-17T22:42:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190326+0000", 2024-09-17T22:42:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1798999999999995e-05 2024-09-17T22:42:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190422+0000", 2024-09-17T22:42:43.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6768999999999995e-05 2024-09-17T22:42:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157253 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4147200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.675302254000002, 2024-09-17T22:42:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043157752000000001, 2024-09-17T22:42:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.886 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157253 2024-09-17T22:42:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147590+0000", 2024-09-17T22:42:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014709 2024-09-17T22:42:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147593+0000", 2024-09-17T22:42:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5840000000000002e-06 2024-09-17T22:42:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147598+0000", 2024-09-17T22:42:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2469999999999999e-06 2024-09-17T22:42:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147602+0000", 2024-09-17T22:42:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0049999999999998e-06 2024-09-17T22:42:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190475+0000", 2024-09-17T22:42:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872767999999999 2024-09-17T22:42:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190528+0000", 2024-09-17T22:42:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2889999999999997e-05 2024-09-17T22:42:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190601+0000", 2024-09-17T22:42:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3168000000000001e-05 2024-09-17T22:42:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.890 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157255 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4153344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.675136273, 2024-09-17T22:42:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043132251000000003, 2024-09-17T22:42:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157255 2024-09-17T22:42:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147820+0000", 2024-09-17T22:42:43.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021123600000000001 2024-09-17T22:42:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147824+0000", 2024-09-17T22:42:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6780000000000002e-06 2024-09-17T22:42:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147833+0000", 2024-09-17T22:42:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4250000000000004e-06 2024-09-17T22:42:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147842+0000", 2024-09-17T22:42:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0679999999999997e-06 2024-09-17T22:42:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190632+0000", 2024-09-17T22:42:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042790863999999998 2024-09-17T22:42:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190674+0000", 2024-09-17T22:42:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1804000000000002e-05 2024-09-17T22:42:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190741+0000", 2024-09-17T22:42:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7175999999999995e-05 2024-09-17T22:42:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157257 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4159488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.674895782999997, 2024-09-17T22:42:43.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.04307035, 2024-09-17T22:42:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157257 2024-09-17T22:42:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148844+0000", 2024-09-17T22:42:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00099455599999999996 2024-09-17T22:42:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148848+0000", 2024-09-17T22:42:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0180000000000003e-06 2024-09-17T22:42:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148858+0000", 2024-09-17T22:42:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.825e-06 2024-09-17T22:42:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148865+0000", 2024-09-17T22:42:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8290000000000002e-06 2024-09-17T22:42:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190789+0000", 2024-09-17T22:42:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041924630999999997 2024-09-17T22:42:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190860+0000", 2024-09-17T22:42:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0557999999999998e-05 2024-09-17T22:42:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190920+0000", 2024-09-17T22:42:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9932999999999997e-05 2024-09-17T22:42:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.899 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157259 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4165632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.673869764000003, 2024-09-17T22:42:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042261411999999998, 2024-09-17T22:42:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.900 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157259 2024-09-17T22:42:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:42:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148878+0000", 2024-09-17T22:42:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:42:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148885+0000", 2024-09-17T22:42:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7290000000000003e-06 2024-09-17T22:42:43.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148893+0000", 2024-09-17T22:42:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8110000000000003e-06 2024-09-17T22:42:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190967+0000", 2024-09-17T22:42:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042073948999999999 2024-09-17T22:42:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191069+0000", 2024-09-17T22:42:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102075 2024-09-17T22:42:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191137+0000", 2024-09-17T22:42:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8281000000000006e-05 2024-09-17T22:42:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157261 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4171776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.904 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.673820157000002, 2024-09-17T22:42:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042437255, 2024-09-17T22:42:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157261 2024-09-17T22:42:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.905 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148926+0000", 2024-09-17T22:42:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T22:42:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148928+0000", 2024-09-17T22:42:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4329999999999998e-06 2024-09-17T22:42:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148936+0000", 2024-09-17T22:42:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6969999999999993e-06 2024-09-17T22:42:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148942+0000", 2024-09-17T22:42:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9479999999999996e-06 2024-09-17T22:42:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191185+0000", 2024-09-17T22:42:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042243147000000002 2024-09-17T22:42:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191266+0000", 2024-09-17T22:42:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1186000000000005e-05 2024-09-17T22:42:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191362+0000", 2024-09-17T22:42:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6384000000000004e-05 2024-09-17T22:42:43.908 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157263 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4177920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.673795757999997, 2024-09-17T22:42:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042636395000000001, 2024-09-17T22:42:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.909 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157263 2024-09-17T22:42:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.37e-07 2024-09-17T22:42:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148952+0000", 2024-09-17T22:42:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.88e-06 2024-09-17T22:42:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148959+0000", 2024-09-17T22:42:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5649999999999998e-06 2024-09-17T22:42:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148963+0000", 2024-09-17T22:42:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8570000000000003e-06 2024-09-17T22:42:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191408+0000", 2024-09-17T22:42:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042444707999999998 2024-09-17T22:42:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191489+0000", 2024-09-17T22:42:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0612999999999995e-05 2024-09-17T22:42:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191586+0000", 2024-09-17T22:42:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7235e-05 2024-09-17T22:42:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157265 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4184064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.673774209000001, 2024-09-17T22:42:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042791830000000003, 2024-09-17T22:42:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157265 2024-09-17T22:42:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148972+0000", 2024-09-17T22:42:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:42:43.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148973+0000", 2024-09-17T22:42:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.894e-06 2024-09-17T22:42:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148985+0000", 2024-09-17T22:42:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1995e-05 2024-09-17T22:42:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148990+0000", 2024-09-17T22:42:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8269999999999997e-06 2024-09-17T22:42:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191630+0000", 2024-09-17T22:42:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042639639 2024-09-17T22:42:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191686+0000", 2024-09-17T22:42:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6066000000000002e-05 2024-09-17T22:42:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191763+0000", 2024-09-17T22:42:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7035999999999999e-05 2024-09-17T22:42:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157267 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4190208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 53.673748396000001, 2024-09-17T22:42:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043028813999999999, 2024-09-17T22:42:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:43.918 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157267 2024-09-17T22:42:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:42:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148999+0000", 2024-09-17T22:42:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.432e-06 2024-09-17T22:42:43.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149002+0000", 2024-09-17T22:42:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7850000000000002e-06 2024-09-17T22:42:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149005+0000", 2024-09-17T22:42:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:42:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:43.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191878+0000", 2024-09-17T22:42:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872312000000003 2024-09-17T22:42:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191961+0000", 2024-09-17T22:42:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3418000000000007e-05 2024-09-17T22:42:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.192026+0000", 2024-09-17T22:42:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4689000000000004e-05 2024-09-17T22:42:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.923 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:43.923 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:43.923 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:43.924 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:42:43.925 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:43.922+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:44.025 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:44.023+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:44.067 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:44.068 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:44.068 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:44.076 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:42:44.127 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:44.124+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:44.178 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:44.178 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:44.179 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:44.179 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:44.179 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:44.187 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:42:44.228 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:44.226+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:44.292 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:44.292 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:44.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:44.292 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:44.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.292 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188693 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5881856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:44.292 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:44.292 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 35.461602546000002, 2024-09-17T22:42:44.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028193223, 2024-09-17T22:42:44.292 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.293 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.293 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.293 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.293 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.293 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.293 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188693 2024-09-17T22:42:44.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.293 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:44.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:44.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:44.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:42:44.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827090+0000", 2024-09-17T22:42:44.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2170000000000001e-06 2024-09-17T22:42:44.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827098+0000", 2024-09-17T22:42:44.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1510000000000004e-06 2024-09-17T22:42:44.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827103+0000", 2024-09-17T22:42:44.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0839999999999999e-06 2024-09-17T22:42:44.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855158+0000", 2024-09-17T22:42:44.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028055217 2024-09-17T22:42:44.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855201+0000", 2024-09-17T22:42:44.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3272000000000002e-05 2024-09-17T22:42:44.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855280+0000", 2024-09-17T22:42:44.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8845999999999997e-05 2024-09-17T22:42:44.296 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.297 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.297 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.297 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188694 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5971968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:44.297 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:44.297 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 35.461578658999997, 2024-09-17T22:42:44.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028312278, 2024-09-17T22:42:44.297 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.297 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.297 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.297 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.297 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.297 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.298 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188694 2024-09-17T22:42:44.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.298 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:44.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:44.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:44.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:42:44.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827114+0000", 2024-09-17T22:42:44.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3029999999999998e-06 2024-09-17T22:42:44.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827121+0000", 2024-09-17T22:42:44.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3699999999999997e-06 2024-09-17T22:42:44.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827126+0000", 2024-09-17T22:42:44.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7350000000000004e-06 2024-09-17T22:42:44.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855307+0000", 2024-09-17T22:42:44.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028180851999999999 2024-09-17T22:42:44.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855346+0000", 2024-09-17T22:42:44.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9274999999999999e-05 2024-09-17T22:42:44.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855423+0000", 2024-09-17T22:42:44.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7294999999999998e-05 2024-09-17T22:42:44.301 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.301 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.301 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.301 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188695 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:44.301 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:44.301 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 35.461557880999997, 2024-09-17T22:42:44.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028435876999999998, 2024-09-17T22:42:44.302 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.302 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.302 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.302 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.302 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.302 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.302 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188695 2024-09-17T22:42:44.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.302 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:44.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:44.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:44.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:42:44.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827138+0000", 2024-09-17T22:42:44.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2580000000000001e-06 2024-09-17T22:42:44.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827145+0000", 2024-09-17T22:42:44.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0129999999999996e-06 2024-09-17T22:42:44.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827150+0000", 2024-09-17T22:42:44.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3749999999999996e-06 2024-09-17T22:42:44.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855447+0000", 2024-09-17T22:42:44.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028297224999999999 2024-09-17T22:42:44.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855487+0000", 2024-09-17T22:42:44.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0454000000000003e-05 2024-09-17T22:42:44.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855567+0000", 2024-09-17T22:42:44.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0149999999999997e-05 2024-09-17T22:42:44.306 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.306 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.306 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.306 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188696 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:44.306 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:44.306 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 35.461534096999998, 2024-09-17T22:42:44.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028543570000000001, 2024-09-17T22:42:44.306 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.306 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.306 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.306 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.307 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.307 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.307 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188696 2024-09-17T22:42:44.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.307 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:44.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:44.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827156+0000", 2024-09-17T22:42:44.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T22:42:44.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827158+0000", 2024-09-17T22:42:44.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.875e-06 2024-09-17T22:42:44.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827164+0000", 2024-09-17T22:42:44.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7950000000000002e-06 2024-09-17T22:42:44.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827169+0000", 2024-09-17T22:42:44.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2880000000000003e-06 2024-09-17T22:42:44.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855582+0000", 2024-09-17T22:42:44.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028412929 2024-09-17T22:42:44.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855624+0000", 2024-09-17T22:42:44.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2484000000000002e-05 2024-09-17T22:42:44.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855699+0000", 2024-09-17T22:42:44.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4869000000000004e-05 2024-09-17T22:42:44.310 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.310 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.310 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.311 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188697 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6242304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:44.311 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:44.311 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 35.461502568, 2024-09-17T22:42:44.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028658598, 2024-09-17T22:42:44.311 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.311 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.311 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.311 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.311 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.311 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.311 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188697 2024-09-17T22:42:44.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.312 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:44.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:44.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:44.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8800000000000003e-07 2024-09-17T22:42:44.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827190+0000", 2024-09-17T22:42:44.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.351e-06 2024-09-17T22:42:44.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827198+0000", 2024-09-17T22:42:44.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.909e-06 2024-09-17T22:42:44.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827203+0000", 2024-09-17T22:42:44.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4020000000000001e-06 2024-09-17T22:42:44.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855720+0000", 2024-09-17T22:42:44.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028516909999999999 2024-09-17T22:42:44.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855762+0000", 2024-09-17T22:42:44.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2184999999999997e-05 2024-09-17T22:42:44.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855845+0000", 2024-09-17T22:42:44.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3353e-05 2024-09-17T22:42:44.315 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.315 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.315 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.315 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202883 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2342912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.315 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:42:44.316 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.563866186999999, 2024-09-17T22:42:44.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028098636, 2024-09-17T22:42:44.316 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.316 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.316 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.316 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.316 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.316 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.316 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202883 2024-09-17T22:42:44.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.316 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:42:44.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:42:44.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724824+0000", 2024-09-17T22:42:44.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:42:44.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724826+0000", 2024-09-17T22:42:44.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.086e-06 2024-09-17T22:42:44.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724832+0000", 2024-09-17T22:42:44.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9100000000000002e-06 2024-09-17T22:42:44.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724837+0000", 2024-09-17T22:42:44.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9439999999999996e-06 2024-09-17T22:42:44.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752790+0000", 2024-09-17T22:42:44.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027953657999999999 2024-09-17T22:42:44.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752848+0000", 2024-09-17T22:42:44.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7677999999999999e-05 2024-09-17T22:42:44.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752922+0000", 2024-09-17T22:42:44.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3957999999999999e-05 2024-09-17T22:42:44.320 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.320 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.320 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.320 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202884 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2473984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.320 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:42:44.320 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.563837962000001, 2024-09-17T22:42:44.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028206208999999999, 2024-09-17T22:42:44.320 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.320 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.320 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.321 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.321 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.321 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.321 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202884 2024-09-17T22:42:44.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.321 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:42:44.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:42:44.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724852+0000", 2024-09-17T22:42:44.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2599999999999998e-07 2024-09-17T22:42:44.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724854+0000", 2024-09-17T22:42:44.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.26e-06 2024-09-17T22:42:44.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724871+0000", 2024-09-17T22:42:44.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6589999999999999e-05 2024-09-17T22:42:44.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724877+0000", 2024-09-17T22:42:44.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.054e-06 2024-09-17T22:42:44.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752951+0000", 2024-09-17T22:42:44.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028073747 2024-09-17T22:42:44.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752993+0000", 2024-09-17T22:42:44.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2082000000000001e-05 2024-09-17T22:42:44.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753058+0000", 2024-09-17T22:42:44.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5049999999999996e-05 2024-09-17T22:42:44.324 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.324 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.324 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.325 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202885 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.325 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:44.325 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.563804658999999, 2024-09-17T22:42:44.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028333028, 2024-09-17T22:42:44.325 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.325 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.325 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.325 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.325 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.325 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.325 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202885 2024-09-17T22:42:44.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.325 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:44.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:44.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:44.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3300000000000002e-07 2024-09-17T22:42:44.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724892+0000", 2024-09-17T22:42:44.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2779999999999999e-06 2024-09-17T22:42:44.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724899+0000", 2024-09-17T22:42:44.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9110000000000004e-06 2024-09-17T22:42:44.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724904+0000", 2024-09-17T22:42:44.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2360000000000004e-06 2024-09-17T22:42:44.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753094+0000", 2024-09-17T22:42:44.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02819033 2024-09-17T22:42:44.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753142+0000", 2024-09-17T22:42:44.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.829e-05 2024-09-17T22:42:44.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753218+0000", 2024-09-17T22:42:44.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5450000000000004e-05 2024-09-17T22:42:44.329 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.329 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.329 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.329 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202886 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.329 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:44.329 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.563779460999999, 2024-09-17T22:42:44.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028442512, 2024-09-17T22:42:44.330 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.330 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.330 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.330 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.330 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.330 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.330 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202886 2024-09-17T22:42:44.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.330 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:44.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:44.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:44.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:42:44.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724912+0000", 2024-09-17T22:42:44.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.967e-06 2024-09-17T22:42:44.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724918+0000", 2024-09-17T22:42:44.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8069999999999996e-06 2024-09-17T22:42:44.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724923+0000", 2024-09-17T22:42:44.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.386e-06 2024-09-17T22:42:44.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753244+0000", 2024-09-17T22:42:44.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028321567999999998 2024-09-17T22:42:44.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753289+0000", 2024-09-17T22:42:44.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4620000000000003e-05 2024-09-17T22:42:44.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753353+0000", 2024-09-17T22:42:44.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3795999999999997e-05 2024-09-17T22:42:44.333 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.333 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.333 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.334 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202887 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.334 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:44.334 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.563761547999999, 2024-09-17T22:42:44.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028565838999999999, 2024-09-17T22:42:44.334 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.334 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.334 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.334 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.334 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.334 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.334 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202887 2024-09-17T22:42:44.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.335 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:44.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:44.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:44.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:42:44.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724930+0000", 2024-09-17T22:42:44.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8980000000000001e-06 2024-09-17T22:42:44.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724936+0000", 2024-09-17T22:42:44.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7849999999999997e-06 2024-09-17T22:42:44.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724942+0000", 2024-09-17T22:42:44.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9649999999999996e-06 2024-09-17T22:42:44.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753380+0000", 2024-09-17T22:42:44.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028438049 2024-09-17T22:42:44.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753425+0000", 2024-09-17T22:42:44.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5163000000000001e-05 2024-09-17T22:42:44.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753494+0000", 2024-09-17T22:42:44.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8588000000000001e-05 2024-09-17T22:42:44.338 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.338 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.338 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.338 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202888 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.338 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:42:44.338 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.563742049999998, 2024-09-17T22:42:44.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028703314000000001, 2024-09-17T22:42:44.339 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.339 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.339 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.339 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.339 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.339 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.339 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202888 2024-09-17T22:42:44.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.339 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:42:44.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:42:44.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724948+0000", 2024-09-17T22:42:44.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:42:44.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724950+0000", 2024-09-17T22:42:44.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8029999999999999e-06 2024-09-17T22:42:44.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724955+0000", 2024-09-17T22:42:44.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8760000000000002e-06 2024-09-17T22:42:44.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724960+0000", 2024-09-17T22:42:44.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5979999999999999e-06 2024-09-17T22:42:44.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753527+0000", 2024-09-17T22:42:44.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028566864000000001 2024-09-17T22:42:44.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753568+0000", 2024-09-17T22:42:44.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0775000000000002e-05 2024-09-17T22:42:44.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753651+0000", 2024-09-17T22:42:44.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3039000000000002e-05 2024-09-17T22:42:44.342 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.342 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.343 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.343 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202889 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.343 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:44.343 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.563724478000001, 2024-09-17T22:42:44.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028820964000000001, 2024-09-17T22:42:44.343 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.343 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.343 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.343 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.343 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.343 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.344 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202889 2024-09-17T22:42:44.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.344 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:44.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:44.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:44.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2599999999999998e-07 2024-09-17T22:42:44.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724967+0000", 2024-09-17T22:42:44.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T22:42:44.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724973+0000", 2024-09-17T22:42:44.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6060000000000002e-06 2024-09-17T22:42:44.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724978+0000", 2024-09-17T22:42:44.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8230000000000002e-06 2024-09-17T22:42:44.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753687+0000", 2024-09-17T22:42:44.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028709495000000002 2024-09-17T22:42:44.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753727+0000", 2024-09-17T22:42:44.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9953999999999997e-05 2024-09-17T22:42:44.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753786+0000", 2024-09-17T22:42:44.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8941999999999999e-05 2024-09-17T22:42:44.347 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.347 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.347 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.347 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202890 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.347 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:44.348 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.563706910000001, 2024-09-17T22:42:44.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028933838, 2024-09-17T22:42:44.348 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.348 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.348 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.348 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.348 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.348 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.348 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202890 2024-09-17T22:42:44.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.348 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:44.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:44.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:44.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5999999999999999e-07 2024-09-17T22:42:44.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724985+0000", 2024-09-17T22:42:44.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8169999999999999e-06 2024-09-17T22:42:44.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724991+0000", 2024-09-17T22:42:44.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:42:44.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.350 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.350 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724995+0000", 2024-09-17T22:42:44.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4530000000000004e-06 2024-09-17T22:42:44.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753811+0000", 2024-09-17T22:42:44.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028815584000000002 2024-09-17T22:42:44.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753854+0000", 2024-09-17T22:42:44.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3356999999999998e-05 2024-09-17T22:42:44.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753916+0000", 2024-09-17T22:42:44.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2488000000000002e-05 2024-09-17T22:42:44.352 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.352 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.352 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.352 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202891 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.352 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:44.352 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.563689452999999, 2024-09-17T22:42:44.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029067554999999998, 2024-09-17T22:42:44.352 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.352 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.352 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202891 2024-09-17T22:42:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:44.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:44.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:42:44.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725002+0000", 2024-09-17T22:42:44.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:42:44.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725008+0000", 2024-09-17T22:42:44.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6629999999999998e-06 2024-09-17T22:42:44.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725016+0000", 2024-09-17T22:42:44.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8269999999999998e-06 2024-09-17T22:42:44.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753949+0000", 2024-09-17T22:42:44.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028932625999999999 2024-09-17T22:42:44.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753994+0000", 2024-09-17T22:42:44.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5717000000000003e-05 2024-09-17T22:42:44.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754068+0000", 2024-09-17T22:42:44.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3312000000000006e-05 2024-09-17T22:42:44.356 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.356 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.357 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.357 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202892 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.357 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:44.357 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.563646746, 2024-09-17T22:42:44.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029159939999999999, 2024-09-17T22:42:44.357 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.357 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.357 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.357 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.357 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.358 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.358 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202892 2024-09-17T22:42:44.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.358 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:44.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:44.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:44.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7999999999999995e-07 2024-09-17T22:42:44.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725045+0000", 2024-09-17T22:42:44.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-17T22:42:44.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725052+0000", 2024-09-17T22:42:44.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3629999999999999e-06 2024-09-17T22:42:44.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725056+0000", 2024-09-17T22:42:44.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3250000000000001e-06 2024-09-17T22:42:44.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754094+0000", 2024-09-17T22:42:44.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029038270000000001 2024-09-17T22:42:44.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754134+0000", 2024-09-17T22:42:44.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9722999999999999e-05 2024-09-17T22:42:44.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754203+0000", 2024-09-17T22:42:44.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8524999999999999e-05 2024-09-17T22:42:44.361 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.362 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.362 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.362 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202893 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.362 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:42:44.362 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.563627169, 2024-09-17T22:42:44.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029341651, 2024-09-17T22:42:44.362 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.362 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.362 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.362 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.362 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.362 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.363 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202893 2024-09-17T22:42:44.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.363 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:42:44.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:42:44.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725063+0000", 2024-09-17T22:42:44.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6699999999999999e-07 2024-09-17T22:42:44.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725065+0000", 2024-09-17T22:42:44.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2120000000000002e-06 2024-09-17T22:42:44.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725071+0000", 2024-09-17T22:42:44.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5169999999999996e-06 2024-09-17T22:42:44.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725076+0000", 2024-09-17T22:42:44.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.78e-06 2024-09-17T22:42:44.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754233+0000", 2024-09-17T22:42:44.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029156277000000001 2024-09-17T22:42:44.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754298+0000", 2024-09-17T22:42:44.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5320000000000005e-05 2024-09-17T22:42:44.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754404+0000", 2024-09-17T22:42:44.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106078 2024-09-17T22:42:44.366 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.366 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.366 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.366 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202894 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.366 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:42:44.366 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.56360729, 2024-09-17T22:42:44.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029456572, 2024-09-17T22:42:44.367 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.367 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.367 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.367 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.367 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.367 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.367 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202894 2024-09-17T22:42:44.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.367 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:42:44.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:42:44.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725083+0000", 2024-09-17T22:42:44.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5900000000000002e-07 2024-09-17T22:42:44.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725085+0000", 2024-09-17T22:42:44.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.979e-06 2024-09-17T22:42:44.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725091+0000", 2024-09-17T22:42:44.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1480000000000003e-06 2024-09-17T22:42:44.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725095+0000", 2024-09-17T22:42:44.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1339999999999997e-06 2024-09-17T22:42:44.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754429+0000", 2024-09-17T22:42:44.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029333758000000001 2024-09-17T22:42:44.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754471+0000", 2024-09-17T22:42:44.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2274999999999998e-05 2024-09-17T22:42:44.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754539+0000", 2024-09-17T22:42:44.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7818999999999996e-05 2024-09-17T22:42:44.371 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.371 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.371 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.371 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202895 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.371 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:44.371 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.563588893999999, 2024-09-17T22:42:44.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029635578999999999, 2024-09-17T22:42:44.371 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.371 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.372 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.372 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.372 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.372 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.372 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202895 2024-09-17T22:42:44.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.372 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:44.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:44.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:44.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.08e-07 2024-09-17T22:42:44.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725107+0000", 2024-09-17T22:42:44.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9020000000000004e-06 2024-09-17T22:42:44.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725113+0000", 2024-09-17T22:42:44.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7400000000000001e-06 2024-09-17T22:42:44.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725117+0000", 2024-09-17T22:42:44.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:42:44.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754567+0000", 2024-09-17T22:42:44.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029449411000000002 2024-09-17T22:42:44.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754638+0000", 2024-09-17T22:42:44.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1432000000000003e-05 2024-09-17T22:42:44.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754736+0000", 2024-09-17T22:42:44.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8066000000000006e-05 2024-09-17T22:42:44.375 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.375 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.376 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.376 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202896 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.376 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:42:44.376 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.563567162999998, 2024-09-17T22:42:44.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029760534000000002, 2024-09-17T22:42:44.376 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.376 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.376 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.376 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.377 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202896 2024-09-17T22:42:44.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.377 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:42:44.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:42:44.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725123+0000", 2024-09-17T22:42:44.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:42:44.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725125+0000", 2024-09-17T22:42:44.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.863e-06 2024-09-17T22:42:44.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725130+0000", 2024-09-17T22:42:44.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7250000000000002e-06 2024-09-17T22:42:44.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725135+0000", 2024-09-17T22:42:44.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:42:44.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754763+0000", 2024-09-17T22:42:44.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029628202999999999 2024-09-17T22:42:44.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754805+0000", 2024-09-17T22:42:44.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2370000000000003e-05 2024-09-17T22:42:44.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754883+0000", 2024-09-17T22:42:44.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7565999999999996e-05 2024-09-17T22:42:44.380 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.380 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.380 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.380 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202897 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.380 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:44.381 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 26.563549803000001, 2024-09-17T22:42:44.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029947972999999999, 2024-09-17T22:42:44.381 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.381 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.381 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.381 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.381 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.381 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.381 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202897 2024-09-17T22:42:44.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.381 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:44.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:44.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:44.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:42:44.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725146+0000", 2024-09-17T22:42:44.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6030000000000004e-06 2024-09-17T22:42:44.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725152+0000", 2024-09-17T22:42:44.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9970000000000004e-06 2024-09-17T22:42:44.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725156+0000", 2024-09-17T22:42:44.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-06 2024-09-17T22:42:44.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754909+0000", 2024-09-17T22:42:44.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029752958999999999 2024-09-17T22:42:44.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754987+0000", 2024-09-17T22:42:44.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8208999999999996e-05 2024-09-17T22:42:44.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.755088+0000", 2024-09-17T22:42:44.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100348 2024-09-17T22:42:44.385 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.385 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.385 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.385 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.385 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.385 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:44.386 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:42:44.387 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:44.384+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:44.487 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:44.484+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:44.535 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:44.536 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:44.536 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:44.536 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:44.544 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:42:44.588 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:44.586+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:44.648 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:44.648 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:44.648 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:44.648 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:44.648 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:44.657 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:42:44.689 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:44.687+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:44.790 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:44.788+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:44.800 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:44.800 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:44.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:44.801 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:44.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.801 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202977 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.801 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:42:44.801 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.071434870000001, 2024-09-17T22:42:44.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023068750999999998, 2024-09-17T22:42:44.801 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.801 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.801 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.801 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.801 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.801 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.802 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202977 2024-09-17T22:42:44.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.802 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:42:44.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:42:44.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725980+0000", 2024-09-17T22:42:44.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:42:44.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.803 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725982+0000", 2024-09-17T22:42:44.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0669999999999999e-06 2024-09-17T22:42:44.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.803 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.803 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725988+0000", 2024-09-17T22:42:44.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6610000000000003e-06 2024-09-17T22:42:44.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725993+0000", 2024-09-17T22:42:44.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:42:44.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748888+0000", 2024-09-17T22:42:44.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022894930000000001 2024-09-17T22:42:44.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748951+0000", 2024-09-17T22:42:44.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3638000000000003e-05 2024-09-17T22:42:44.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749048+0000", 2024-09-17T22:42:44.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6343999999999998e-05 2024-09-17T22:42:44.805 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.805 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.805 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.806 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202978 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2211840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.806 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:44.806 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.071414357999998, 2024-09-17T22:42:44.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023238050999999999, 2024-09-17T22:42:44.806 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.806 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.806 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.806 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.806 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.806 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.806 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202978 2024-09-17T22:42:44.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.807 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:44.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:44.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:44.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.58e-07 2024-09-17T22:42:44.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726002+0000", 2024-09-17T22:42:44.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.193e-06 2024-09-17T22:42:44.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726009+0000", 2024-09-17T22:42:44.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8000000000000001e-06 2024-09-17T22:42:44.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726014+0000", 2024-09-17T22:42:44.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9840000000000001e-06 2024-09-17T22:42:44.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749090+0000", 2024-09-17T22:42:44.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023075917000000001 2024-09-17T22:42:44.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749163+0000", 2024-09-17T22:42:44.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3219999999999994e-05 2024-09-17T22:42:44.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749238+0000", 2024-09-17T22:42:44.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4479000000000003e-05 2024-09-17T22:42:44.810 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.810 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.810 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.810 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202979 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2342912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.810 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:42:44.811 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.071392907, 2024-09-17T22:42:44.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023393536999999999, 2024-09-17T22:42:44.811 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.811 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.811 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.811 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.811 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.811 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.811 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202979 2024-09-17T22:42:44.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.811 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:42:44.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:42:44.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726022+0000", 2024-09-17T22:42:44.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:42:44.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726024+0000", 2024-09-17T22:42:44.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1969999999999999e-06 2024-09-17T22:42:44.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726030+0000", 2024-09-17T22:42:44.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.691e-06 2024-09-17T22:42:44.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726035+0000", 2024-09-17T22:42:44.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8029999999999996e-06 2024-09-17T22:42:44.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749280+0000", 2024-09-17T22:42:44.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023244668 2024-09-17T22:42:44.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749344+0000", 2024-09-17T22:42:44.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4418000000000006e-05 2024-09-17T22:42:44.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749415+0000", 2024-09-17T22:42:44.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0272999999999996e-05 2024-09-17T22:42:44.815 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.815 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.815 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.815 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202980 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2473984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.815 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:44.815 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.071372138000001, 2024-09-17T22:42:44.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023562221000000001, 2024-09-17T22:42:44.815 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.816 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.816 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.816 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202980 2024-09-17T22:42:44.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.816 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:44.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:44.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:44.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:42:44.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726044+0000", 2024-09-17T22:42:44.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9489999999999999e-06 2024-09-17T22:42:44.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726066+0000", 2024-09-17T22:42:44.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1508000000000001e-05 2024-09-17T22:42:44.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726072+0000", 2024-09-17T22:42:44.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3450000000000002e-06 2024-09-17T22:42:44.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749455+0000", 2024-09-17T22:42:44.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023382824999999999 2024-09-17T22:42:44.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749527+0000", 2024-09-17T22:42:44.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1728000000000002e-05 2024-09-17T22:42:44.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749604+0000", 2024-09-17T22:42:44.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7428999999999993e-05 2024-09-17T22:42:44.819 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.819 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.820 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.820 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202981 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.820 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:42:44.820 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.071333863, 2024-09-17T22:42:44.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023677800999999998, 2024-09-17T22:42:44.820 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.820 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.820 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.820 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.820 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.820 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.821 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202981 2024-09-17T22:42:44.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.821 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:42:44.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:42:44.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726081+0000", 2024-09-17T22:42:44.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5499999999999998e-07 2024-09-17T22:42:44.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726083+0000", 2024-09-17T22:42:44.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1780000000000002e-06 2024-09-17T22:42:44.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726090+0000", 2024-09-17T22:42:44.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8510000000000001e-06 2024-09-17T22:42:44.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726101+0000", 2024-09-17T22:42:44.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.133e-05 2024-09-17T22:42:44.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749639+0000", 2024-09-17T22:42:44.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023537679999999998 2024-09-17T22:42:44.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749688+0000", 2024-09-17T22:42:44.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9548000000000001e-05 2024-09-17T22:42:44.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749758+0000", 2024-09-17T22:42:44.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9758999999999994e-05 2024-09-17T22:42:44.824 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.824 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.824 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.824 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202982 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.824 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:44.824 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.071306280000002, 2024-09-17T22:42:44.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023841469000000001, 2024-09-17T22:42:44.825 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.825 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.825 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.825 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202982 2024-09-17T22:42:44.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.825 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:44.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:44.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:44.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:42:44.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726110+0000", 2024-09-17T22:42:44.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.902e-06 2024-09-17T22:42:44.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726117+0000", 2024-09-17T22:42:44.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6189999999999996e-06 2024-09-17T22:42:44.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726122+0000", 2024-09-17T22:42:44.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.198e-06 2024-09-17T22:42:44.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749795+0000", 2024-09-17T22:42:44.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023673397999999998 2024-09-17T22:42:44.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749877+0000", 2024-09-17T22:42:44.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1910999999999996e-05 2024-09-17T22:42:44.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749949+0000", 2024-09-17T22:42:44.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2001000000000004e-05 2024-09-17T22:42:44.829 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.829 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.829 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.829 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202983 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.829 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:44.829 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.071285221, 2024-09-17T22:42:44.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024014081, 2024-09-17T22:42:44.829 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.829 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.829 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.829 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.830 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.830 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.830 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202983 2024-09-17T22:42:44.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.830 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:44.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:44.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:44.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:42:44.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726131+0000", 2024-09-17T22:42:44.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9429999999999999e-06 2024-09-17T22:42:44.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726138+0000", 2024-09-17T22:42:44.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T22:42:44.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726143+0000", 2024-09-17T22:42:44.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0629999999999996e-06 2024-09-17T22:42:44.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749991+0000", 2024-09-17T22:42:44.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023847671000000001 2024-09-17T22:42:44.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750065+0000", 2024-09-17T22:42:44.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4569999999999999e-05 2024-09-17T22:42:44.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750143+0000", 2024-09-17T22:42:44.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7649000000000001e-05 2024-09-17T22:42:44.833 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.834 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.834 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.834 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202984 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.834 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:42:44.834 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.07126452, 2024-09-17T22:42:44.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024177605000000001, 2024-09-17T22:42:44.834 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.834 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.834 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.835 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202984 2024-09-17T22:42:44.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.835 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:42:44.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:42:44.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726150+0000", 2024-09-17T22:42:44.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T22:42:44.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726152+0000", 2024-09-17T22:42:44.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:42:44.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726162+0000", 2024-09-17T22:42:44.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7839999999999998e-06 2024-09-17T22:42:44.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726167+0000", 2024-09-17T22:42:44.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5690000000000004e-06 2024-09-17T22:42:44.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750184+0000", 2024-09-17T22:42:44.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024016909999999999 2024-09-17T22:42:44.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750255+0000", 2024-09-17T22:42:44.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1137999999999996e-05 2024-09-17T22:42:44.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750327+0000", 2024-09-17T22:42:44.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1903000000000006e-05 2024-09-17T22:42:44.838 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.838 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.838 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.838 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202985 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.839 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:44.839 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.071239132999999, 2024-09-17T22:42:44.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024280500999999999, 2024-09-17T22:42:44.839 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.839 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.839 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.839 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.839 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.839 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.839 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202985 2024-09-17T22:42:44.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.839 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:44.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:44.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:44.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8100000000000003e-07 2024-09-17T22:42:44.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726182+0000", 2024-09-17T22:42:44.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:42:44.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726190+0000", 2024-09-17T22:42:44.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:42:44.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726195+0000", 2024-09-17T22:42:44.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2809999999999999e-06 2024-09-17T22:42:44.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750366+0000", 2024-09-17T22:42:44.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024171635 2024-09-17T22:42:44.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750410+0000", 2024-09-17T22:42:44.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3823999999999999e-05 2024-09-17T22:42:44.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750455+0000", 2024-09-17T22:42:44.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5039e-05 2024-09-17T22:42:44.843 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.843 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.843 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.843 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202986 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.843 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:42:44.843 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.071211553000001, 2024-09-17T22:42:44.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024453077, 2024-09-17T22:42:44.843 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.844 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.844 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.844 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.844 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.844 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.844 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202986 2024-09-17T22:42:44.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.844 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:42:44.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:42:44.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726203+0000", 2024-09-17T22:42:44.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T22:42:44.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726205+0000", 2024-09-17T22:42:44.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0250000000000001e-06 2024-09-17T22:42:44.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726211+0000", 2024-09-17T22:42:44.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1340000000000001e-06 2024-09-17T22:42:44.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726222+0000", 2024-09-17T22:42:44.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1105e-05 2024-09-17T22:42:44.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750520+0000", 2024-09-17T22:42:44.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024297462999999998 2024-09-17T22:42:44.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750585+0000", 2024-09-17T22:42:44.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5940999999999998e-05 2024-09-17T22:42:44.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750655+0000", 2024-09-17T22:42:44.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0012000000000007e-05 2024-09-17T22:42:44.847 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.848 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.848 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.848 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202987 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.848 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:44.848 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.071185173, 2024-09-17T22:42:44.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024622569, 2024-09-17T22:42:44.848 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.848 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.848 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.848 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.848 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.849 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202987 2024-09-17T22:42:44.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.849 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:44.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:44.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:44.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:42:44.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726238+0000", 2024-09-17T22:42:44.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7430000000000008e-06 2024-09-17T22:42:44.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726245+0000", 2024-09-17T22:42:44.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6640000000000001e-06 2024-09-17T22:42:44.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726249+0000", 2024-09-17T22:42:44.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9119999999999997e-06 2024-09-17T22:42:44.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750694+0000", 2024-09-17T22:42:44.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024444105000000001 2024-09-17T22:42:44.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750767+0000", 2024-09-17T22:42:44.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3577000000000004e-05 2024-09-17T22:42:44.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750851+0000", 2024-09-17T22:42:44.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4189000000000003e-05 2024-09-17T22:42:44.852 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.852 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.852 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.853 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202988 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.853 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:44.853 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.071158311000001, 2024-09-17T22:42:44.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024808883, 2024-09-17T22:42:44.853 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.853 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.853 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.853 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.853 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.853 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.853 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202988 2024-09-17T22:42:44.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.853 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:44.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:44.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:44.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5499999999999999e-07 2024-09-17T22:42:44.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726258+0000", 2024-09-17T22:42:44.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8020000000000001e-06 2024-09-17T22:42:44.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726272+0000", 2024-09-17T22:42:44.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3828e-05 2024-09-17T22:42:44.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726277+0000", 2024-09-17T22:42:44.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2379999999999997e-06 2024-09-17T22:42:44.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750895+0000", 2024-09-17T22:42:44.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024618543 2024-09-17T22:42:44.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750978+0000", 2024-09-17T22:42:44.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3030999999999998e-05 2024-09-17T22:42:44.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751065+0000", 2024-09-17T22:42:44.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6086000000000002e-05 2024-09-17T22:42:44.857 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.857 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.857 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.857 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202989 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.857 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:42:44.857 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.071129438, 2024-09-17T22:42:44.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024976788, 2024-09-17T22:42:44.858 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.858 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.858 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.858 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.858 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.858 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.858 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202989 2024-09-17T22:42:44.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.858 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:42:44.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:42:44.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726285+0000", 2024-09-17T22:42:44.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T22:42:44.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726287+0000", 2024-09-17T22:42:44.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9860000000000001e-06 2024-09-17T22:42:44.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726294+0000", 2024-09-17T22:42:44.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2849999999999997e-06 2024-09-17T22:42:44.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726299+0000", 2024-09-17T22:42:44.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0150000000000003e-06 2024-09-17T22:42:44.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751109+0000", 2024-09-17T22:42:44.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024810175 2024-09-17T22:42:44.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751184+0000", 2024-09-17T22:42:44.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4640000000000004e-05 2024-09-17T22:42:44.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751261+0000", 2024-09-17T22:42:44.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7317000000000005e-05 2024-09-17T22:42:44.862 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.862 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.862 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.862 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202990 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.862 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:44.862 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.071109223000001, 2024-09-17T22:42:44.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025156767999999999, 2024-09-17T22:42:44.862 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.862 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.863 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.863 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202990 2024-09-17T22:42:44.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.863 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:44.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:44.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:44.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:42:44.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726307+0000", 2024-09-17T22:42:44.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7090000000000001e-06 2024-09-17T22:42:44.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726312+0000", 2024-09-17T22:42:44.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4909999999999996e-06 2024-09-17T22:42:44.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726317+0000", 2024-09-17T22:42:44.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3800000000000004e-06 2024-09-17T22:42:44.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751308+0000", 2024-09-17T22:42:44.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024991143 2024-09-17T22:42:44.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751385+0000", 2024-09-17T22:42:44.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6793999999999997e-05 2024-09-17T22:42:44.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751461+0000", 2024-09-17T22:42:44.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6891999999999995e-05 2024-09-17T22:42:44.866 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.866 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.867 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.867 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202991 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.867 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:44.867 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.071091189000001, 2024-09-17T22:42:44.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025289477000000001, 2024-09-17T22:42:44.867 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.867 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.867 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.867 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.867 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.867 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.867 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202991 2024-09-17T22:42:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:44.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.27e-07 2024-09-17T22:42:44.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726325+0000", 2024-09-17T22:42:44.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1459999999999999e-06 2024-09-17T22:42:44.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726333+0000", 2024-09-17T22:42:44.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1289999999999996e-06 2024-09-17T22:42:44.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726339+0000", 2024-09-17T22:42:44.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2530000000000004e-06 2024-09-17T22:42:44.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751499+0000", 2024-09-17T22:42:44.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025160109 2024-09-17T22:42:44.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751543+0000", 2024-09-17T22:42:44.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4317000000000003e-05 2024-09-17T22:42:44.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751612+0000", 2024-09-17T22:42:44.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9096000000000004e-05 2024-09-17T22:42:44.871 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.871 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.871 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.871 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202992 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.872 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:44.872 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.071069125000001, 2024-09-17T22:42:44.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025420585999999998, 2024-09-17T22:42:44.872 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.872 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.872 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.872 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.872 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.872 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.872 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202992 2024-09-17T22:42:44.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.872 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:44.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:44.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:44.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5199999999999998e-07 2024-09-17T22:42:44.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726347+0000", 2024-09-17T22:42:44.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9939999999999999e-06 2024-09-17T22:42:44.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726358+0000", 2024-09-17T22:42:44.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0421999999999999e-05 2024-09-17T22:42:44.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726363+0000", 2024-09-17T22:42:44.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:42:44.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751644+0000", 2024-09-17T22:42:44.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025280727999999999 2024-09-17T22:42:44.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751712+0000", 2024-09-17T22:42:44.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8207999999999995e-05 2024-09-17T22:42:44.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751765+0000", 2024-09-17T22:42:44.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3533999999999999e-05 2024-09-17T22:42:44.876 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.876 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.876 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.876 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202993 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:44.876 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:44.876 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 27.071045133999998, 2024-09-17T22:42:44.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025516259999999999, 2024-09-17T22:42:44.876 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.876 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.877 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.877 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.877 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.877 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.877 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202993 2024-09-17T22:42:44.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.877 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:44.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:44.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:44.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:42:44.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726371+0000", 2024-09-17T22:42:44.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T22:42:44.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726380+0000", 2024-09-17T22:42:44.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9109999999999999e-06 2024-09-17T22:42:44.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726385+0000", 2024-09-17T22:42:44.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7380000000000002e-06 2024-09-17T22:42:44.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751785+0000", 2024-09-17T22:42:44.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025400526 2024-09-17T22:42:44.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751824+0000", 2024-09-17T22:42:44.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8989999999999998e-05 2024-09-17T22:42:44.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751885+0000", 2024-09-17T22:42:44.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0937000000000003e-05 2024-09-17T22:42:44.880 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.880 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.881 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.881 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:215737 3.b 3:d5ee24db:::rbd_data.115f2f67ca35.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-17T22:42:44.881 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:42:44.881 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4962764390000007, 2024-09-17T22:42:44.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025642133000000001, 2024-09-17T22:42:44.881 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.881 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.881 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:44.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.881 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 215737 2024-09-17T22:42:44.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.882 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:42:44.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:42:44.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301138+0000", 2024-09-17T22:42:44.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8100000000000002e-07 2024-09-17T22:42:44.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301143+0000", 2024-09-17T22:42:44.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6999999999999999e-06 2024-09-17T22:42:44.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301152+0000", 2024-09-17T22:42:44.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4979999999999995e-06 2024-09-17T22:42:44.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301159+0000", 2024-09-17T22:42:44.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9430000000000003e-06 2024-09-17T22:42:44.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301201+0000", 2024-09-17T22:42:44.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1219e-05 2024-09-17T22:42:44.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301247+0000", 2024-09-17T22:42:44.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5924999999999998e-05 2024-09-17T22:42:44.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:42:44.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301870+0000", 2024-09-17T22:42:44.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00062316100000000001 2024-09-17T22:42:44.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:44.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326658+0000", 2024-09-17T22:42:44.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024788719000000001 2024-09-17T22:42:44.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:42:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326694+0000", 2024-09-17T22:42:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5858999999999997e-05 2024-09-17T22:42:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326727+0000", 2024-09-17T22:42:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3176999999999998e-05 2024-09-17T22:42:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326780+0000", 2024-09-17T22:42:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2151000000000003e-05 2024-09-17T22:42:44.886 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:215738 3.b 3:d5937cde:::rbd_data.115f2f67ca35.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-17T22:42:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:42:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4962399069999996, 2024-09-17T22:42:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025650267000000001, 2024-09-17T22:42:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:44.887 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:44.888 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 215738 2024-09-17T22:42:44.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.888 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:42:44.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:42:44.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301175+0000", 2024-09-17T22:42:44.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2600000000000002e-07 2024-09-17T22:42:44.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301178+0000", 2024-09-17T22:42:44.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2049999999999998e-06 2024-09-17T22:42:44.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301185+0000", 2024-09-17T22:42:44.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8900000000000001e-06 2024-09-17T22:42:44.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301190+0000", 2024-09-17T22:42:44.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6910000000000002e-06 2024-09-17T22:42:44.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.304605+0000", 2024-09-17T22:42:44.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003414594 2024-09-17T22:42:44.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.304658+0000", 2024-09-17T22:42:44.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3334000000000001e-05 2024-09-17T22:42:44.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:42:44.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.308265+0000", 2024-09-17T22:42:44.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036068680000000001 2024-09-17T22:42:44.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:44.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326688+0000", 2024-09-17T22:42:44.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018422851000000001 2024-09-17T22:42:44.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:42:44.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326785+0000", 2024-09-17T22:42:44.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7050000000000001e-05 2024-09-17T22:42:44.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:44.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326806+0000", 2024-09-17T22:42:44.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1299999999999999e-05 2024-09-17T22:42:44.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326824+0000", 2024-09-17T22:42:44.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7958e-05 2024-09-17T22:42:44.893 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.893 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.893 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.893 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop_reply(client.4447.0:215737 3.b e87/23 ondisk, result = 0)", 2024-09-17T22:42:44.893 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:42:44.893 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 9.4937846960000005, 2024-09-17T22:42:44.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023035287000000002, 2024-09-17T22:42:44.893 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:44.893 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:44.893 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:44.893 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:44.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:44.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:42:44.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:44.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:42:44.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:44.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:44.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303630+0000", 2024-09-17T22:42:44.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2e-07 2024-09-17T22:42:44.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:44.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303633+0000", 2024-09-17T22:42:44.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7470000000000001e-06 2024-09-17T22:42:44.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:44.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303641+0000", 2024-09-17T22:42:44.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7979999999999995e-06 2024-09-17T22:42:44.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:44.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303646+0000", 2024-09-17T22:42:44.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9910000000000002e-06 2024-09-17T22:42:44.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:44.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326647+0000", 2024-09-17T22:42:44.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023000936999999999 2024-09-17T22:42:44.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:44.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326654+0000", 2024-09-17T22:42:44.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2529999999999998e-06 2024-09-17T22:42:44.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:44.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:44.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:44.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326665+0000", 2024-09-17T22:42:44.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0040999999999999e-05 2024-09-17T22:42:44.897 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.897 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.897 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.897 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:44.897 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:44.897 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:44.898 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:44.896+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:44.899 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:42:44.999 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:44.996+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:45.005 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:45.006 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:45.006 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:45.006 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:45.014 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:42:45.099 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:45.097+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:45.157 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:45.157 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:45.157 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:45.157 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:45.157 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:45.166 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:42:45.201 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:45.199+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:45.302 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:45.299+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:45.311 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:42:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:45.311 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.948895358, 2024-09-17T22:42:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.249158453, 2024-09-17T22:42:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:42:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:42:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-17T22:42:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358680+0000", 2024-09-17T22:42:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.592e-06 2024-09-17T22:42:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358688+0000", 2024-09-17T22:42:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1610000000000003e-06 2024-09-17T22:42:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358697+0000", 2024-09-17T22:42:45.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2580000000000001e-06 2024-09-17T22:42:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358706+0000", 2024-09-17T22:42:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1640000000000007e-06 2024-09-17T22:42:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358766+0000", 2024-09-17T22:42:45.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9893999999999999e-05 2024-09-17T22:42:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:42:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358778+0000", 2024-09-17T22:42:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1148999999999999e-05 2024-09-17T22:42:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606347+0000", 2024-09-17T22:42:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.247569971 2024-09-17T22:42:45.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606533+0000", 2024-09-17T22:42:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018599400000000001 2024-09-17T22:42:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:42:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606752+0000", 2024-09-17T22:42:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000218373 2024-09-17T22:42:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:42:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607455+0000", 2024-09-17T22:42:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070311799999999997 2024-09-17T22:42:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607735+0000", 2024-09-17T22:42:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027972300000000001 2024-09-17T22:42:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:42:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607793+0000", 2024-09-17T22:42:45.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8529999999999997e-05 2024-09-17T22:42:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607837+0000", 2024-09-17T22:42:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3526000000000003e-05 2024-09-17T22:42:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186749 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 11960320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.884254036000002, 2024-09-17T22:42:45.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029226166000000001, 2024-09-17T22:42:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186749 2024-09-17T22:42:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-09-17T22:42:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423328+0000", 2024-09-17T22:42:45.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6790000000000005e-06 2024-09-17T22:42:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423334+0000", 2024-09-17T22:42:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0009999999999998e-06 2024-09-17T22:42:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423338+0000", 2024-09-17T22:42:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6419999999999998e-06 2024-09-17T22:42:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452464+0000", 2024-09-17T22:42:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029125389000000002 2024-09-17T22:42:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452501+0000", 2024-09-17T22:42:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7656e-05 2024-09-17T22:42:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452546+0000", 2024-09-17T22:42:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4468999999999997e-05 2024-09-17T22:42:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.323 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186750 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12050432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.884228245999999, 2024-09-17T22:42:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029307863999999999, 2024-09-17T22:42:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186750 2024-09-17T22:42:45.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423346+0000", 2024-09-17T22:42:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.84e-07 2024-09-17T22:42:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423348+0000", 2024-09-17T22:42:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7540000000000001e-06 2024-09-17T22:42:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423353+0000", 2024-09-17T22:42:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:42:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423362+0000", 2024-09-17T22:42:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9220000000000003e-06 2024-09-17T22:42:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452571+0000", 2024-09-17T22:42:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029209174000000001 2024-09-17T22:42:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452612+0000", 2024-09-17T22:42:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1363999999999999e-05 2024-09-17T22:42:45.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452653+0000", 2024-09-17T22:42:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0893999999999998e-05 2024-09-17T22:42:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186751 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12140544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.884205756, 2024-09-17T22:42:45.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029387488, 2024-09-17T22:42:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186751 2024-09-17T22:42:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:45.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1399999999999997e-07 2024-09-17T22:42:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423375+0000", 2024-09-17T22:42:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8889999999999996e-06 2024-09-17T22:42:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423381+0000", 2024-09-17T22:42:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5690000000000004e-06 2024-09-17T22:42:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423386+0000", 2024-09-17T22:42:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9559999999999996e-06 2024-09-17T22:42:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452679+0000", 2024-09-17T22:42:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029292820000000001 2024-09-17T22:42:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452718+0000", 2024-09-17T22:42:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8961000000000002e-05 2024-09-17T22:42:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452755+0000", 2024-09-17T22:42:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7879000000000001e-05 2024-09-17T22:42:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.332 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186752 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12230656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.884174504999997, 2024-09-17T22:42:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029463269, 2024-09-17T22:42:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:45.333 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186752 2024-09-17T22:42:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423400+0000", 2024-09-17T22:42:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:42:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423401+0000", 2024-09-17T22:42:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3599999999999999e-06 2024-09-17T22:42:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423404+0000", 2024-09-17T22:42:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5999999999999998e-06 2024-09-17T22:42:45.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423407+0000", 2024-09-17T22:42:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6869999999999999e-06 2024-09-17T22:42:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452782+0000", 2024-09-17T22:42:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029375107000000001 2024-09-17T22:42:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452821+0000", 2024-09-17T22:42:45.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9122999999999998e-05 2024-09-17T22:42:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452862+0000", 2024-09-17T22:42:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1019000000000001e-05 2024-09-17T22:42:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186753 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.884162482999997, 2024-09-17T22:42:45.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029553297999999999, 2024-09-17T22:42:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186753 2024-09-17T22:42:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.98e-07 2024-09-17T22:42:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423412+0000", 2024-09-17T22:42:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8700000000000004e-07 2024-09-17T22:42:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423415+0000", 2024-09-17T22:42:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:42:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423418+0000", 2024-09-17T22:42:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.283e-06 2024-09-17T22:42:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452893+0000", 2024-09-17T22:42:45.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029475881999999998 2024-09-17T22:42:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452928+0000", 2024-09-17T22:42:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4109000000000002e-05 2024-09-17T22:42:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452964+0000", 2024-09-17T22:42:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6937000000000002e-05 2024-09-17T22:42:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186754 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12410880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.884153345999998, 2024-09-17T22:42:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029656021000000001, 2024-09-17T22:42:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186754 2024-09-17T22:42:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.342 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423421+0000", 2024-09-17T22:42:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9999999999999999e-07 2024-09-17T22:42:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423425+0000", 2024-09-17T22:42:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6519999999999997e-06 2024-09-17T22:42:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423428+0000", 2024-09-17T22:42:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1370000000000002e-06 2024-09-17T22:42:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423431+0000", 2024-09-17T22:42:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4090000000000001e-06 2024-09-17T22:42:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452992+0000", 2024-09-17T22:42:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029561545000000002 2024-09-17T22:42:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453031+0000", 2024-09-17T22:42:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8834000000000001e-05 2024-09-17T22:42:45.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453076+0000", 2024-09-17T22:42:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5244000000000003e-05 2024-09-17T22:42:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186755 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12500992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.884140064999997, 2024-09-17T22:42:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029745649999999998, 2024-09-17T22:42:45.346 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186755 2024-09-17T22:42:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.17e-07 2024-09-17T22:42:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423435+0000", 2024-09-17T22:42:45.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:42:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423438+0000", 2024-09-17T22:42:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8210000000000002e-06 2024-09-17T22:42:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423443+0000", 2024-09-17T22:42:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1070000000000004e-06 2024-09-17T22:42:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453103+0000", 2024-09-17T22:42:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029660399 2024-09-17T22:42:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453141+0000", 2024-09-17T22:42:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.748e-05 2024-09-17T22:42:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453179+0000", 2024-09-17T22:42:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8671000000000003e-05 2024-09-17T22:42:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187439 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8437760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.766702967999997, 2024-09-17T22:42:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029278048000000001, 2024-09-17T22:42:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187439 2024-09-17T22:42:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.351 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:45.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6199999999999999e-07 2024-09-17T22:42:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540872+0000", 2024-09-17T22:42:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.09e-07 2024-09-17T22:42:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540875+0000", 2024-09-17T22:42:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8990000000000001e-06 2024-09-17T22:42:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540877+0000", 2024-09-17T22:42:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1789999999999998e-06 2024-09-17T22:42:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570074+0000", 2024-09-17T22:42:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029197358 2024-09-17T22:42:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570111+0000", 2024-09-17T22:42:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6170000000000001e-05 2024-09-17T22:42:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570149+0000", 2024-09-17T22:42:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8170999999999998e-05 2024-09-17T22:42:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187440 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8527872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.766687498000003, 2024-09-17T22:42:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029359758999999999, 2024-09-17T22:42:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.355 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187440 2024-09-17T22:42:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2700000000000001e-07 2024-09-17T22:42:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540888+0000", 2024-09-17T22:42:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-17T22:42:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540893+0000", 2024-09-17T22:42:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9110000000000001e-06 2024-09-17T22:42:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540895+0000", 2024-09-17T22:42:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.712e-06 2024-09-17T22:42:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570168+0000", 2024-09-17T22:42:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029273114999999999 2024-09-17T22:42:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570208+0000", 2024-09-17T22:42:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9227999999999998e-05 2024-09-17T22:42:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570246+0000", 2024-09-17T22:42:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8253000000000001e-05 2024-09-17T22:42:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.359 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187441 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8617984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.766674534000003, 2024-09-17T22:42:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029442211999999999, 2024-09-17T22:42:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187441 2024-09-17T22:42:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:42:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540900+0000", 2024-09-17T22:42:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7999999999999993e-07 2024-09-17T22:42:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540903+0000", 2024-09-17T22:42:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9129999999999999e-06 2024-09-17T22:42:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540909+0000", 2024-09-17T22:42:45.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4269999999999999e-06 2024-09-17T22:42:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570267+0000", 2024-09-17T22:42:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029358308 2024-09-17T22:42:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570304+0000", 2024-09-17T22:42:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6860999999999998e-05 2024-09-17T22:42:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570341+0000", 2024-09-17T22:42:45.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7509000000000003e-05 2024-09-17T22:42:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187442 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8708096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.766661827, 2024-09-17T22:42:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029522606999999999, 2024-09-17T22:42:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:45.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187442 2024-09-17T22:42:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.22e-07 2024-09-17T22:42:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540917+0000", 2024-09-17T22:42:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999998e-06 2024-09-17T22:42:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540920+0000", 2024-09-17T22:42:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4989999999999999e-06 2024-09-17T22:42:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540923+0000", 2024-09-17T22:42:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3870000000000002e-06 2024-09-17T22:42:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570361+0000", 2024-09-17T22:42:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029438345000000001 2024-09-17T22:42:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570397+0000", 2024-09-17T22:42:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5429000000000002e-05 2024-09-17T22:42:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570434+0000", 2024-09-17T22:42:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7824999999999998e-05 2024-09-17T22:42:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.368 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187443 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8798208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.766647900000002, 2024-09-17T22:42:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029603640000000001, 2024-09-17T22:42:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187443 2024-09-17T22:42:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.369 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.05e-07 2024-09-17T22:42:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540927+0000", 2024-09-17T22:42:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:42:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540930+0000", 2024-09-17T22:42:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9229999999999998e-06 2024-09-17T22:42:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540932+0000", 2024-09-17T22:42:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.294e-06 2024-09-17T22:42:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570454+0000", 2024-09-17T22:42:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029522230999999999 2024-09-17T22:42:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570491+0000", 2024-09-17T22:42:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6835999999999998e-05 2024-09-17T22:42:45.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570529+0000", 2024-09-17T22:42:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8167999999999998e-05 2024-09-17T22:42:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187444 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8888320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.766638426, 2024-09-17T22:42:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029687584999999999, 2024-09-17T22:42:45.373 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187444 2024-09-17T22:42:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-09-17T22:42:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540936+0000", 2024-09-17T22:42:45.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3900000000000003e-07 2024-09-17T22:42:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540939+0000", 2024-09-17T22:42:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.994e-06 2024-09-17T22:42:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540942+0000", 2024-09-17T22:42:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1739999999999999e-06 2024-09-17T22:42:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570549+0000", 2024-09-17T22:42:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029607652000000002 2024-09-17T22:42:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570585+0000", 2024-09-17T22:42:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5775000000000002e-05 2024-09-17T22:42:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570623+0000", 2024-09-17T22:42:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7839999999999997e-05 2024-09-17T22:42:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.377 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187445 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8978432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.766629537999997, 2024-09-17T22:42:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029772837999999999, 2024-09-17T22:42:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187445 2024-09-17T22:42:45.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:42:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540945+0000", 2024-09-17T22:42:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3699999999999999e-07 2024-09-17T22:42:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540948+0000", 2024-09-17T22:42:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.898e-06 2024-09-17T22:42:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540950+0000", 2024-09-17T22:42:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2249999999999999e-06 2024-09-17T22:42:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570643+0000", 2024-09-17T22:42:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029692463999999998 2024-09-17T22:42:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570680+0000", 2024-09-17T22:42:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6772e-05 2024-09-17T22:42:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570717+0000", 2024-09-17T22:42:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7438999999999998e-05 2024-09-17T22:42:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187446 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9068544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:45.382 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.766620799999998, 2024-09-17T22:42:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029856321000000002, 2024-09-17T22:42:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187446 2024-09-17T22:42:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:42:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540958+0000", 2024-09-17T22:42:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3449999999999996e-06 2024-09-17T22:42:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540962+0000", 2024-09-17T22:42:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3529999999999999e-06 2024-09-17T22:42:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540964+0000", 2024-09-17T22:42:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3810000000000002e-06 2024-09-17T22:42:45.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570737+0000", 2024-09-17T22:42:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029772824 2024-09-17T22:42:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570770+0000", 2024-09-17T22:42:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2835e-05 2024-09-17T22:42:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570809+0000", 2024-09-17T22:42:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9379999999999999e-05 2024-09-17T22:42:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187447 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9158656~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.766606832000001, 2024-09-17T22:42:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02993262, 2024-09-17T22:42:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:45.387 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187447 2024-09-17T22:42:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-09-17T22:42:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540968+0000", 2024-09-17T22:42:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6500000000000008e-07 2024-09-17T22:42:45.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540971+0000", 2024-09-17T22:42:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9869999999999999e-06 2024-09-17T22:42:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540973+0000", 2024-09-17T22:42:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3120000000000001e-06 2024-09-17T22:42:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570829+0000", 2024-09-17T22:42:45.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029855742000000001 2024-09-17T22:42:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570861+0000", 2024-09-17T22:42:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2152e-05 2024-09-17T22:42:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570899+0000", 2024-09-17T22:42:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8241999999999997e-05 2024-09-17T22:42:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187448 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9248768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.766597802, 2024-09-17T22:42:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030014865000000002, 2024-09-17T22:42:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:45.392 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187448 2024-09-17T22:42:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9600000000000001e-07 2024-09-17T22:42:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540977+0000", 2024-09-17T22:42:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.16e-07 2024-09-17T22:42:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540980+0000", 2024-09-17T22:42:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.892e-06 2024-09-17T22:42:45.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540982+0000", 2024-09-17T22:42:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3410000000000001e-06 2024-09-17T22:42:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570919+0000", 2024-09-17T22:42:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029936800999999999 2024-09-17T22:42:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570953+0000", 2024-09-17T22:42:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3939000000000002e-05 2024-09-17T22:42:45.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570991+0000", 2024-09-17T22:42:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7780000000000001e-05 2024-09-17T22:42:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187449 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9338880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:45.396 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.766588622999997, 2024-09-17T22:42:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030110238000000001, 2024-09-17T22:42:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187449 2024-09-17T22:42:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:45.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1500000000000001e-07 2024-09-17T22:42:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540986+0000", 2024-09-17T22:42:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9500000000000001e-07 2024-09-17T22:42:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540989+0000", 2024-09-17T22:42:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7949999999999999e-06 2024-09-17T22:42:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540994+0000", 2024-09-17T22:42:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0359999999999997e-06 2024-09-17T22:42:45.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571011+0000", 2024-09-17T22:42:45.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030016915000000002 2024-09-17T22:42:45.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571056+0000", 2024-09-17T22:42:45.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5457999999999997e-05 2024-09-17T22:42:45.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571095+0000", 2024-09-17T22:42:45.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8924000000000002e-05 2024-09-17T22:42:45.401 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.401 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.401 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.401 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187450 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9428992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:45.401 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:45.401 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.766576976000003, 2024-09-17T22:42:45.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030188204999999999, 2024-09-17T22:42:45.401 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:45.401 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:45.401 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:45.401 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:45.401 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:45.402 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:45.402 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187450 2024-09-17T22:42:45.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.402 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:45.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:45.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:45.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:45.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:45.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:45.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:45.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:45.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.16e-07 2024-09-17T22:42:45.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:45.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540998+0000", 2024-09-17T22:42:45.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9000000000000005e-07 2024-09-17T22:42:45.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:45.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.541001+0000", 2024-09-17T22:42:45.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0680000000000001e-06 2024-09-17T22:42:45.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:45.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.541006+0000", 2024-09-17T22:42:45.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4720000000000003e-06 2024-09-17T22:42:45.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:45.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571110+0000", 2024-09-17T22:42:45.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030103431 2024-09-17T22:42:45.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:45.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571147+0000", 2024-09-17T22:42:45.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6748999999999997e-05 2024-09-17T22:42:45.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:45.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:45.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:45.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571185+0000", 2024-09-17T22:42:45.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8278999999999997e-05 2024-09-17T22:42:45.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.405 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:45.405 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:45.405 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:45.406 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:42:45.407 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:42:45.408 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:45.404+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:45.508 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:45.506+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:45.556 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:45.556 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:45.567 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:42:45.610 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:45.608+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:45.670 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:42:45.671 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:45.672 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:45.672 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:45.672 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:45.672 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:45.681 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:42:45.709 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:45.707+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:45.776 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:45.794 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:45.794 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:45.810 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:45.808+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:45.811 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:45.811 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:45.828 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:45.828 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:45.830 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:45.830 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:45.839 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:42:45.845 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:45.845 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:45.862 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:45.862 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:45.879 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:45.879 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:45.896 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:45.896 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:45.899 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:45.899 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:45.941 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:45.938+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:45.944 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:45.945 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:45.953 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:42:46.037 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:46.035+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:46.096 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:46.096 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:46.105 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:42:46.146 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:46.143+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:46.213 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:46.211+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:46.248 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:46.248 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:46.257 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:42:46.314 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:46.312+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:46.364 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:46.364 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:46.373 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:42:46.398 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is set 2024-09-17T22:42:46.414 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:46.412+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:46.515 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:46.513+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:46.524 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:46.525 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:46.535 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:42:46.566 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:42:46.566 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:42:46.566 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T22:42:46.566 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 0.502762 2024-09-17T22:42:46.567 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 0.5027616716034101 2024-09-17T22:42:46.618 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:46.616+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:46.679 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:46.679 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:46.689 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:42:46.729 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:46.727+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:46.818 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:46.816+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:46.832 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:46.832 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:46.840 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:42:46.924 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:46.922+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:46.984 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:46.984 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:46.993 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:42:47.020 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:47.018+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:47.121 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:47.119+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:47.136 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:47.137 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:47.145 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:42:47.229 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:47.227+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:47.289 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:47.289 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:47.298 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:42:47.323 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:47.321+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:47.404 INFO:teuthology.orchestra.run.smithi115.stderr:set osd.3 primary-affinity to 0.502762 (8329482) 2024-09-17T22:42:47.424 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:47.421+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:47.450 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:47.450 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:47.460 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:42:47.544 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:47.542+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:47.604 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:47.604 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:47.612 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:42:47.625 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:47.623+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:47.726 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:47.724+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:47.755 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:47.756 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:47.765 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:42:47.827 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:47.824+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:47.867 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:47.868 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:47.876 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:42:47.900 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:42:47.927 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:47.925+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:47.979 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:47.979 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:47.987 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:42:48.006 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:48.025 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:48.026 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:48.028 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:48.026+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:48.042 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:48.043 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:48.059 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:48.059 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:48.076 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:48.076 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:48.093 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:48.094 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:48.096 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:48.096 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:48.096 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:48.096 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:48.106 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:42:48.110 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:48.111 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:48.127 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:48.128 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:48.129 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:48.127+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:48.131 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:48.132 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:48.213 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:48.213 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:48.213 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:48.213 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:48.214 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:48.222 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:42:48.230 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:48.227+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:48.326 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:48.326 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:48.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:42:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.96449440799999, 2024-09-17T22:42:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.249158453, 2024-09-17T22:42:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:42:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:42:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-17T22:42:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358680+0000", 2024-09-17T22:42:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.592e-06 2024-09-17T22:42:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358688+0000", 2024-09-17T22:42:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1610000000000003e-06 2024-09-17T22:42:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358697+0000", 2024-09-17T22:42:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2580000000000001e-06 2024-09-17T22:42:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358706+0000", 2024-09-17T22:42:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1640000000000007e-06 2024-09-17T22:42:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358766+0000", 2024-09-17T22:42:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9893999999999999e-05 2024-09-17T22:42:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:42:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358778+0000", 2024-09-17T22:42:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1148999999999999e-05 2024-09-17T22:42:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606347+0000", 2024-09-17T22:42:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.247569971 2024-09-17T22:42:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606533+0000", 2024-09-17T22:42:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018599400000000001 2024-09-17T22:42:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:42:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606752+0000", 2024-09-17T22:42:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000218373 2024-09-17T22:42:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:42:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607455+0000", 2024-09-17T22:42:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070311799999999997 2024-09-17T22:42:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607735+0000", 2024-09-17T22:42:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027972300000000001 2024-09-17T22:42:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:42:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607793+0000", 2024-09-17T22:42:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8529999999999997e-05 2024-09-17T22:42:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607837+0000", 2024-09-17T22:42:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3526000000000003e-05 2024-09-17T22:42:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186749 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 11960320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.899853086, 2024-09-17T22:42:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029226166000000001, 2024-09-17T22:42:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186749 2024-09-17T22:42:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-09-17T22:42:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423328+0000", 2024-09-17T22:42:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6790000000000005e-06 2024-09-17T22:42:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423334+0000", 2024-09-17T22:42:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0009999999999998e-06 2024-09-17T22:42:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423338+0000", 2024-09-17T22:42:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6419999999999998e-06 2024-09-17T22:42:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452464+0000", 2024-09-17T22:42:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029125389000000002 2024-09-17T22:42:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452501+0000", 2024-09-17T22:42:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7656e-05 2024-09-17T22:42:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452546+0000", 2024-09-17T22:42:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4468999999999997e-05 2024-09-17T22:42:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186750 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12050432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.899827295999998, 2024-09-17T22:42:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029307863999999999, 2024-09-17T22:42:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186750 2024-09-17T22:42:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423346+0000", 2024-09-17T22:42:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.84e-07 2024-09-17T22:42:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423348+0000", 2024-09-17T22:42:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7540000000000001e-06 2024-09-17T22:42:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423353+0000", 2024-09-17T22:42:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:42:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423362+0000", 2024-09-17T22:42:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9220000000000003e-06 2024-09-17T22:42:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452571+0000", 2024-09-17T22:42:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029209174000000001 2024-09-17T22:42:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452612+0000", 2024-09-17T22:42:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1363999999999999e-05 2024-09-17T22:42:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452653+0000", 2024-09-17T22:42:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0893999999999998e-05 2024-09-17T22:42:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186751 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12140544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.899804805999999, 2024-09-17T22:42:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029387488, 2024-09-17T22:42:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186751 2024-09-17T22:42:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1399999999999997e-07 2024-09-17T22:42:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423375+0000", 2024-09-17T22:42:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8889999999999996e-06 2024-09-17T22:42:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423381+0000", 2024-09-17T22:42:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5690000000000004e-06 2024-09-17T22:42:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423386+0000", 2024-09-17T22:42:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9559999999999996e-06 2024-09-17T22:42:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452679+0000", 2024-09-17T22:42:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029292820000000001 2024-09-17T22:42:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452718+0000", 2024-09-17T22:42:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8961000000000002e-05 2024-09-17T22:42:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452755+0000", 2024-09-17T22:42:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7879000000000001e-05 2024-09-17T22:42:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186752 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12230656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.899773555000003, 2024-09-17T22:42:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029463269, 2024-09-17T22:42:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186752 2024-09-17T22:42:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423400+0000", 2024-09-17T22:42:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:42:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423401+0000", 2024-09-17T22:42:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3599999999999999e-06 2024-09-17T22:42:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423404+0000", 2024-09-17T22:42:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5999999999999998e-06 2024-09-17T22:42:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423407+0000", 2024-09-17T22:42:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6869999999999999e-06 2024-09-17T22:42:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452782+0000", 2024-09-17T22:42:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029375107000000001 2024-09-17T22:42:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452821+0000", 2024-09-17T22:42:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9122999999999998e-05 2024-09-17T22:42:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452862+0000", 2024-09-17T22:42:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1019000000000001e-05 2024-09-17T22:42:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186753 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.899761533000003, 2024-09-17T22:42:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029553297999999999, 2024-09-17T22:42:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186753 2024-09-17T22:42:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.98e-07 2024-09-17T22:42:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423412+0000", 2024-09-17T22:42:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8700000000000004e-07 2024-09-17T22:42:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423415+0000", 2024-09-17T22:42:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:42:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423418+0000", 2024-09-17T22:42:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.283e-06 2024-09-17T22:42:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452893+0000", 2024-09-17T22:42:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029475881999999998 2024-09-17T22:42:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452928+0000", 2024-09-17T22:42:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4109000000000002e-05 2024-09-17T22:42:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452964+0000", 2024-09-17T22:42:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6937000000000002e-05 2024-09-17T22:42:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186754 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12410880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.899752395999997, 2024-09-17T22:42:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029656021000000001, 2024-09-17T22:42:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186754 2024-09-17T22:42:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423421+0000", 2024-09-17T22:42:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9999999999999999e-07 2024-09-17T22:42:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423425+0000", 2024-09-17T22:42:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6519999999999997e-06 2024-09-17T22:42:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423428+0000", 2024-09-17T22:42:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1370000000000002e-06 2024-09-17T22:42:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423431+0000", 2024-09-17T22:42:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4090000000000001e-06 2024-09-17T22:42:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452992+0000", 2024-09-17T22:42:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029561545000000002 2024-09-17T22:42:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453031+0000", 2024-09-17T22:42:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8834000000000001e-05 2024-09-17T22:42:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453076+0000", 2024-09-17T22:42:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5244000000000003e-05 2024-09-17T22:42:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186755 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12500992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.899739115000003, 2024-09-17T22:42:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029745649999999998, 2024-09-17T22:42:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186755 2024-09-17T22:42:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.17e-07 2024-09-17T22:42:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423435+0000", 2024-09-17T22:42:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:42:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423438+0000", 2024-09-17T22:42:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8210000000000002e-06 2024-09-17T22:42:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423443+0000", 2024-09-17T22:42:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1070000000000004e-06 2024-09-17T22:42:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453103+0000", 2024-09-17T22:42:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029660399 2024-09-17T22:42:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453141+0000", 2024-09-17T22:42:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.748e-05 2024-09-17T22:42:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453179+0000", 2024-09-17T22:42:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8671000000000003e-05 2024-09-17T22:42:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187439 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8437760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.782302018000003, 2024-09-17T22:42:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029278048000000001, 2024-09-17T22:42:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187439 2024-09-17T22:42:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6199999999999999e-07 2024-09-17T22:42:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540872+0000", 2024-09-17T22:42:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.09e-07 2024-09-17T22:42:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540875+0000", 2024-09-17T22:42:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8990000000000001e-06 2024-09-17T22:42:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540877+0000", 2024-09-17T22:42:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1789999999999998e-06 2024-09-17T22:42:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570074+0000", 2024-09-17T22:42:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029197358 2024-09-17T22:42:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570111+0000", 2024-09-17T22:42:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6170000000000001e-05 2024-09-17T22:42:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570149+0000", 2024-09-17T22:42:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8170999999999998e-05 2024-09-17T22:42:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187440 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8527872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.782286548000002, 2024-09-17T22:42:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029359758999999999, 2024-09-17T22:42:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187440 2024-09-17T22:42:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2700000000000001e-07 2024-09-17T22:42:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540888+0000", 2024-09-17T22:42:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-17T22:42:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540893+0000", 2024-09-17T22:42:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9110000000000001e-06 2024-09-17T22:42:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540895+0000", 2024-09-17T22:42:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.712e-06 2024-09-17T22:42:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570168+0000", 2024-09-17T22:42:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029273114999999999 2024-09-17T22:42:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570208+0000", 2024-09-17T22:42:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9227999999999998e-05 2024-09-17T22:42:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570246+0000", 2024-09-17T22:42:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8253000000000001e-05 2024-09-17T22:42:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187441 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8617984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.782273584000002, 2024-09-17T22:42:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029442211999999999, 2024-09-17T22:42:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187441 2024-09-17T22:42:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:42:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540900+0000", 2024-09-17T22:42:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7999999999999993e-07 2024-09-17T22:42:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540903+0000", 2024-09-17T22:42:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9129999999999999e-06 2024-09-17T22:42:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540909+0000", 2024-09-17T22:42:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4269999999999999e-06 2024-09-17T22:42:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570267+0000", 2024-09-17T22:42:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029358308 2024-09-17T22:42:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570304+0000", 2024-09-17T22:42:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6860999999999998e-05 2024-09-17T22:42:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570341+0000", 2024-09-17T22:42:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7509000000000003e-05 2024-09-17T22:42:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187442 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8708096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.782260876999999, 2024-09-17T22:42:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029522606999999999, 2024-09-17T22:42:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187442 2024-09-17T22:42:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.22e-07 2024-09-17T22:42:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540917+0000", 2024-09-17T22:42:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999998e-06 2024-09-17T22:42:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540920+0000", 2024-09-17T22:42:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4989999999999999e-06 2024-09-17T22:42:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540923+0000", 2024-09-17T22:42:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3870000000000002e-06 2024-09-17T22:42:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570361+0000", 2024-09-17T22:42:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029438345000000001 2024-09-17T22:42:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570397+0000", 2024-09-17T22:42:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5429000000000002e-05 2024-09-17T22:42:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570434+0000", 2024-09-17T22:42:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7824999999999998e-05 2024-09-17T22:42:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187443 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8798208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.782246950000001, 2024-09-17T22:42:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029603640000000001, 2024-09-17T22:42:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187443 2024-09-17T22:42:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.05e-07 2024-09-17T22:42:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540927+0000", 2024-09-17T22:42:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:42:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540930+0000", 2024-09-17T22:42:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9229999999999998e-06 2024-09-17T22:42:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540932+0000", 2024-09-17T22:42:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.294e-06 2024-09-17T22:42:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570454+0000", 2024-09-17T22:42:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029522230999999999 2024-09-17T22:42:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570491+0000", 2024-09-17T22:42:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6835999999999998e-05 2024-09-17T22:42:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570529+0000", 2024-09-17T22:42:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8167999999999998e-05 2024-09-17T22:42:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187444 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8888320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.782237475999999, 2024-09-17T22:42:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029687584999999999, 2024-09-17T22:42:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187444 2024-09-17T22:42:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-09-17T22:42:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540936+0000", 2024-09-17T22:42:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3900000000000003e-07 2024-09-17T22:42:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540939+0000", 2024-09-17T22:42:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.994e-06 2024-09-17T22:42:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540942+0000", 2024-09-17T22:42:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1739999999999999e-06 2024-09-17T22:42:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570549+0000", 2024-09-17T22:42:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029607652000000002 2024-09-17T22:42:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570585+0000", 2024-09-17T22:42:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5775000000000002e-05 2024-09-17T22:42:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570623+0000", 2024-09-17T22:42:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7839999999999997e-05 2024-09-17T22:42:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187445 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8978432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.782228588000002, 2024-09-17T22:42:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029772837999999999, 2024-09-17T22:42:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187445 2024-09-17T22:42:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:42:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540945+0000", 2024-09-17T22:42:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3699999999999999e-07 2024-09-17T22:42:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540948+0000", 2024-09-17T22:42:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.898e-06 2024-09-17T22:42:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540950+0000", 2024-09-17T22:42:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2249999999999999e-06 2024-09-17T22:42:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570643+0000", 2024-09-17T22:42:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029692463999999998 2024-09-17T22:42:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570680+0000", 2024-09-17T22:42:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6772e-05 2024-09-17T22:42:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570717+0000", 2024-09-17T22:42:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7438999999999998e-05 2024-09-17T22:42:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187446 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9068544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.782219849999997, 2024-09-17T22:42:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029856321000000002, 2024-09-17T22:42:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187446 2024-09-17T22:42:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:42:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540958+0000", 2024-09-17T22:42:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3449999999999996e-06 2024-09-17T22:42:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540962+0000", 2024-09-17T22:42:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3529999999999999e-06 2024-09-17T22:42:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540964+0000", 2024-09-17T22:42:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3810000000000002e-06 2024-09-17T22:42:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570737+0000", 2024-09-17T22:42:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029772824 2024-09-17T22:42:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570770+0000", 2024-09-17T22:42:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2835e-05 2024-09-17T22:42:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570809+0000", 2024-09-17T22:42:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9379999999999999e-05 2024-09-17T22:42:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187447 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9158656~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.782205882, 2024-09-17T22:42:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02993262, 2024-09-17T22:42:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187447 2024-09-17T22:42:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-09-17T22:42:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540968+0000", 2024-09-17T22:42:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6500000000000008e-07 2024-09-17T22:42:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540971+0000", 2024-09-17T22:42:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9869999999999999e-06 2024-09-17T22:42:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540973+0000", 2024-09-17T22:42:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3120000000000001e-06 2024-09-17T22:42:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570829+0000", 2024-09-17T22:42:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029855742000000001 2024-09-17T22:42:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570861+0000", 2024-09-17T22:42:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2152e-05 2024-09-17T22:42:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570899+0000", 2024-09-17T22:42:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8241999999999997e-05 2024-09-17T22:42:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187448 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9248768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.782196851999998, 2024-09-17T22:42:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030014865000000002, 2024-09-17T22:42:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187448 2024-09-17T22:42:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9600000000000001e-07 2024-09-17T22:42:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540977+0000", 2024-09-17T22:42:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.16e-07 2024-09-17T22:42:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540980+0000", 2024-09-17T22:42:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.892e-06 2024-09-17T22:42:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540982+0000", 2024-09-17T22:42:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3410000000000001e-06 2024-09-17T22:42:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570919+0000", 2024-09-17T22:42:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029936800999999999 2024-09-17T22:42:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570953+0000", 2024-09-17T22:42:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3939000000000002e-05 2024-09-17T22:42:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570991+0000", 2024-09-17T22:42:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7780000000000001e-05 2024-09-17T22:42:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187449 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9338880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.782187673000003, 2024-09-17T22:42:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030110238000000001, 2024-09-17T22:42:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187449 2024-09-17T22:42:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1500000000000001e-07 2024-09-17T22:42:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540986+0000", 2024-09-17T22:42:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9500000000000001e-07 2024-09-17T22:42:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540989+0000", 2024-09-17T22:42:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7949999999999999e-06 2024-09-17T22:42:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540994+0000", 2024-09-17T22:42:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0359999999999997e-06 2024-09-17T22:42:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571011+0000", 2024-09-17T22:42:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030016915000000002 2024-09-17T22:42:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571056+0000", 2024-09-17T22:42:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5457999999999997e-05 2024-09-17T22:42:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571095+0000", 2024-09-17T22:42:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8924000000000002e-05 2024-09-17T22:42:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187450 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9428992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.782176026000002, 2024-09-17T22:42:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030188204999999999, 2024-09-17T22:42:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187450 2024-09-17T22:42:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.16e-07 2024-09-17T22:42:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540998+0000", 2024-09-17T22:42:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9000000000000005e-07 2024-09-17T22:42:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.541001+0000", 2024-09-17T22:42:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0680000000000001e-06 2024-09-17T22:42:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.541006+0000", 2024-09-17T22:42:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4720000000000003e-06 2024-09-17T22:42:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571110+0000", 2024-09-17T22:42:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030103431 2024-09-17T22:42:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571147+0000", 2024-09-17T22:42:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6748999999999997e-05 2024-09-17T22:42:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571185+0000", 2024-09-17T22:42:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8278999999999997e-05 2024-09-17T22:42:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.425 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:48.426 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:42:48.428 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:42:48.428 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:48.424+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:48.527 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:48.525+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:48.574 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:48.574 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:48.575 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:48.575 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:48.584 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:42:48.628 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:48.626+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:48.688 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:48.688 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:48.688 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:48.688 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:48.688 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:48.697 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:42:48.729 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:48.727+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:48.800 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:48.800 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145246 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3850240~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.467551143999998, 2024-09-17T22:42:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052644991000000002, 2024-09-17T22:42:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.801 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145246 2024-09-17T22:42:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:48.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329899+0000", 2024-09-17T22:42:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-09-17T22:42:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329901+0000", 2024-09-17T22:42:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1619999999999998e-06 2024-09-17T22:42:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329907+0000", 2024-09-17T22:42:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9059999999999999e-06 2024-09-17T22:42:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329911+0000", 2024-09-17T22:42:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7419999999999997e-06 2024-09-17T22:42:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382383+0000", 2024-09-17T22:42:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052471583000000002 2024-09-17T22:42:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382457+0000", 2024-09-17T22:42:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4309999999999998e-05 2024-09-17T22:42:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382543+0000", 2024-09-17T22:42:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5918e-05 2024-09-17T22:42:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.805 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145247 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3946496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.467532493999997, 2024-09-17T22:42:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052805863000000001, 2024-09-17T22:42:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.806 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145247 2024-09-17T22:42:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3200000000000001e-07 2024-09-17T22:42:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329919+0000", 2024-09-17T22:42:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.888e-06 2024-09-17T22:42:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329925+0000", 2024-09-17T22:42:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5369999999999997e-06 2024-09-17T22:42:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329929+0000", 2024-09-17T22:42:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.639e-06 2024-09-17T22:42:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382570+0000", 2024-09-17T22:42:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052641213999999999 2024-09-17T22:42:48.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382636+0000", 2024-09-17T22:42:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6008999999999998e-05 2024-09-17T22:42:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382723+0000", 2024-09-17T22:42:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6243999999999996e-05 2024-09-17T22:42:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.810 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145248 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3856384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.467514446000003, 2024-09-17T22:42:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053002858, 2024-09-17T22:42:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.811 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145248 2024-09-17T22:42:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.13e-07 2024-09-17T22:42:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329937+0000", 2024-09-17T22:42:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9300000000000002e-06 2024-09-17T22:42:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329943+0000", 2024-09-17T22:42:48.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-09-17T22:42:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329948+0000", 2024-09-17T22:42:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8579999999999999e-06 2024-09-17T22:42:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382763+0000", 2024-09-17T22:42:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052814537000000002 2024-09-17T22:42:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382840+0000", 2024-09-17T22:42:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6879999999999996e-05 2024-09-17T22:42:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382938+0000", 2024-09-17T22:42:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8159999999999995e-05 2024-09-17T22:42:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.815 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145249 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3952640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.467495544999998, 2024-09-17T22:42:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053188602000000001, 2024-09-17T22:42:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145249 2024-09-17T22:42:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.816 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:48.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:48.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3700000000000001e-07 2024-09-17T22:42:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329956+0000", 2024-09-17T22:42:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1430000000000001e-06 2024-09-17T22:42:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329962+0000", 2024-09-17T22:42:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8270000000000003e-06 2024-09-17T22:42:48.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329967+0000", 2024-09-17T22:42:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7249999999999997e-06 2024-09-17T22:42:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382969+0000", 2024-09-17T22:42:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053001952999999997 2024-09-17T22:42:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383040+0000", 2024-09-17T22:42:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1272999999999993e-05 2024-09-17T22:42:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383142+0000", 2024-09-17T22:42:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010234399999999999 2024-09-17T22:42:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145250 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3862528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.467476746000003, 2024-09-17T22:42:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053359537999999998, 2024-09-17T22:42:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145250 2024-09-17T22:42:48.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:42:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329975+0000", 2024-09-17T22:42:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2199999999999999e-06 2024-09-17T22:42:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329981+0000", 2024-09-17T22:42:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1020000000000002e-06 2024-09-17T22:42:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329991+0000", 2024-09-17T22:42:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6800000000000005e-06 2024-09-17T22:42:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383171+0000", 2024-09-17T22:42:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053180142 2024-09-17T22:42:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383242+0000", 2024-09-17T22:42:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1310000000000007e-05 2024-09-17T22:42:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383332+0000", 2024-09-17T22:42:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9703999999999994e-05 2024-09-17T22:42:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.825 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145251 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3958784~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.467451046000001, 2024-09-17T22:42:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053438537000000001, 2024-09-17T22:42:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.826 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145251 2024-09-17T22:42:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329999+0000", 2024-09-17T22:42:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-07 2024-09-17T22:42:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330001+0000", 2024-09-17T22:42:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:42:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330006+0000", 2024-09-17T22:42:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000004e-06 2024-09-17T22:42:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330011+0000", 2024-09-17T22:42:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4039999999999997e-06 2024-09-17T22:42:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383350+0000", 2024-09-17T22:42:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053339011999999998 2024-09-17T22:42:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383387+0000", 2024-09-17T22:42:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7085999999999997e-05 2024-09-17T22:42:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383437+0000", 2024-09-17T22:42:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9988000000000003e-05 2024-09-17T22:42:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145252 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3868672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.467432598999999, 2024-09-17T22:42:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053526724999999997, 2024-09-17T22:42:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145252 2024-09-17T22:42:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331054+0000", 2024-09-17T22:42:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010371670000000001 2024-09-17T22:42:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331059+0000", 2024-09-17T22:42:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1680000000000003e-06 2024-09-17T22:42:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331075+0000", 2024-09-17T22:42:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6028e-05 2024-09-17T22:42:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331084+0000", 2024-09-17T22:42:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3980000000000005e-06 2024-09-17T22:42:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383464+0000", 2024-09-17T22:42:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052379851999999998 2024-09-17T22:42:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383499+0000", 2024-09-17T22:42:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4610000000000002e-05 2024-09-17T22:42:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383543+0000", 2024-09-17T22:42:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4502000000000002e-05 2024-09-17T22:42:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145255 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3971072~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.466302507000002, 2024-09-17T22:42:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052677527000000002, 2024-09-17T22:42:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145255 2024-09-17T22:42:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8800000000000002e-07 2024-09-17T22:42:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331150+0000", 2024-09-17T22:42:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1160000000000002e-06 2024-09-17T22:42:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331156+0000", 2024-09-17T22:42:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7469999999999999e-06 2024-09-17T22:42:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331162+0000", 2024-09-17T22:42:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-17T22:42:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383750+0000", 2024-09-17T22:42:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052588441999999999 2024-09-17T22:42:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383782+0000", 2024-09-17T22:42:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1610999999999999e-05 2024-09-17T22:42:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383824+0000", 2024-09-17T22:42:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2512000000000002e-05 2024-09-17T22:42:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145256 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3880960~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.466281086000002, 2024-09-17T22:42:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052744775000000001, 2024-09-17T22:42:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145256 2024-09-17T22:42:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331169+0000", 2024-09-17T22:42:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:42:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331175+0000", 2024-09-17T22:42:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5660000000000003e-06 2024-09-17T22:42:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331182+0000", 2024-09-17T22:42:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0169999999999999e-06 2024-09-17T22:42:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331188+0000", 2024-09-17T22:42:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2750000000000003e-06 2024-09-17T22:42:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383839+0000", 2024-09-17T22:42:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052651728000000002 2024-09-17T22:42:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383872+0000", 2024-09-17T22:42:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2347e-05 2024-09-17T22:42:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383913+0000", 2024-09-17T22:42:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1362000000000001e-05 2024-09-17T22:42:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145257 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3977216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.466255038, 2024-09-17T22:42:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052809598999999999, 2024-09-17T22:42:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145257 2024-09-17T22:42:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333051+0000", 2024-09-17T22:42:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018567189999999999 2024-09-17T22:42:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333056+0000", 2024-09-17T22:42:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-17T22:42:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333067+0000", 2024-09-17T22:42:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1704e-05 2024-09-17T22:42:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333074+0000", 2024-09-17T22:42:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1280000000000002e-06 2024-09-17T22:42:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383927+0000", 2024-09-17T22:42:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050853094000000001 2024-09-17T22:42:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383960+0000", 2024-09-17T22:42:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2891000000000001e-05 2024-09-17T22:42:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.384004+0000", 2024-09-17T22:42:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3565e-05 2024-09-17T22:42:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146168 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2467840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.402766417000002, 2024-09-17T22:42:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052652501999999997, 2024-09-17T22:42:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146168 2024-09-17T22:42:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394989+0000", 2024-09-17T22:42:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030618300000000002 2024-09-17T22:42:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394994+0000", 2024-09-17T22:42:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.566e-06 2024-09-17T22:42:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395003+0000", 2024-09-17T22:42:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3619999999999995e-06 2024-09-17T22:42:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395009+0000", 2024-09-17T22:42:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3589999999999996e-06 2024-09-17T22:42:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447180+0000", 2024-09-17T22:42:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052170643000000003 2024-09-17T22:42:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447251+0000", 2024-09-17T22:42:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1382000000000002e-05 2024-09-17T22:42:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447335+0000", 2024-09-17T22:42:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4006999999999997e-05 2024-09-17T22:42:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146172 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2480128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.402411532000002, 2024-09-17T22:42:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052714629999999998, 2024-09-17T22:42:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146172 2024-09-17T22:42:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395486+0000", 2024-09-17T22:42:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044787000000000001 2024-09-17T22:42:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395491+0000", 2024-09-17T22:42:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8550000000000001e-06 2024-09-17T22:42:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395503+0000", 2024-09-17T22:42:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2271e-05 2024-09-17T22:42:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395510+0000", 2024-09-17T22:42:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6320000000000008e-06 2024-09-17T22:42:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447565+0000", 2024-09-17T22:42:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052055016000000003 2024-09-17T22:42:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447644+0000", 2024-09-17T22:42:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8306000000000005e-05 2024-09-17T22:42:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447752+0000", 2024-09-17T22:42:48.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010868 2024-09-17T22:42:48.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.859 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.859 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146181 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2504704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.859 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:48.859 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.401544141999999, 2024-09-17T22:42:48.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052850394000000002, 2024-09-17T22:42:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146181 2024-09-17T22:42:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396210+0000", 2024-09-17T22:42:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000305214 2024-09-17T22:42:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396214+0000", 2024-09-17T22:42:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9389999999999999e-06 2024-09-17T22:42:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396228+0000", 2024-09-17T22:42:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3365e-05 2024-09-17T22:42:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396235+0000", 2024-09-17T22:42:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2559999999999996e-06 2024-09-17T22:42:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448568+0000", 2024-09-17T22:42:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052333084000000002 2024-09-17T22:42:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448646+0000", 2024-09-17T22:42:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8287000000000004e-05 2024-09-17T22:42:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448756+0000", 2024-09-17T22:42:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109249 2024-09-17T22:42:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146183 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2510848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.401203594999998, 2024-09-17T22:42:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052725453999999998, 2024-09-17T22:42:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146183 2024-09-17T22:42:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396508+0000", 2024-09-17T22:42:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026271299999999998 2024-09-17T22:42:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396512+0000", 2024-09-17T22:42:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8039999999999999e-06 2024-09-17T22:42:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396522+0000", 2024-09-17T22:42:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9289999999999993e-06 2024-09-17T22:42:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396528+0000", 2024-09-17T22:42:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2990000000000002e-06 2024-09-17T22:42:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448798+0000", 2024-09-17T22:42:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052269985999999997 2024-09-17T22:42:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448871+0000", 2024-09-17T22:42:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2581000000000002e-05 2024-09-17T22:42:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448971+0000", 2024-09-17T22:42:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100142 2024-09-17T22:42:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146391 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.393476102999998, 2024-09-17T22:42:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052666719000000001, 2024-09-17T22:42:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146391 2024-09-17T22:42:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404098+0000", 2024-09-17T22:42:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000124509 2024-09-17T22:42:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404107+0000", 2024-09-17T22:42:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4220000000000006e-06 2024-09-17T22:42:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404117+0000", 2024-09-17T22:42:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011e-05 2024-09-17T22:42:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404123+0000", 2024-09-17T22:42:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1530000000000002e-06 2024-09-17T22:42:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456481+0000", 2024-09-17T22:42:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052357923000000001 2024-09-17T22:42:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456551+0000", 2024-09-17T22:42:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9869000000000005e-05 2024-09-17T22:42:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456640+0000", 2024-09-17T22:42:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8733000000000006e-05 2024-09-17T22:42:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146393 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2748416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.393316040000002, 2024-09-17T22:42:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052739239, 2024-09-17T22:42:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146393 2024-09-17T22:42:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404443+0000", 2024-09-17T22:42:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030977199999999999 2024-09-17T22:42:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404447+0000", 2024-09-17T22:42:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6370000000000001e-06 2024-09-17T22:42:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404469+0000", 2024-09-17T22:42:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2739999999999999e-05 2024-09-17T22:42:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404477+0000", 2024-09-17T22:42:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6559999999999992e-06 2024-09-17T22:42:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456734+0000", 2024-09-17T22:42:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052256889000000001 2024-09-17T22:42:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456796+0000", 2024-09-17T22:42:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2485999999999998e-05 2024-09-17T22:42:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456873+0000", 2024-09-17T22:42:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6058999999999998e-05 2024-09-17T22:42:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146397 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2760704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.392940275999997, 2024-09-17T22:42:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052788928999999998, 2024-09-17T22:42:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146397 2024-09-17T22:42:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404763+0000", 2024-09-17T22:42:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025351399999999999 2024-09-17T22:42:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404767+0000", 2024-09-17T22:42:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:42:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404777+0000", 2024-09-17T22:42:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9310000000000003e-06 2024-09-17T22:42:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404783+0000", 2024-09-17T22:42:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.336e-06 2024-09-17T22:42:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457126+0000", 2024-09-17T22:42:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052342975999999999 2024-09-17T22:42:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457200+0000", 2024-09-17T22:42:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4146000000000005e-05 2024-09-17T22:42:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457298+0000", 2024-09-17T22:42:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7807999999999996e-05 2024-09-17T22:42:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146399 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2766848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.392650134999997, 2024-09-17T22:42:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052829779, 2024-09-17T22:42:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146399 2024-09-17T22:42:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404800+0000", 2024-09-17T22:42:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8500000000000001e-07 2024-09-17T22:42:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404802+0000", 2024-09-17T22:42:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5349999999999999e-06 2024-09-17T22:42:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404811+0000", 2024-09-17T22:42:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9390000000000003e-06 2024-09-17T22:42:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404817+0000", 2024-09-17T22:42:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.524e-06 2024-09-17T22:42:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457461+0000", 2024-09-17T22:42:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052643784999999998 2024-09-17T22:42:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457539+0000", 2024-09-17T22:42:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8589000000000003e-05 2024-09-17T22:42:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457629+0000", 2024-09-17T22:42:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9821999999999995e-05 2024-09-17T22:42:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146401 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2772992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.392624888, 2024-09-17T22:42:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052910077, 2024-09-17T22:42:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146401 2024-09-17T22:42:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405106+0000", 2024-09-17T22:42:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000281286 2024-09-17T22:42:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405110+0000", 2024-09-17T22:42:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1459999999999998e-06 2024-09-17T22:42:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405120+0000", 2024-09-17T22:42:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0546e-05 2024-09-17T22:42:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405127+0000", 2024-09-17T22:42:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4300000000000003e-06 2024-09-17T22:42:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457647+0000", 2024-09-17T22:42:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052520229000000002 2024-09-17T22:42:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457684+0000", 2024-09-17T22:42:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6609999999999997e-05 2024-09-17T22:42:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457735+0000", 2024-09-17T22:42:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.083e-05 2024-09-17T22:42:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146402 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2779136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.392311476000003, 2024-09-17T22:42:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052689812000000003, 2024-09-17T22:42:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146402 2024-09-17T22:42:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405365+0000", 2024-09-17T22:42:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022744599999999999 2024-09-17T22:42:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405370+0000", 2024-09-17T22:42:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2769999999999999e-06 2024-09-17T22:42:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405380+0000", 2024-09-17T22:42:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0349e-05 2024-09-17T22:42:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405386+0000", 2024-09-17T22:42:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5169999999999996e-06 2024-09-17T22:42:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457750+0000", 2024-09-17T22:42:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052363448999999999 2024-09-17T22:42:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457784+0000", 2024-09-17T22:42:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4363999999999998e-05 2024-09-17T22:42:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457828+0000", 2024-09-17T22:42:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999999e-05 2024-09-17T22:42:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.897 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.897 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:48.897 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:48.897 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:48.898 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:42:48.899 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:48.896+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:48.999 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:48.997+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:49.043 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:49.043 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:49.043 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:49.043 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:49.052 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:42:49.100 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:49.098+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:49.153 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:49.153 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:49.153 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:49.153 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:49.153 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:49.161 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:42:49.201 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:49.198+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:49.265 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4229.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-17T22:42:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 99.867975755000003, 2024-09-17T22:42:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25708877200000002, 2024-09-17T22:42:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.266 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4229", 2024-09-17T22:42:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/3080427038", 2024-09-17T22:42:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2 2024-09-17T22:42:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394611+0000", 2024-09-17T22:42:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3149999999999999e-06 2024-09-17T22:42:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394619+0000", 2024-09-17T22:42:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240000000000003e-06 2024-09-17T22:42:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394632+0000", 2024-09-17T22:42:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2921e-05 2024-09-17T22:42:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394639+0000", 2024-09-17T22:42:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-09-17T22:42:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394703+0000", 2024-09-17T22:42:49.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4407999999999997e-05 2024-09-17T22:42:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:42:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394711+0000", 2024-09-17T22:42:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5939999999999996e-06 2024-09-17T22:42:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651545+0000", 2024-09-17T22:42:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25683409200000001 2024-09-17T22:42:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651698+0000", 2024-09-17T22:42:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000153678 2024-09-17T22:42:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38736 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.485379750000007, 2024-09-17T22:42:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051395418999999998, 2024-09-17T22:42:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.271 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38736 2024-09-17T22:42:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6899999999999998e-07 2024-09-17T22:42:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777212+0000", 2024-09-17T22:42:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1240000000000002e-06 2024-09-17T22:42:49.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777219+0000", 2024-09-17T22:42:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5409999999999997e-06 2024-09-17T22:42:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777224+0000", 2024-09-17T22:42:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:42:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.826131+0000", 2024-09-17T22:42:49.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048907369999999999 2024-09-17T22:42:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828457+0000", 2024-09-17T22:42:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023256209999999999 2024-09-17T22:42:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828601+0000", 2024-09-17T22:42:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014438799999999999 2024-09-17T22:42:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.275 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38752 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 673792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.485355678999994, 2024-09-17T22:42:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051582301999999997, 2024-09-17T22:42:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.276 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38752 2024-09-17T22:42:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3000000000000001e-07 2024-09-17T22:42:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777232+0000", 2024-09-17T22:42:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9709999999999998e-06 2024-09-17T22:42:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777238+0000", 2024-09-17T22:42:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7209999999999999e-06 2024-09-17T22:42:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777243+0000", 2024-09-17T22:42:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5210000000000004e-06 2024-09-17T22:42:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828633+0000", 2024-09-17T22:42:49.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051390915000000002 2024-09-17T22:42:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828711+0000", 2024-09-17T22:42:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7474999999999999e-05 2024-09-17T22:42:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828812+0000", 2024-09-17T22:42:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010116900000000001 2024-09-17T22:42:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.280 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38753 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.485336406000002, 2024-09-17T22:42:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051770444999999998, 2024-09-17T22:42:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:49.281 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38753 2024-09-17T22:42:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:42:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777251+0000", 2024-09-17T22:42:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8059999999999999e-06 2024-09-17T22:42:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777257+0000", 2024-09-17T22:42:49.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2480000000000001e-06 2024-09-17T22:42:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777262+0000", 2024-09-17T22:42:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000004e-06 2024-09-17T22:42:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828842+0000", 2024-09-17T22:42:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051580203999999998 2024-09-17T22:42:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:49.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828917+0000", 2024-09-17T22:42:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4980000000000004e-05 2024-09-17T22:42:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829019+0000", 2024-09-17T22:42:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102248 2024-09-17T22:42:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38766 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:49.285 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.485317421000005, 2024-09-17T22:42:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051958839999999999, 2024-09-17T22:42:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38766 2024-09-17T22:42:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777269+0000", 2024-09-17T22:42:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:42:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777271+0000", 2024-09-17T22:42:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-17T22:42:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777276+0000", 2024-09-17T22:42:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:42:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777281+0000", 2024-09-17T22:42:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:42:49.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829055+0000", 2024-09-17T22:42:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051773570999999997 2024-09-17T22:42:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829128+0000", 2024-09-17T22:42:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3131000000000002e-05 2024-09-17T22:42:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829227+0000", 2024-09-17T22:42:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8800000000000003e-05 2024-09-17T22:42:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38767 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.485297438000003, 2024-09-17T22:42:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137018, 2024-09-17T22:42:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38767 2024-09-17T22:42:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:49.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:42:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777290+0000", 2024-09-17T22:42:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.88e-06 2024-09-17T22:42:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777296+0000", 2024-09-17T22:42:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-09-17T22:42:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777300+0000", 2024-09-17T22:42:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4510000000000002e-06 2024-09-17T22:42:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829257+0000", 2024-09-17T22:42:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051956975000000002 2024-09-17T22:42:49.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829328+0000", 2024-09-17T22:42:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1026999999999996e-05 2024-09-17T22:42:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829425+0000", 2024-09-17T22:42:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.679e-05 2024-09-17T22:42:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.294 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38784 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 675840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.485278984000004, 2024-09-17T22:42:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052344522999999997, 2024-09-17T22:42:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38784 2024-09-17T22:42:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777307+0000", 2024-09-17T22:42:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:42:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777309+0000", 2024-09-17T22:42:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.063e-06 2024-09-17T22:42:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777314+0000", 2024-09-17T22:42:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-17T22:42:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777319+0000", 2024-09-17T22:42:49.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999996e-06 2024-09-17T22:42:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829456+0000", 2024-09-17T22:42:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137151 2024-09-17T22:42:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829527+0000", 2024-09-17T22:42:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1588000000000006e-05 2024-09-17T22:42:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829651+0000", 2024-09-17T22:42:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012365 2024-09-17T22:42:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38785 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 749568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.485260869000001, 2024-09-17T22:42:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052546572999999999, 2024-09-17T22:42:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.299 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38785 2024-09-17T22:42:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:42:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777327+0000", 2024-09-17T22:42:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:42:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777333+0000", 2024-09-17T22:42:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7309999999999998e-06 2024-09-17T22:42:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777337+0000", 2024-09-17T22:42:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:42:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829686+0000", 2024-09-17T22:42:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052349159999999999 2024-09-17T22:42:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829766+0000", 2024-09-17T22:42:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9326999999999995e-05 2024-09-17T22:42:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829871+0000", 2024-09-17T22:42:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000105466 2024-09-17T22:42:49.303 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38798 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.484627261, 2024-09-17T22:42:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052117564999999998, 2024-09-17T22:42:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.304 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38798 2024-09-17T22:42:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777959+0000", 2024-09-17T22:42:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9599999999999999e-07 2024-09-17T22:42:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777963+0000", 2024-09-17T22:42:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2660000000000003e-06 2024-09-17T22:42:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777971+0000", 2024-09-17T22:42:49.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3489999999999992e-06 2024-09-17T22:42:49.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777978+0000", 2024-09-17T22:42:49.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:42:49.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829906+0000", 2024-09-17T22:42:49.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051927635999999999 2024-09-17T22:42:49.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:49.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829986+0000", 2024-09-17T22:42:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0444000000000004e-05 2024-09-17T22:42:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830076+0000", 2024-09-17T22:42:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9473000000000002e-05 2024-09-17T22:42:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38799 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 882688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:49.308 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.484595822000003, 2024-09-17T22:42:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052359606000000003, 2024-09-17T22:42:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38799 2024-09-17T22:42:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8299999999999997e-07 2024-09-17T22:42:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777992+0000", 2024-09-17T22:42:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.176e-06 2024-09-17T22:42:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777998+0000", 2024-09-17T22:42:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.173e-06 2024-09-17T22:42:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778003+0000", 2024-09-17T22:42:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6349999999999997e-06 2024-09-17T22:42:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830112+0000", 2024-09-17T22:42:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052109366999999997 2024-09-17T22:42:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830192+0000", 2024-09-17T22:42:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9814000000000005e-05 2024-09-17T22:42:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830349+0000", 2024-09-17T22:42:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015695800000000001 2024-09-17T22:42:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.312 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38814 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.484312418000002, 2024-09-17T22:42:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052274339000000003, 2024-09-17T22:42:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:49.313 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38814 2024-09-17T22:42:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:42:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778276+0000", 2024-09-17T22:42:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2029999999999999e-06 2024-09-17T22:42:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778282+0000", 2024-09-17T22:42:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0059999999999998e-06 2024-09-17T22:42:49.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778287+0000", 2024-09-17T22:42:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0039999999999999e-06 2024-09-17T22:42:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830382+0000", 2024-09-17T22:42:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052095379999999997 2024-09-17T22:42:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:49.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830455+0000", 2024-09-17T22:42:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3127000000000006e-05 2024-09-17T22:42:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830547+0000", 2024-09-17T22:42:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2207999999999995e-05 2024-09-17T22:42:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38828 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 163840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:49.317 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.484130010000001, 2024-09-17T22:42:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052466039999999999, 2024-09-17T22:42:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38828 2024-09-17T22:42:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778456+0000", 2024-09-17T22:42:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7599999999999996e-07 2024-09-17T22:42:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778460+0000", 2024-09-17T22:42:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8229999999999997e-06 2024-09-17T22:42:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778470+0000", 2024-09-17T22:42:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8199999999999992e-06 2024-09-17T22:42:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778476+0000", 2024-09-17T22:42:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2589999999999997e-06 2024-09-17T22:42:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830578+0000", 2024-09-17T22:42:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052102273999999997 2024-09-17T22:42:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830667+0000", 2024-09-17T22:42:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8701000000000003e-05 2024-09-17T22:42:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830921+0000", 2024-09-17T22:42:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025438700000000002 2024-09-17T22:42:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38829 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.484098613, 2024-09-17T22:42:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052648882000000001, 2024-09-17T22:42:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.322 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38829 2024-09-17T22:42:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7e-07 2024-09-17T22:42:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778490+0000", 2024-09-17T22:42:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:42:49.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778495+0000", 2024-09-17T22:42:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9329999999999998e-06 2024-09-17T22:42:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778500+0000", 2024-09-17T22:42:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9509999999999997e-06 2024-09-17T22:42:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830958+0000", 2024-09-17T22:42:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05245785 2024-09-17T22:42:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831051+0000", 2024-09-17T22:42:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2989e-05 2024-09-17T22:42:49.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831136+0000", 2024-09-17T22:42:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4448000000000002e-05 2024-09-17T22:42:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38891 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:49.327 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.480786554999995, 2024-09-17T22:42:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050847934999999997, 2024-09-17T22:42:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38891 2024-09-17T22:42:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-09-17T22:42:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781802+0000", 2024-09-17T22:42:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:42:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781810+0000", 2024-09-17T22:42:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8019999999999992e-06 2024-09-17T22:42:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781816+0000", 2024-09-17T22:42:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.285e-06 2024-09-17T22:42:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832485+0000", 2024-09-17T22:42:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050668462999999997 2024-09-17T22:42:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832565+0000", 2024-09-17T22:42:49.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9692999999999997e-05 2024-09-17T22:42:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832647+0000", 2024-09-17T22:42:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2224000000000004e-05 2024-09-17T22:42:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38892 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:49.332 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.480758305999998, 2024-09-17T22:42:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050946120999999997, 2024-09-17T22:42:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38892 2024-09-17T22:42:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781828+0000", 2024-09-17T22:42:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5400000000000002e-07 2024-09-17T22:42:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781830+0000", 2024-09-17T22:42:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1959999999999998e-06 2024-09-17T22:42:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781837+0000", 2024-09-17T22:42:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8310000000000003e-06 2024-09-17T22:42:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781843+0000", 2024-09-17T22:42:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1e-06 2024-09-17T22:42:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832674+0000", 2024-09-17T22:42:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050831182000000003 2024-09-17T22:42:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832725+0000", 2024-09-17T22:42:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1588000000000001e-05 2024-09-17T22:42:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832773+0000", 2024-09-17T22:42:49.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7769999999999998e-05 2024-09-17T22:42:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38909 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 36864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.480735535999997, 2024-09-17T22:42:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051046518999999999, 2024-09-17T22:42:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:49.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38909 2024-09-17T22:42:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:49.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:42:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781852+0000", 2024-09-17T22:42:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0590000000000001e-06 2024-09-17T22:42:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781859+0000", 2024-09-17T22:42:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3380000000000001e-06 2024-09-17T22:42:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781864+0000", 2024-09-17T22:42:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:42:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832804+0000", 2024-09-17T22:42:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050939614000000001 2024-09-17T22:42:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832845+0000", 2024-09-17T22:42:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1014999999999999e-05 2024-09-17T22:42:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832896+0000", 2024-09-17T22:42:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1836999999999999e-05 2024-09-17T22:42:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38910 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.480715226000001, 2024-09-17T22:42:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051216818999999997, 2024-09-17T22:42:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38910 2024-09-17T22:42:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781871+0000", 2024-09-17T22:42:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:42:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781873+0000", 2024-09-17T22:42:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:42:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781879+0000", 2024-09-17T22:42:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0700000000000003e-06 2024-09-17T22:42:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781884+0000", 2024-09-17T22:42:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-09-17T22:42:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832932+0000", 2024-09-17T22:42:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051048219999999998 2024-09-17T22:42:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833001+0000", 2024-09-17T22:42:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8504999999999995e-05 2024-09-17T22:42:49.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833087+0000", 2024-09-17T22:42:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6464000000000004e-05 2024-09-17T22:42:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38923 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.480694905999997, 2024-09-17T22:42:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051379573999999997, 2024-09-17T22:42:49.346 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38923 2024-09-17T22:42:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:42:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781893+0000", 2024-09-17T22:42:49.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-09-17T22:42:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781900+0000", 2024-09-17T22:42:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3860000000000004e-06 2024-09-17T22:42:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781904+0000", 2024-09-17T22:42:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9300000000000002e-06 2024-09-17T22:42:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833115+0000", 2024-09-17T22:42:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051210845999999997 2024-09-17T22:42:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833188+0000", 2024-09-17T22:42:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2276999999999999e-05 2024-09-17T22:42:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833270+0000", 2024-09-17T22:42:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2501000000000002e-05 2024-09-17T22:42:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.350 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38941 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.479902370999994, 2024-09-17T22:42:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050969459000000002, 2024-09-17T22:42:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.351 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38941 2024-09-17T22:42:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782684+0000", 2024-09-17T22:42:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:42:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782686+0000", 2024-09-17T22:42:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.396e-06 2024-09-17T22:42:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782693+0000", 2024-09-17T22:42:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1999999999999997e-06 2024-09-17T22:42:49.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782699+0000", 2024-09-17T22:42:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7910000000000001e-06 2024-09-17T22:42:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833471+0000", 2024-09-17T22:42:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050772333000000003 2024-09-17T22:42:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833536+0000", 2024-09-17T22:42:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4380000000000004e-05 2024-09-17T22:42:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833653+0000", 2024-09-17T22:42:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011691799999999999 2024-09-17T22:42:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:151189 3.14 3:2b696b13:::rbd_data.115f14f2d1d5.0000000000000009:e [read 3127296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.549681626000002, 2024-09-17T22:42:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.054254700000000003, 2024-09-17T22:42:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 151189 2024-09-17T22:42:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713051+0000", 2024-09-17T22:42:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014722100000000001 2024-09-17T22:42:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713055+0000", 2024-09-17T22:42:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.16e-06 2024-09-17T22:42:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713065+0000", 2024-09-17T22:42:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8390000000000002e-06 2024-09-17T22:42:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713073+0000", 2024-09-17T22:42:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0390000000000005e-06 2024-09-17T22:42:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762644+0000", 2024-09-17T22:42:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049571160000000003 2024-09-17T22:42:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762739+0000", 2024-09-17T22:42:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4565e-05 2024-09-17T22:42:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.767159+0000", 2024-09-17T22:42:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044197159999999997 2024-09-17T22:42:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:49.360 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:49.361 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:42:49.362 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:49.359+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:49.418 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is unset 2024-09-17T22:42:49.461 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:49.459+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:49.487 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:49.487 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:49.488 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:49.488 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:49.499 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:42:49.583 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:49.581+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:49.643 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:49.643 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:49.643 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:49.644 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:49.644 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:49.652 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:42:49.663 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:49.662+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:49.764 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:49.762+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:49.797 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158673 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:49.797 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552317764999998, 2024-09-17T22:42:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039400614, 2024-09-17T22:42:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158673 2024-09-17T22:42:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.798 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:49.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.04e-07 2024-09-17T22:42:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241563+0000", 2024-09-17T22:42:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5799999999999998e-07 2024-09-17T22:42:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241566+0000", 2024-09-17T22:42:49.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8279999999999999e-06 2024-09-17T22:42:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241568+0000", 2024-09-17T22:42:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2709999999999999e-06 2024-09-17T22:42:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280954+0000", 2024-09-17T22:42:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039386051999999998 2024-09-17T22:42:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280962+0000", 2024-09-17T22:42:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3010000000000007e-06 2024-09-17T22:42:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158675 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552308474, 2024-09-17T22:42:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039408033000000002, 2024-09-17T22:42:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158675 2024-09-17T22:42:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.803 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.36e-07 2024-09-17T22:42:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241572+0000", 2024-09-17T22:42:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6599999999999994e-07 2024-09-17T22:42:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241575+0000", 2024-09-17T22:42:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8389999999999998e-06 2024-09-17T22:42:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241577+0000", 2024-09-17T22:42:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4119999999999999e-06 2024-09-17T22:42:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280975+0000", 2024-09-17T22:42:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039397205999999997 2024-09-17T22:42:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280979+0000", 2024-09-17T22:42:49.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.374e-06 2024-09-17T22:42:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158677 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552286031000001, 2024-09-17T22:42:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039409307999999997, 2024-09-17T22:42:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.807 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158677 2024-09-17T22:42:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241594+0000", 2024-09-17T22:42:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:42:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241595+0000", 2024-09-17T22:42:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2729999999999999e-06 2024-09-17T22:42:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241602+0000", 2024-09-17T22:42:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:42:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241606+0000", 2024-09-17T22:42:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3019999999999999e-06 2024-09-17T22:42:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280994+0000", 2024-09-17T22:42:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039388484000000001 2024-09-17T22:42:49.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281003+0000", 2024-09-17T22:42:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5459999999999992e-06 2024-09-17T22:42:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158679 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552270352999997, 2024-09-17T22:42:49.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039413403999999999, 2024-09-17T22:42:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158679 2024-09-17T22:42:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4999999999999999e-07 2024-09-17T22:42:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241613+0000", 2024-09-17T22:42:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.207e-06 2024-09-17T22:42:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241616+0000", 2024-09-17T22:42:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.275e-06 2024-09-17T22:42:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241618+0000", 2024-09-17T22:42:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:42:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281017+0000", 2024-09-17T22:42:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039398385000000001 2024-09-17T22:42:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281022+0000", 2024-09-17T22:42:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.896e-06 2024-09-17T22:42:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.815 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158681 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552258174999999, 2024-09-17T22:42:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039425292000000001, 2024-09-17T22:42:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158681 2024-09-17T22:42:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:49.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:42:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241622+0000", 2024-09-17T22:42:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:42:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241625+0000", 2024-09-17T22:42:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9629999999999998e-06 2024-09-17T22:42:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241628+0000", 2024-09-17T22:42:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4339999999999999e-06 2024-09-17T22:42:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281039+0000", 2024-09-17T22:42:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039410672000000001 2024-09-17T22:42:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281047+0000", 2024-09-17T22:42:49.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9969999999999998e-06 2024-09-17T22:42:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158682 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552248890999998, 2024-09-17T22:42:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039436455000000002, 2024-09-17T22:42:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.820 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158682 2024-09-17T22:42:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:49.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:42:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241632+0000", 2024-09-17T22:42:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:42:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241635+0000", 2024-09-17T22:42:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8940000000000002e-06 2024-09-17T22:42:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241637+0000", 2024-09-17T22:42:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3659999999999999e-06 2024-09-17T22:42:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281061+0000", 2024-09-17T22:42:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039423664999999997 2024-09-17T22:42:49.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281067+0000", 2024-09-17T22:42:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.285e-06 2024-09-17T22:42:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158684 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552239544999999, 2024-09-17T22:42:49.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462980000000002, 2024-09-17T22:42:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158684 2024-09-17T22:42:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241641+0000", 2024-09-17T22:42:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:42:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241644+0000", 2024-09-17T22:42:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8870000000000001e-06 2024-09-17T22:42:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241646+0000", 2024-09-17T22:42:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3530000000000002e-06 2024-09-17T22:42:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281093+0000", 2024-09-17T22:42:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039446947000000003 2024-09-17T22:42:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281103+0000", 2024-09-17T22:42:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6299999999999993e-06 2024-09-17T22:42:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158686 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552230375000001, 2024-09-17T22:42:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039474214000000001, 2024-09-17T22:42:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158686 2024-09-17T22:42:49.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:42:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241650+0000", 2024-09-17T22:42:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2999999999999999e-07 2024-09-17T22:42:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241653+0000", 2024-09-17T22:42:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9689999999999999e-06 2024-09-17T22:42:49.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241655+0000", 2024-09-17T22:42:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3460000000000001e-06 2024-09-17T22:42:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281118+0000", 2024-09-17T22:42:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462876000000001 2024-09-17T22:42:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281123+0000", 2024-09-17T22:42:49.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8840000000000002e-06 2024-09-17T22:42:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158688 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552221144000001, 2024-09-17T22:42:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039490458999999999, 2024-09-17T22:42:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.833 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158688 2024-09-17T22:42:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:42:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241659+0000", 2024-09-17T22:42:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3699999999999999e-07 2024-09-17T22:42:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241662+0000", 2024-09-17T22:42:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8499999999999998e-06 2024-09-17T22:42:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241665+0000", 2024-09-17T22:42:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.356e-06 2024-09-17T22:42:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281141+0000", 2024-09-17T22:42:49.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039476031000000002 2024-09-17T22:42:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281149+0000", 2024-09-17T22:42:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0709999999999995e-06 2024-09-17T22:42:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158690 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552212085000001, 2024-09-17T22:42:49.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039502378999999997, 2024-09-17T22:42:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158690 2024-09-17T22:42:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:49.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241668+0000", 2024-09-17T22:42:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241669+0000", 2024-09-17T22:42:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1009999999999999e-06 2024-09-17T22:42:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241672+0000", 2024-09-17T22:42:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9059999999999998e-06 2024-09-17T22:42:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241674+0000", 2024-09-17T22:42:49.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4439999999999998e-06 2024-09-17T22:42:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281162+0000", 2024-09-17T22:42:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039488189 2024-09-17T22:42:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281170+0000", 2024-09-17T22:42:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5310000000000004e-06 2024-09-17T22:42:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.841 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158692 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552202651999998, 2024-09-17T22:42:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039531064999999997, 2024-09-17T22:42:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158692 2024-09-17T22:42:49.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241678+0000", 2024-09-17T22:42:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0800000000000003e-07 2024-09-17T22:42:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241681+0000", 2024-09-17T22:42:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8729999999999999e-06 2024-09-17T22:42:49.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241683+0000", 2024-09-17T22:42:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.447e-06 2024-09-17T22:42:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281199+0000", 2024-09-17T22:42:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039515388999999998 2024-09-17T22:42:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281208+0000", 2024-09-17T22:42:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2399999999999996e-06 2024-09-17T22:42:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158694 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552193481000003, 2024-09-17T22:42:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039543986000000003, 2024-09-17T22:42:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.846 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158694 2024-09-17T22:42:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:42:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241687+0000", 2024-09-17T22:42:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:42:49.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241691+0000", 2024-09-17T22:42:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.811e-06 2024-09-17T22:42:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241693+0000", 2024-09-17T22:42:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.385e-06 2024-09-17T22:42:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281223+0000", 2024-09-17T22:42:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039529332 2024-09-17T22:42:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281230+0000", 2024-09-17T22:42:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2139999999999999e-06 2024-09-17T22:42:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158696 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.850 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552183092999996, 2024-09-17T22:42:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039570458000000003, 2024-09-17T22:42:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158696 2024-09-17T22:42:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:42:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:42:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:42:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.33e-07 2024-09-17T22:42:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241700+0000", 2024-09-17T22:42:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8949999999999998e-06 2024-09-17T22:42:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241703+0000", 2024-09-17T22:42:49.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3159999999999999e-06 2024-09-17T22:42:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281257+0000", 2024-09-17T22:42:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039553862000000002 2024-09-17T22:42:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281267+0000", 2024-09-17T22:42:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.024e-05 2024-09-17T22:42:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.854 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158698 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552173015000001, 2024-09-17T22:42:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039576493999999997, 2024-09-17T22:42:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.855 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158698 2024-09-17T22:42:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241707+0000", 2024-09-17T22:42:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:42:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241708+0000", 2024-09-17T22:42:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5000000000000001e-07 2024-09-17T22:42:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241711+0000", 2024-09-17T22:42:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9670000000000001e-06 2024-09-17T22:42:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241713+0000", 2024-09-17T22:42:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2689999999999998e-06 2024-09-17T22:42:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281278+0000", 2024-09-17T22:42:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039565206999999998 2024-09-17T22:42:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281283+0000", 2024-09-17T22:42:49.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:42:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158700 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552163804000003, 2024-09-17T22:42:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039604292999999999, 2024-09-17T22:42:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.859 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158700 2024-09-17T22:42:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:49.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241717+0000", 2024-09-17T22:42:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.78e-07 2024-09-17T22:42:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241720+0000", 2024-09-17T22:42:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8119999999999999e-06 2024-09-17T22:42:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241722+0000", 2024-09-17T22:42:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-17T22:42:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281311+0000", 2024-09-17T22:42:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039588894999999999 2024-09-17T22:42:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281320+0000", 2024-09-17T22:42:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9989999999999997e-06 2024-09-17T22:42:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158702 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:49.863 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552152481999997, 2024-09-17T22:42:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039616142, 2024-09-17T22:42:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158702 2024-09-17T22:42:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4400000000000001e-07 2024-09-17T22:42:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241728+0000", 2024-09-17T22:42:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0839999999999999e-06 2024-09-17T22:42:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241731+0000", 2024-09-17T22:42:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0690000000000002e-06 2024-09-17T22:42:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241734+0000", 2024-09-17T22:42:49.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4159999999999998e-06 2024-09-17T22:42:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281334+0000", 2024-09-17T22:42:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039600110000000001 2024-09-17T22:42:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281343+0000", 2024-09-17T22:42:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2189999999999993e-06 2024-09-17T22:42:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.867 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158704 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552142617999998, 2024-09-17T22:42:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039641485999999997, 2024-09-17T22:42:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.868 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158704 2024-09-17T22:42:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:42:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241738+0000", 2024-09-17T22:42:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0589999999999999e-06 2024-09-17T22:42:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241741+0000", 2024-09-17T22:42:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.943e-06 2024-09-17T22:42:49.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241744+0000", 2024-09-17T22:42:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4819999999999999e-06 2024-09-17T22:42:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281370+0000", 2024-09-17T22:42:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039625661 2024-09-17T22:42:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281378+0000", 2024-09-17T22:42:49.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1310000000000006e-06 2024-09-17T22:42:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158705 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552132067999999, 2024-09-17T22:42:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039652637999999997, 2024-09-17T22:42:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.872 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158705 2024-09-17T22:42:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:49.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:42:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-07 2024-09-17T22:42:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:42:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0599999999999999e-07 2024-09-17T22:42:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241751+0000", 2024-09-17T22:42:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.926e-06 2024-09-17T22:42:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241754+0000", 2024-09-17T22:42:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4360000000000001e-06 2024-09-17T22:42:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281392+0000", 2024-09-17T22:42:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039638566 2024-09-17T22:42:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281400+0000", 2024-09-17T22:42:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.571e-06 2024-09-17T22:42:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158707 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552110966999997, 2024-09-17T22:42:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039654809999999999, 2024-09-17T22:42:49.876 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158707 2024-09-17T22:42:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241769+0000", 2024-09-17T22:42:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4299999999999999e-07 2024-09-17T22:42:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241770+0000", 2024-09-17T22:42:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3769999999999999e-06 2024-09-17T22:42:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241773+0000", 2024-09-17T22:42:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2109999999999998e-06 2024-09-17T22:42:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241776+0000", 2024-09-17T22:42:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5069999999999999e-06 2024-09-17T22:42:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281415+0000", 2024-09-17T22:42:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039639499000000002 2024-09-17T22:42:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281423+0000", 2024-09-17T22:42:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8730000000000007e-06 2024-09-17T22:42:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158709 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 59.552100357999997, 2024-09-17T22:42:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039661937000000001, 2024-09-17T22:42:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:49.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158709 2024-09-17T22:42:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:49.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242843+0000", 2024-09-17T22:42:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010634780000000001 2024-09-17T22:42:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242845+0000", 2024-09-17T22:42:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7209999999999999e-06 2024-09-17T22:42:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242851+0000", 2024-09-17T22:42:49.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6949999999999997e-06 2024-09-17T22:42:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242855+0000", 2024-09-17T22:42:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7960000000000002e-06 2024-09-17T22:42:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281434+0000", 2024-09-17T22:42:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038579519 2024-09-17T22:42:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:49.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281441+0000", 2024-09-17T22:42:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7279999999999998e-06 2024-09-17T22:42:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:49.885 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:49.885 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:49.886 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:42:49.887 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:49.884+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:49.987 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:49.985+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:50.031 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:50.031 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:50.031 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:50.031 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:50.041 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:42:50.124 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:50.122+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:50.133 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:42:50.184 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:50.184 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:50.184 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:50.184 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:50.184 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:50.189 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:50.187+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:50.192 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:42:50.236 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:50.253 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:50.253 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:50.269 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:50.269 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:50.285 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:50.285 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:50.290 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:50.288+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:50.302 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:50.302 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:50.319 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:50.319 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:50.336 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:50.336 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:50.336 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:50.337 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:50.345 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:42:50.352 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:50.353 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:50.356 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:50.356 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:50.391 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:50.389+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:50.457 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:50.457 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:50.467 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:42:50.492 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:50.490+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:50.593 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:50.591+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:50.612 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:50.612 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:50.621 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:42:50.695 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:50.692+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:50.725 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:50.726 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:50.734 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:42:50.818 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:50.816+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:50.878 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:50.878 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:50.887 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:42:50.929 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:50.926+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:50.996 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:50.994+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:51.030 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:51.031 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:51.040 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:42:51.097 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:51.095+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:51.183 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:51.183 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:51.192 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:42:51.197 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:51.195+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:51.297 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:51.298 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:51.298 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:51.295+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:51.306 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:42:51.398 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:51.396+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:51.435 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:42:51.457 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:51.457 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:51.468 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:42:51.509 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:51.506+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:51.600 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:51.598+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:51.622 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:51.622 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:51.631 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:42:51.701 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:51.699+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:51.736 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:51.736 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:51.745 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:42:51.802 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:51.800+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:51.847 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:51.848 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:51.856 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:42:51.903 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:51.901+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:52.003 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:52.003 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:52.004 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:52.001+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:52.012 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:42:52.104 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:52.102+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:52.156 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:52.157 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:52.165 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:42:52.205 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:52.203+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:52.267 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:52.268 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:52.276 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:42:52.306 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:52.304+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:52.358 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:42:52.379 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:52.379 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:52.389 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:42:52.424 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:42:52.424 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:42:52.425 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T22:42:52.425 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T22:42:52.433 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:52.431+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:52.436 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is unset 2024-09-17T22:42:52.472 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:52.494 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:52.494 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:52.507 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:52.504+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:52.509 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:52.509 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:52.516 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:52.516 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:52.521 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:42:52.542 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:52.543 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:52.564 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:52.564 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:52.581 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:52.581 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:52.598 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:52.598 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:52.608 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:52.606+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:52.614 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:52.614 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:52.618 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:52.619 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:52.668 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:52.668 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:52.669 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:52.669 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:52.669 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:52.677 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:42:52.718 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:52.716+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:52.741 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:42:52.742 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:52.742 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 104, 2024-09-17T22:42:52.742 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "551c5ed8-e790-485a-bc95-a9578a29275c", 2024-09-17T22:42:52.742 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-09-17T22:40:50.189230+0000", 2024-09-17T22:42:52.742 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-09-17T22:42:52.431088+0000", 2024-09-17T22:42:52.742 INFO:teuthology.orchestra.run.smithi115.stdout: "last_up_change": "2024-09-17T22:41:53.099515+0000", 2024-09-17T22:42:52.742 INFO:teuthology.orchestra.run.smithi115.stdout: "last_in_change": "2024-09-17T22:40:52.260526+0000", 2024-09-17T22:42:52.742 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T22:42:52.742 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_num": 5799936, 2024-09-17T22:42:52.742 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_set": [ 2024-09-17T22:42:52.742 INFO:teuthology.orchestra.run.smithi115.stdout: "pglog_hardlimit", 2024-09-17T22:42:52.743 INFO:teuthology.orchestra.run.smithi115.stdout: "purged_snapdirs", 2024-09-17T22:42:52.743 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_deletes", 2024-09-17T22:42:52.743 INFO:teuthology.orchestra.run.smithi115.stdout: "sortbitwise" 2024-09-17T22:42:52.743 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:52.743 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_version": 5, 2024-09-17T22:42:52.743 INFO:teuthology.orchestra.run.smithi115.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T22:42:52.743 INFO:teuthology.orchestra.run.smithi115.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T22:42:52.743 INFO:teuthology.orchestra.run.smithi115.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T22:42:52.743 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_snapshot": "", 2024-09-17T22:42:52.743 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_max": 3, 2024-09-17T22:42:52.743 INFO:teuthology.orchestra.run.smithi115.stdout: "max_osd": 8, 2024-09-17T22:42:52.743 INFO:teuthology.orchestra.run.smithi115.stdout: "require_min_compat_client": "luminous", 2024-09-17T22:42:52.744 INFO:teuthology.orchestra.run.smithi115.stdout: "min_compat_client": "luminous", 2024-09-17T22:42:52.744 INFO:teuthology.orchestra.run.smithi115.stdout: "require_osd_release": "squid", 2024-09-17T22:42:52.744 INFO:teuthology.orchestra.run.smithi115.stdout: "allow_crimson": false, 2024-09-17T22:42:52.744 INFO:teuthology.orchestra.run.smithi115.stdout: "pools": [ 2024-09-17T22:42:52.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.744 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 1, 2024-09-17T22:42:52.744 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_name": ".mgr", 2024-09-17T22:42:52.744 INFO:teuthology.orchestra.run.smithi115.stdout: "create_time": "2024-09-17T22:41:02.338356+0000", 2024-09-17T22:42:52.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-09-17T22:42:52.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_names": "hashpspool", 2024-09-17T22:42:52.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type": 1, 2024-09-17T22:42:52.744 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 2, 2024-09-17T22:42:52.745 INFO:teuthology.orchestra.run.smithi115.stdout: "min_size": 1, 2024-09-17T22:42:52.745 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_rule": 0, 2024-09-17T22:42:52.745 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:42:52.745 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:42:52.745 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:42:52.745 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:42:52.745 INFO:teuthology.orchestra.run.smithi115.stdout: "is_stretch_pool": false, 2024-09-17T22:42:52.745 INFO:teuthology.orchestra.run.smithi115.stdout: "object_hash": 2, 2024-09-17T22:42:52.745 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:42:52.745 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num": 1, 2024-09-17T22:42:52.745 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num": 1, 2024-09-17T22:42:52.745 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num_target": 1, 2024-09-17T22:42:52.745 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_target": 1, 2024-09-17T22:42:52.746 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_pending": 1, 2024-09-17T22:42:52.746 INFO:teuthology.orchestra.run.smithi115.stdout: "last_pg_merge_meta": { 2024-09-17T22:42:52.746 INFO:teuthology.orchestra.run.smithi115.stdout: "source_pgid": "0.0", 2024-09-17T22:42:52.746 INFO:teuthology.orchestra.run.smithi115.stdout: "ready_epoch": 0, 2024-09-17T22:42:52.746 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_started": 0, 2024-09-17T22:42:52.746 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_clean": 0, 2024-09-17T22:42:52.746 INFO:teuthology.orchestra.run.smithi115.stdout: "source_version": "0'0", 2024-09-17T22:42:52.746 INFO:teuthology.orchestra.run.smithi115.stdout: "target_version": "0'0" 2024-09-17T22:42:52.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.746 INFO:teuthology.orchestra.run.smithi115.stdout: "last_change": "17", 2024-09-17T22:42:52.746 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend": "0", 2024-09-17T22:42:52.746 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:42:52.746 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:42:52.747 INFO:teuthology.orchestra.run.smithi115.stdout: "auid": 0, 2024-09-17T22:42:52.747 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:42:52.747 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_seq": 0, 2024-09-17T22:42:52.747 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_epoch": 0, 2024-09-17T22:42:52.747 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_snaps": [], 2024-09-17T22:42:52.747 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps": "[]", 2024-09-17T22:42:52.747 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_bytes": 0, 2024-09-17T22:42:52.747 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_objects": 0, 2024-09-17T22:42:52.747 INFO:teuthology.orchestra.run.smithi115.stdout: "tiers": [], 2024-09-17T22:42:52.747 INFO:teuthology.orchestra.run.smithi115.stdout: "tier_of": -1, 2024-09-17T22:42:52.747 INFO:teuthology.orchestra.run.smithi115.stdout: "read_tier": -1, 2024-09-17T22:42:52.747 INFO:teuthology.orchestra.run.smithi115.stdout: "write_tier": -1, 2024-09-17T22:42:52.747 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_mode": "none", 2024-09-17T22:42:52.747 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_bytes": 0, 2024-09-17T22:42:52.747 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_objects": 0, 2024-09-17T22:42:52.748 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:42:52.748 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:42:52.748 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:42:52.748 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_flush_age": 0, 2024-09-17T22:42:52.748 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_evict_age": 0, 2024-09-17T22:42:52.748 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profile": "", 2024-09-17T22:42:52.748 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_params": { 2024-09-17T22:42:52.748 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "none" 2024-09-17T22:42:52.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.748 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_period": 0, 2024-09-17T22:42:52.748 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_count": 0, 2024-09-17T22:42:52.748 INFO:teuthology.orchestra.run.smithi115.stdout: "use_gmt_hitset": true, 2024-09-17T22:42:52.748 INFO:teuthology.orchestra.run.smithi115.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:42:52.749 INFO:teuthology.orchestra.run.smithi115.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:42:52.749 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:42:52.749 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:42:52.749 INFO:teuthology.orchestra.run.smithi115.stdout: "grade_table": [], 2024-09-17T22:42:52.749 INFO:teuthology.orchestra.run.smithi115.stdout: "stripe_width": 0, 2024-09-17T22:42:52.749 INFO:teuthology.orchestra.run.smithi115.stdout: "expected_num_objects": 0, 2024-09-17T22:42:52.749 INFO:teuthology.orchestra.run.smithi115.stdout: "fast_read": false, 2024-09-17T22:42:52.749 INFO:teuthology.orchestra.run.smithi115.stdout: "options": { 2024-09-17T22:42:52.749 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_max": 32, 2024-09-17T22:42:52.749 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_min": 1 2024-09-17T22:42:52.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.749 INFO:teuthology.orchestra.run.smithi115.stdout: "application_metadata": { 2024-09-17T22:42:52.749 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": {} 2024-09-17T22:42:52.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.750 INFO:teuthology.orchestra.run.smithi115.stdout: "read_balance": { 2024-09-17T22:42:52.750 INFO:teuthology.orchestra.run.smithi115.stdout: "score_type": "Fair distribution", 2024-09-17T22:42:52.750 INFO:teuthology.orchestra.run.smithi115.stdout: "score_acting": 6.9000000953674316, 2024-09-17T22:42:52.750 INFO:teuthology.orchestra.run.smithi115.stdout: "score_stable": 6.9000000953674316, 2024-09-17T22:42:52.750 INFO:teuthology.orchestra.run.smithi115.stdout: "optimal_score": 0.28999999165534973, 2024-09-17T22:42:52.750 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_acting": 2, 2024-09-17T22:42:52.750 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_stable": 2, 2024-09-17T22:42:52.750 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity_weighted": 0.81284332275390625, 2024-09-17T22:42:52.750 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-17T22:42:52.750 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity_weighted": 0.81000000238418579 2024-09-17T22:42:52.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.751 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-17T22:42:52.751 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_name": "rbd", 2024-09-17T22:42:52.751 INFO:teuthology.orchestra.run.smithi115.stdout: "create_time": "2024-09-17T22:41:08.097861+0000", 2024-09-17T22:42:52.751 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 8193, 2024-09-17T22:42:52.751 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T22:42:52.751 INFO:teuthology.orchestra.run.smithi115.stdout: "type": 1, 2024-09-17T22:42:52.751 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 2, 2024-09-17T22:42:52.751 INFO:teuthology.orchestra.run.smithi115.stdout: "min_size": 1, 2024-09-17T22:42:52.751 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_rule": 0, 2024-09-17T22:42:52.751 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:42:52.751 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:42:52.751 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:42:52.751 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:42:52.751 INFO:teuthology.orchestra.run.smithi115.stdout: "is_stretch_pool": false, 2024-09-17T22:42:52.752 INFO:teuthology.orchestra.run.smithi115.stdout: "object_hash": 2, 2024-09-17T22:42:52.752 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:42:52.752 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num": 8, 2024-09-17T22:42:52.752 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num": 8, 2024-09-17T22:42:52.752 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num_target": 8, 2024-09-17T22:42:52.752 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_target": 8, 2024-09-17T22:42:52.752 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_pending": 8, 2024-09-17T22:42:52.752 INFO:teuthology.orchestra.run.smithi115.stdout: "last_pg_merge_meta": { 2024-09-17T22:42:52.752 INFO:teuthology.orchestra.run.smithi115.stdout: "source_pgid": "0.0", 2024-09-17T22:42:52.752 INFO:teuthology.orchestra.run.smithi115.stdout: "ready_epoch": 0, 2024-09-17T22:42:52.752 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_started": 0, 2024-09-17T22:42:52.752 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_clean": 0, 2024-09-17T22:42:52.752 INFO:teuthology.orchestra.run.smithi115.stdout: "source_version": "0'0", 2024-09-17T22:42:52.753 INFO:teuthology.orchestra.run.smithi115.stdout: "target_version": "0'0" 2024-09-17T22:42:52.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.753 INFO:teuthology.orchestra.run.smithi115.stdout: "last_change": "21", 2024-09-17T22:42:52.753 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend": "0", 2024-09-17T22:42:52.753 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:42:52.753 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:42:52.753 INFO:teuthology.orchestra.run.smithi115.stdout: "auid": 0, 2024-09-17T22:42:52.753 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:42:52.753 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_seq": 2, 2024-09-17T22:42:52.753 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_epoch": 21, 2024-09-17T22:42:52.753 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_snaps": [], 2024-09-17T22:42:52.753 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps": "[]", 2024-09-17T22:42:52.753 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_bytes": 0, 2024-09-17T22:42:52.754 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_objects": 0, 2024-09-17T22:42:52.754 INFO:teuthology.orchestra.run.smithi115.stdout: "tiers": [], 2024-09-17T22:42:52.754 INFO:teuthology.orchestra.run.smithi115.stdout: "tier_of": -1, 2024-09-17T22:42:52.754 INFO:teuthology.orchestra.run.smithi115.stdout: "read_tier": -1, 2024-09-17T22:42:52.754 INFO:teuthology.orchestra.run.smithi115.stdout: "write_tier": -1, 2024-09-17T22:42:52.754 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_mode": "none", 2024-09-17T22:42:52.754 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_bytes": 0, 2024-09-17T22:42:52.754 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_objects": 0, 2024-09-17T22:42:52.754 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:42:52.754 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:42:52.754 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:42:52.754 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_flush_age": 0, 2024-09-17T22:42:52.754 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_evict_age": 0, 2024-09-17T22:42:52.755 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profile": "", 2024-09-17T22:42:52.755 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_params": { 2024-09-17T22:42:52.755 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "none" 2024-09-17T22:42:52.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.755 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_period": 0, 2024-09-17T22:42:52.755 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_count": 0, 2024-09-17T22:42:52.755 INFO:teuthology.orchestra.run.smithi115.stdout: "use_gmt_hitset": true, 2024-09-17T22:42:52.755 INFO:teuthology.orchestra.run.smithi115.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:42:52.755 INFO:teuthology.orchestra.run.smithi115.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:42:52.755 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:42:52.755 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:42:52.755 INFO:teuthology.orchestra.run.smithi115.stdout: "grade_table": [], 2024-09-17T22:42:52.755 INFO:teuthology.orchestra.run.smithi115.stdout: "stripe_width": 0, 2024-09-17T22:42:52.755 INFO:teuthology.orchestra.run.smithi115.stdout: "expected_num_objects": 0, 2024-09-17T22:42:52.756 INFO:teuthology.orchestra.run.smithi115.stdout: "fast_read": false, 2024-09-17T22:42:52.756 INFO:teuthology.orchestra.run.smithi115.stdout: "options": {}, 2024-09-17T22:42:52.756 INFO:teuthology.orchestra.run.smithi115.stdout: "application_metadata": { 2024-09-17T22:42:52.756 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd": {} 2024-09-17T22:42:52.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.756 INFO:teuthology.orchestra.run.smithi115.stdout: "read_balance": { 2024-09-17T22:42:52.756 INFO:teuthology.orchestra.run.smithi115.stdout: "score_type": "Fair distribution", 2024-09-17T22:42:52.756 INFO:teuthology.orchestra.run.smithi115.stdout: "score_acting": 2.619999885559082, 2024-09-17T22:42:52.756 INFO:teuthology.orchestra.run.smithi115.stdout: "score_stable": 2.619999885559082, 2024-09-17T22:42:52.756 INFO:teuthology.orchestra.run.smithi115.stdout: "optimal_score": 0.86000001430511475, 2024-09-17T22:42:52.756 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_acting": 2.25, 2024-09-17T22:42:52.756 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_stable": 2.25, 2024-09-17T22:42:52.756 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity_weighted": 0.81284332275390625, 2024-09-17T22:42:52.756 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-17T22:42:52.757 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity_weighted": 0.81000000238418579 2024-09-17T22:42:52.757 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.757 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 3, 2024-09-17T22:42:52.757 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_name": "pool_client.0", 2024-09-17T22:42:52.757 INFO:teuthology.orchestra.run.smithi115.stdout: "create_time": "2024-09-17T22:41:17.501542+0000", 2024-09-17T22:42:52.757 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 8193, 2024-09-17T22:42:52.757 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T22:42:52.757 INFO:teuthology.orchestra.run.smithi115.stdout: "type": 1, 2024-09-17T22:42:52.757 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 2, 2024-09-17T22:42:52.757 INFO:teuthology.orchestra.run.smithi115.stdout: "min_size": 1, 2024-09-17T22:42:52.757 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_rule": 0, 2024-09-17T22:42:52.757 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:42:52.757 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:42:52.758 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:42:52.758 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:42:52.758 INFO:teuthology.orchestra.run.smithi115.stdout: "is_stretch_pool": false, 2024-09-17T22:42:52.758 INFO:teuthology.orchestra.run.smithi115.stdout: "object_hash": 2, 2024-09-17T22:42:52.758 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:42:52.758 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num": 32, 2024-09-17T22:42:52.758 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num": 32, 2024-09-17T22:42:52.758 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num_target": 32, 2024-09-17T22:42:52.758 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_target": 32, 2024-09-17T22:42:52.758 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_pending": 32, 2024-09-17T22:42:52.758 INFO:teuthology.orchestra.run.smithi115.stdout: "last_pg_merge_meta": { 2024-09-17T22:42:52.758 INFO:teuthology.orchestra.run.smithi115.stdout: "source_pgid": "0.0", 2024-09-17T22:42:52.758 INFO:teuthology.orchestra.run.smithi115.stdout: "ready_epoch": 0, 2024-09-17T22:42:52.758 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_started": 0, 2024-09-17T22:42:52.759 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_clean": 0, 2024-09-17T22:42:52.759 INFO:teuthology.orchestra.run.smithi115.stdout: "source_version": "0'0", 2024-09-17T22:42:52.759 INFO:teuthology.orchestra.run.smithi115.stdout: "target_version": "0'0" 2024-09-17T22:42:52.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.759 INFO:teuthology.orchestra.run.smithi115.stdout: "last_change": "104", 2024-09-17T22:42:52.759 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend": "0", 2024-09-17T22:42:52.759 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:42:52.759 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:42:52.759 INFO:teuthology.orchestra.run.smithi115.stdout: "auid": 0, 2024-09-17T22:42:52.759 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:42:52.759 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_seq": 40, 2024-09-17T22:42:52.759 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_epoch": 104, 2024-09-17T22:42:52.759 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_snaps": [], 2024-09-17T22:42:52.759 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps": "[]", 2024-09-17T22:42:52.759 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_bytes": 0, 2024-09-17T22:42:52.760 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_objects": 0, 2024-09-17T22:42:52.760 INFO:teuthology.orchestra.run.smithi115.stdout: "tiers": [], 2024-09-17T22:42:52.760 INFO:teuthology.orchestra.run.smithi115.stdout: "tier_of": -1, 2024-09-17T22:42:52.760 INFO:teuthology.orchestra.run.smithi115.stdout: "read_tier": -1, 2024-09-17T22:42:52.760 INFO:teuthology.orchestra.run.smithi115.stdout: "write_tier": -1, 2024-09-17T22:42:52.760 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_mode": "none", 2024-09-17T22:42:52.760 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_bytes": 0, 2024-09-17T22:42:52.760 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_objects": 0, 2024-09-17T22:42:52.760 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:42:52.760 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:42:52.760 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:42:52.760 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_flush_age": 0, 2024-09-17T22:42:52.760 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_evict_age": 0, 2024-09-17T22:42:52.760 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profile": "", 2024-09-17T22:42:52.761 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_params": { 2024-09-17T22:42:52.761 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "none" 2024-09-17T22:42:52.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.761 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_period": 0, 2024-09-17T22:42:52.761 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_count": 0, 2024-09-17T22:42:52.761 INFO:teuthology.orchestra.run.smithi115.stdout: "use_gmt_hitset": true, 2024-09-17T22:42:52.761 INFO:teuthology.orchestra.run.smithi115.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:42:52.761 INFO:teuthology.orchestra.run.smithi115.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:42:52.761 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:42:52.761 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:42:52.761 INFO:teuthology.orchestra.run.smithi115.stdout: "grade_table": [], 2024-09-17T22:42:52.761 INFO:teuthology.orchestra.run.smithi115.stdout: "stripe_width": 0, 2024-09-17T22:42:52.761 INFO:teuthology.orchestra.run.smithi115.stdout: "expected_num_objects": 0, 2024-09-17T22:42:52.762 INFO:teuthology.orchestra.run.smithi115.stdout: "fast_read": false, 2024-09-17T22:42:52.762 INFO:teuthology.orchestra.run.smithi115.stdout: "options": {}, 2024-09-17T22:42:52.762 INFO:teuthology.orchestra.run.smithi115.stdout: "application_metadata": { 2024-09-17T22:42:52.762 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd": {} 2024-09-17T22:42:52.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.762 INFO:teuthology.orchestra.run.smithi115.stdout: "read_balance": { 2024-09-17T22:42:52.762 INFO:teuthology.orchestra.run.smithi115.stdout: "score_type": "Fair distribution", 2024-09-17T22:42:52.762 INFO:teuthology.orchestra.run.smithi115.stdout: "score_acting": 1.309999942779541, 2024-09-17T22:42:52.762 INFO:teuthology.orchestra.run.smithi115.stdout: "score_stable": 1.309999942779541, 2024-09-17T22:42:52.762 INFO:teuthology.orchestra.run.smithi115.stdout: "optimal_score": 1, 2024-09-17T22:42:52.762 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_acting": 1.309999942779541, 2024-09-17T22:42:52.762 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_stable": 1.309999942779541, 2024-09-17T22:42:52.762 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity_weighted": 0.81284332275390625, 2024-09-17T22:42:52.763 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-17T22:42:52.763 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity_weighted": 0.81000000238418579 2024-09-17T22:42:52.763 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.763 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.763 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:52.763 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:42:52.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.763 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 0, 2024-09-17T22:42:52.763 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "3bd5c11f-e7c2-4968-839a-2799c00d93a6", 2024-09-17T22:42:52.763 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 0, 2024-09-17T22:42:52.763 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:52.763 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:52.763 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 0, 2024-09-17T22:42:52.763 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:52.764 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:52.764 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:52.764 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 23, 2024-09-17T22:42:52.764 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 51, 2024-09-17T22:42:52.764 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:52.764 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:52.764 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.764 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.764 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6824", 2024-09-17T22:42:52.764 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:52.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.765 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.765 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6825", 2024-09-17T22:42:52.765 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:52.765 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.765 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.765 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:52.765 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.765 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.765 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6826", 2024-09-17T22:42:52.765 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:52.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.766 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.766 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6827", 2024-09-17T22:42:52.766 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:52.766 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.766 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.766 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:52.766 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.766 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.766 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6830", 2024-09-17T22:42:52.766 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:52.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.767 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.767 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6831", 2024-09-17T22:42:52.767 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:52.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.767 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.767 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:52.767 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.767 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.767 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6828", 2024-09-17T22:42:52.767 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:52.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.768 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.768 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6829", 2024-09-17T22:42:52.768 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:52.768 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.768 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.768 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6825/2782271783", 2024-09-17T22:42:52.768 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6827/2782271783", 2024-09-17T22:42:52.768 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6831/2782271783", 2024-09-17T22:42:52.768 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6829/2782271783", 2024-09-17T22:42:52.768 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:52.768 INFO:teuthology.orchestra.run.smithi115.stdout: "exists" 2024-09-17T22:42:52.769 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.769 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 1, 2024-09-17T22:42:52.769 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "02918203-cf2e-49fc-b994-743b5d81b4a1", 2024-09-17T22:42:52.769 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:52.769 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:52.769 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:52.769 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:52.769 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:52.769 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:52.769 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:52.769 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 99, 2024-09-17T22:42:52.770 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:52.770 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:52.770 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:52.770 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.770 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.770 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6816", 2024-09-17T22:42:52.770 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:52.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.770 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.770 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6817", 2024-09-17T22:42:52.770 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:52.770 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.771 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.771 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:52.771 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.771 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.771 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6818", 2024-09-17T22:42:52.771 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:52.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.771 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.771 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6819", 2024-09-17T22:42:52.771 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:52.772 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.772 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.772 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:52.772 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.772 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.772 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6822", 2024-09-17T22:42:52.772 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:52.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.772 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.772 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6823", 2024-09-17T22:42:52.773 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:52.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.773 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.773 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:52.773 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.773 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.773 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6820", 2024-09-17T22:42:52.773 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:52.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.773 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.774 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6821", 2024-09-17T22:42:52.774 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:52.774 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.774 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.774 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6817/1941780390", 2024-09-17T22:42:52.774 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6819/1941780390", 2024-09-17T22:42:52.774 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6823/1941780390", 2024-09-17T22:42:52.774 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6821/1941780390", 2024-09-17T22:42:52.774 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:52.774 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:52.774 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:52.774 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.775 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 2, 2024-09-17T22:42:52.775 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "22ee34ef-522e-4121-85b7-38bbafeca5de", 2024-09-17T22:42:52.775 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:52.775 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:52.775 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:52.775 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:52.775 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:52.775 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:52.775 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 13, 2024-09-17T22:42:52.775 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 99, 2024-09-17T22:42:52.775 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:52.775 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:52.776 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:52.776 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.776 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.776 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6808", 2024-09-17T22:42:52.776 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:52.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.776 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.776 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6809", 2024-09-17T22:42:52.776 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:52.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.776 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.777 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:52.777 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.777 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.777 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6810", 2024-09-17T22:42:52.777 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:52.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.777 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.777 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6811", 2024-09-17T22:42:52.777 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:52.777 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.777 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.778 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:52.778 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.778 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.778 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6814", 2024-09-17T22:42:52.778 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:52.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.778 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.778 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6815", 2024-09-17T22:42:52.778 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:52.778 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.778 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.779 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:52.779 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.779 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.779 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6812", 2024-09-17T22:42:52.779 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:52.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.779 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.779 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6813", 2024-09-17T22:42:52.779 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:52.779 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.779 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.780 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6809/689014698", 2024-09-17T22:42:52.780 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6811/689014698", 2024-09-17T22:42:52.780 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6815/689014698", 2024-09-17T22:42:52.780 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6813/689014698", 2024-09-17T22:42:52.780 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:52.780 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:52.780 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:52.780 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.780 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 3, 2024-09-17T22:42:52.780 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "5d464921-5696-43c5-b89d-430a54ddefb5", 2024-09-17T22:42:52.780 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:52.780 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:52.781 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:52.781 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 0.50274658203125, 2024-09-17T22:42:52.781 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:52.781 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:52.781 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 13, 2024-09-17T22:42:52.781 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 54, 2024-09-17T22:42:52.781 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:52.781 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:52.781 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:52.781 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.781 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.781 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6800", 2024-09-17T22:42:52.782 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:52.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.782 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.782 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6801", 2024-09-17T22:42:52.782 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:52.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.782 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.782 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:52.782 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.782 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.782 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6802", 2024-09-17T22:42:52.782 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:52.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.783 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.783 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6803", 2024-09-17T22:42:52.783 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:52.783 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.783 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.783 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:52.783 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.783 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.783 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6806", 2024-09-17T22:42:52.784 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:52.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.784 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.784 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6807", 2024-09-17T22:42:52.784 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:52.784 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.784 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.784 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:52.784 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.785 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.785 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6804", 2024-09-17T22:42:52.785 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:52.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.785 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.785 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6805", 2024-09-17T22:42:52.785 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:52.785 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.785 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.785 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6801/453192890", 2024-09-17T22:42:52.786 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6803/453192890", 2024-09-17T22:42:52.786 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6807/453192890", 2024-09-17T22:42:52.786 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6805/453192890", 2024-09-17T22:42:52.786 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:52.786 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:52.786 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:52.786 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.786 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 4, 2024-09-17T22:42:52.786 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "4cb6973a-63ae-4f48-af94-37610aee3c06", 2024-09-17T22:42:52.786 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:52.786 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:52.786 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:52.787 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:52.787 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:52.787 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:52.787 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:52.787 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 51, 2024-09-17T22:42:52.787 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:52.787 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:52.787 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:52.787 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.787 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.787 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6800", 2024-09-17T22:42:52.787 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:52.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.788 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.788 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6801", 2024-09-17T22:42:52.788 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:52.788 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.788 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.788 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:52.788 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.788 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.788 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6802", 2024-09-17T22:42:52.789 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:52.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.789 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.789 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6803", 2024-09-17T22:42:52.789 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:52.789 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.789 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.789 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:52.789 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.789 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.790 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6806", 2024-09-17T22:42:52.790 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:52.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.790 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.790 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6807", 2024-09-17T22:42:52.790 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:52.790 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.790 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.790 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:52.790 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.791 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.791 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6804", 2024-09-17T22:42:52.791 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:52.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.791 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.791 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6805", 2024-09-17T22:42:52.791 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:52.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.791 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.791 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6801/380388860", 2024-09-17T22:42:52.791 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6803/380388860", 2024-09-17T22:42:52.792 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6807/380388860", 2024-09-17T22:42:52.792 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6805/380388860", 2024-09-17T22:42:52.792 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:52.792 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:52.792 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:52.792 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.792 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 5, 2024-09-17T22:42:52.792 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "ef9fe11b-b719-48f4-9c6c-e1b1b355acdc", 2024-09-17T22:42:52.792 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:52.792 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:52.792 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:52.793 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:52.793 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:52.793 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:52.793 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:52.793 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 23, 2024-09-17T22:42:52.793 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:52.793 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:52.793 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:52.793 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.793 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.793 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6808", 2024-09-17T22:42:52.793 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:52.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.794 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.794 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6809", 2024-09-17T22:42:52.794 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:52.794 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.794 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.794 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:52.794 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.794 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.794 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6810", 2024-09-17T22:42:52.794 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:52.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.795 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.795 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6811", 2024-09-17T22:42:52.795 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:52.795 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.795 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.795 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:52.795 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.795 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.795 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6814", 2024-09-17T22:42:52.795 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:52.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.796 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.796 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6815", 2024-09-17T22:42:52.796 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:52.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.796 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.796 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:52.796 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.796 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.796 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6812", 2024-09-17T22:42:52.797 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:52.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.797 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.797 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6813", 2024-09-17T22:42:52.797 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:52.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.797 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.797 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6809/943488949", 2024-09-17T22:42:52.797 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6811/943488949", 2024-09-17T22:42:52.797 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6815/943488949", 2024-09-17T22:42:52.797 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6813/943488949", 2024-09-17T22:42:52.798 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:52.798 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:52.798 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:52.798 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.798 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 6, 2024-09-17T22:42:52.798 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "e4045aec-22c1-401b-bfbf-64f115bc32af", 2024-09-17T22:42:52.798 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:52.798 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:52.798 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:52.798 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:52.798 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:52.798 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:52.799 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:52.799 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 99, 2024-09-17T22:42:52.799 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:52.799 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:52.799 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:52.799 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.799 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.799 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6816", 2024-09-17T22:42:52.799 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:52.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.799 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.800 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6817", 2024-09-17T22:42:52.800 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:52.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.800 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.800 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:52.800 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.800 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.800 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6818", 2024-09-17T22:42:52.800 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:52.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.801 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.801 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6819", 2024-09-17T22:42:52.801 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:52.801 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.801 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.801 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:52.801 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.801 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.801 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6822", 2024-09-17T22:42:52.801 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:52.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.802 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.802 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6823", 2024-09-17T22:42:52.802 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:52.802 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.802 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.802 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:52.802 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.802 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.802 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6820", 2024-09-17T22:42:52.802 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:52.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.803 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.803 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6821", 2024-09-17T22:42:52.803 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:52.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.803 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.803 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6817/1873501077", 2024-09-17T22:42:52.803 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6819/1873501077", 2024-09-17T22:42:52.803 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6823/1873501077", 2024-09-17T22:42:52.803 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6821/1873501077", 2024-09-17T22:42:52.803 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:52.803 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:52.804 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:52.804 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.804 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 7, 2024-09-17T22:42:52.804 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "4a05a5d7-8281-4572-a9d0-6095ee5e168b", 2024-09-17T22:42:52.804 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:52.804 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:52.804 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:52.804 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:52.804 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:52.804 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:52.804 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:52.804 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 54, 2024-09-17T22:42:52.805 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:52.805 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:52.805 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:52.805 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.805 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.805 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6824", 2024-09-17T22:42:52.805 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:52.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.806 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.806 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6825", 2024-09-17T22:42:52.806 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:52.806 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.806 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.806 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:52.806 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.806 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.806 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6826", 2024-09-17T22:42:52.806 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:52.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.807 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.807 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6827", 2024-09-17T22:42:52.807 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:52.807 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.807 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.807 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:52.807 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.807 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.807 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6830", 2024-09-17T22:42:52.807 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:52.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.808 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.808 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6831", 2024-09-17T22:42:52.808 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:52.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.808 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.808 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:52.808 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:52.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.808 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:52.808 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6828", 2024-09-17T22:42:52.809 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:52.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.809 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:52.809 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6829", 2024-09-17T22:42:52.809 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:52.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.809 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.809 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6825/3609357666", 2024-09-17T22:42:52.809 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6827/3609357666", 2024-09-17T22:42:52.809 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6831/3609357666", 2024-09-17T22:42:52.809 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6829/3609357666", 2024-09-17T22:42:52.809 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:52.810 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:52.810 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:52.810 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.810 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.810 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:52.810 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_xinfo": [ 2024-09-17T22:42:52.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.810 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 0, 2024-09-17T22:42:52.810 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:52.810 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:52.810 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:52.810 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:52.810 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:52.810 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:59.351758+0000", 2024-09-17T22:42:52.811 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 51 2024-09-17T22:42:52.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.811 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 1, 2024-09-17T22:42:52.811 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:52.811 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:52.811 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:52.811 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:52.811 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:52.811 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:59.058915+0000", 2024-09-17T22:42:52.811 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:52.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.812 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 2, 2024-09-17T22:42:52.812 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:52.812 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:52.812 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:52.812 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:52.812 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:52.812 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.922383+0000", 2024-09-17T22:42:52.812 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:52.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.812 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 3, 2024-09-17T22:42:52.812 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:52.812 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:52.812 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:52.813 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:52.813 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:52.813 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.802379+0000", 2024-09-17T22:42:52.813 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:52.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.813 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 4, 2024-09-17T22:42:52.813 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:52.813 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:52.813 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:52.813 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:52.813 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:52.814 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.881681+0000", 2024-09-17T22:42:52.814 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:52.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.814 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 5, 2024-09-17T22:42:52.814 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:52.814 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:52.814 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:52.814 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:52.814 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:52.814 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.616649+0000", 2024-09-17T22:42:52.814 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:52.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.815 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 6, 2024-09-17T22:42:52.815 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:52.815 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:52.815 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:52.815 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:52.815 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:52.815 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.589555+0000", 2024-09-17T22:42:52.815 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:52.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.815 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 7, 2024-09-17T22:42:52.815 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:52.816 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:52.816 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:52.816 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:52.816 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:52.816 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:59.047783+0000", 2024-09-17T22:42:52.816 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:52.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.816 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:52.816 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_upmap": [ 2024-09-17T22:42:52.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.816 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.1c", 2024-09-17T22:42:52.816 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:42:52.816 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-09-17T22:42:52.816 INFO:teuthology.orchestra.run.smithi115.stdout: 3 2024-09-17T22:42:52.817 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.817 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.817 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:52.817 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_upmap_items": [ 2024-09-17T22:42:52.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.817 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.12", 2024-09-17T22:42:52.817 INFO:teuthology.orchestra.run.smithi115.stdout: "mappings": [ 2024-09-17T22:42:52.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.817 INFO:teuthology.orchestra.run.smithi115.stdout: "from": 3, 2024-09-17T22:42:52.817 INFO:teuthology.orchestra.run.smithi115.stdout: "to": 0 2024-09-17T22:42:52.817 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.817 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.818 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.16", 2024-09-17T22:42:52.818 INFO:teuthology.orchestra.run.smithi115.stdout: "mappings": [ 2024-09-17T22:42:52.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.818 INFO:teuthology.orchestra.run.smithi115.stdout: "from": 2, 2024-09-17T22:42:52.818 INFO:teuthology.orchestra.run.smithi115.stdout: "to": 0 2024-09-17T22:42:52.818 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.819 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.819 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.1d", 2024-09-17T22:42:52.819 INFO:teuthology.orchestra.run.smithi115.stdout: "mappings": [ 2024-09-17T22:42:52.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.819 INFO:teuthology.orchestra.run.smithi115.stdout: "from": 3, 2024-09-17T22:42:52.819 INFO:teuthology.orchestra.run.smithi115.stdout: "to": 7 2024-09-17T22:42:52.819 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.819 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.819 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.819 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:52.819 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_upmap_primaries": [], 2024-09-17T22:42:52.819 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_temp": [ 2024-09-17T22:42:52.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.820 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.12", 2024-09-17T22:42:52.820 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:42:52.820 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-09-17T22:42:52.820 INFO:teuthology.orchestra.run.smithi115.stdout: 3 2024-09-17T22:42:52.820 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.820 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.16", 2024-09-17T22:42:52.820 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:42:52.820 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-09-17T22:42:52.820 INFO:teuthology.orchestra.run.smithi115.stdout: 2 2024-09-17T22:42:52.820 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.821 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.1d", 2024-09-17T22:42:52.821 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:42:52.821 INFO:teuthology.orchestra.run.smithi115.stdout: 7, 2024-09-17T22:42:52.821 INFO:teuthology.orchestra.run.smithi115.stdout: 3 2024-09-17T22:42:52.821 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.821 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.821 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:52.821 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_temp": [], 2024-09-17T22:42:52.821 INFO:teuthology.orchestra.run.smithi115.stdout: "blocklist": {}, 2024-09-17T22:42:52.821 INFO:teuthology.orchestra.run.smithi115.stdout: "range_blocklist": {}, 2024-09-17T22:42:52.821 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profiles": { 2024-09-17T22:42:52.822 INFO:teuthology.orchestra.run.smithi115.stdout: "default": { 2024-09-17T22:42:52.822 INFO:teuthology.orchestra.run.smithi115.stdout: "crush-failure-domain": "osd", 2024-09-17T22:42:52.822 INFO:teuthology.orchestra.run.smithi115.stdout: "k": "2", 2024-09-17T22:42:52.822 INFO:teuthology.orchestra.run.smithi115.stdout: "m": "1", 2024-09-17T22:42:52.822 INFO:teuthology.orchestra.run.smithi115.stdout: "plugin": "jerasure", 2024-09-17T22:42:52.822 INFO:teuthology.orchestra.run.smithi115.stdout: "technique": "reed_sol_van" 2024-09-17T22:42:52.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:52.822 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps_queue": [ 2024-09-17T22:42:52.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.822 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 3, 2024-09-17T22:42:52.822 INFO:teuthology.orchestra.run.smithi115.stdout: "snaps": [ 2024-09-17T22:42:52.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.823 INFO:teuthology.orchestra.run.smithi115.stdout: "begin": 27, 2024-09-17T22:42:52.823 INFO:teuthology.orchestra.run.smithi115.stdout: "length": 14 2024-09-17T22:42:52.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.823 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.823 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:52.823 INFO:teuthology.orchestra.run.smithi115.stdout: "new_removed_snaps": [ 2024-09-17T22:42:52.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.823 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 3, 2024-09-17T22:42:52.823 INFO:teuthology.orchestra.run.smithi115.stdout: "snaps": [ 2024-09-17T22:42:52.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:52.823 INFO:teuthology.orchestra.run.smithi115.stdout: "begin": 27, 2024-09-17T22:42:52.823 INFO:teuthology.orchestra.run.smithi115.stdout: "length": 1 2024-09-17T22:42:52.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.824 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:52.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.824 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:52.824 INFO:teuthology.orchestra.run.smithi115.stdout: "new_purged_snaps": [], 2024-09-17T22:42:52.824 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_node_flags": {}, 2024-09-17T22:42:52.824 INFO:teuthology.orchestra.run.smithi115.stdout: "device_class_flags": {}, 2024-09-17T22:42:52.824 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": { 2024-09-17T22:42:52.824 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode_enabled": false, 2024-09-17T22:42:52.824 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_bucket_count": 0, 2024-09-17T22:42:52.824 INFO:teuthology.orchestra.run.smithi115.stdout: "degraded_stretch_mode": 0, 2024-09-17T22:42:52.824 INFO:teuthology.orchestra.run.smithi115.stdout: "recovering_stretch_mode": 0, 2024-09-17T22:42:52.824 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode_bucket": 0 2024-09-17T22:42:52.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:52.825 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:52.826 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:52.826 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:52.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158673 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.57621348, 2024-09-17T22:42:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039400614, 2024-09-17T22:42:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158673 2024-09-17T22:42:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.04e-07 2024-09-17T22:42:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241563+0000", 2024-09-17T22:42:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5799999999999998e-07 2024-09-17T22:42:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241566+0000", 2024-09-17T22:42:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8279999999999999e-06 2024-09-17T22:42:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241568+0000", 2024-09-17T22:42:52.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2709999999999999e-06 2024-09-17T22:42:52.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280954+0000", 2024-09-17T22:42:52.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039386051999999998 2024-09-17T22:42:52.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280962+0000", 2024-09-17T22:42:52.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3010000000000007e-06 2024-09-17T22:42:52.831 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.831 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.831 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.831 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158675 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.831 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:52.831 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.576204189000002, 2024-09-17T22:42:52.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039408033000000002, 2024-09-17T22:42:52.831 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.831 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.831 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.831 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.832 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.832 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.832 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158675 2024-09-17T22:42:52.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.832 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:52.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.36e-07 2024-09-17T22:42:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241572+0000", 2024-09-17T22:42:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6599999999999994e-07 2024-09-17T22:42:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241575+0000", 2024-09-17T22:42:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8389999999999998e-06 2024-09-17T22:42:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241577+0000", 2024-09-17T22:42:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4119999999999999e-06 2024-09-17T22:42:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280975+0000", 2024-09-17T22:42:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039397205999999997 2024-09-17T22:42:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280979+0000", 2024-09-17T22:42:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.374e-06 2024-09-17T22:42:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158677 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.576181746000003, 2024-09-17T22:42:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039409307999999997, 2024-09-17T22:42:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158677 2024-09-17T22:42:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241594+0000", 2024-09-17T22:42:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:42:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241595+0000", 2024-09-17T22:42:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2729999999999999e-06 2024-09-17T22:42:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241602+0000", 2024-09-17T22:42:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:42:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241606+0000", 2024-09-17T22:42:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3019999999999999e-06 2024-09-17T22:42:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280994+0000", 2024-09-17T22:42:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039388484000000001 2024-09-17T22:42:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281003+0000", 2024-09-17T22:42:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5459999999999992e-06 2024-09-17T22:42:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158679 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.576166067999999, 2024-09-17T22:42:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039413403999999999, 2024-09-17T22:42:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158679 2024-09-17T22:42:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4999999999999999e-07 2024-09-17T22:42:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241613+0000", 2024-09-17T22:42:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.207e-06 2024-09-17T22:42:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241616+0000", 2024-09-17T22:42:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.275e-06 2024-09-17T22:42:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241618+0000", 2024-09-17T22:42:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:42:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281017+0000", 2024-09-17T22:42:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039398385000000001 2024-09-17T22:42:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281022+0000", 2024-09-17T22:42:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.896e-06 2024-09-17T22:42:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158681 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.576153890000001, 2024-09-17T22:42:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039425292000000001, 2024-09-17T22:42:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158681 2024-09-17T22:42:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:42:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241622+0000", 2024-09-17T22:42:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:42:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241625+0000", 2024-09-17T22:42:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9629999999999998e-06 2024-09-17T22:42:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241628+0000", 2024-09-17T22:42:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4339999999999999e-06 2024-09-17T22:42:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281039+0000", 2024-09-17T22:42:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039410672000000001 2024-09-17T22:42:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281047+0000", 2024-09-17T22:42:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9969999999999998e-06 2024-09-17T22:42:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158682 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.576144606, 2024-09-17T22:42:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039436455000000002, 2024-09-17T22:42:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158682 2024-09-17T22:42:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:42:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241632+0000", 2024-09-17T22:42:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:42:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241635+0000", 2024-09-17T22:42:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8940000000000002e-06 2024-09-17T22:42:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241637+0000", 2024-09-17T22:42:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3659999999999999e-06 2024-09-17T22:42:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281061+0000", 2024-09-17T22:42:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039423664999999997 2024-09-17T22:42:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281067+0000", 2024-09-17T22:42:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.285e-06 2024-09-17T22:42:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158684 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.576135260000001, 2024-09-17T22:42:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462980000000002, 2024-09-17T22:42:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158684 2024-09-17T22:42:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241641+0000", 2024-09-17T22:42:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:42:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241644+0000", 2024-09-17T22:42:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8870000000000001e-06 2024-09-17T22:42:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241646+0000", 2024-09-17T22:42:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3530000000000002e-06 2024-09-17T22:42:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281093+0000", 2024-09-17T22:42:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039446947000000003 2024-09-17T22:42:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281103+0000", 2024-09-17T22:42:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6299999999999993e-06 2024-09-17T22:42:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158686 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.576126090000002, 2024-09-17T22:42:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039474214000000001, 2024-09-17T22:42:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158686 2024-09-17T22:42:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:42:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241650+0000", 2024-09-17T22:42:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2999999999999999e-07 2024-09-17T22:42:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241653+0000", 2024-09-17T22:42:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9689999999999999e-06 2024-09-17T22:42:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241655+0000", 2024-09-17T22:42:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3460000000000001e-06 2024-09-17T22:42:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281118+0000", 2024-09-17T22:42:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462876000000001 2024-09-17T22:42:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281123+0000", 2024-09-17T22:42:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8840000000000002e-06 2024-09-17T22:42:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158688 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.576116859000003, 2024-09-17T22:42:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039490458999999999, 2024-09-17T22:42:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158688 2024-09-17T22:42:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:42:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241659+0000", 2024-09-17T22:42:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3699999999999999e-07 2024-09-17T22:42:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241662+0000", 2024-09-17T22:42:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8499999999999998e-06 2024-09-17T22:42:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241665+0000", 2024-09-17T22:42:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.356e-06 2024-09-17T22:42:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281141+0000", 2024-09-17T22:42:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039476031000000002 2024-09-17T22:42:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281149+0000", 2024-09-17T22:42:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0709999999999995e-06 2024-09-17T22:42:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158690 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.576107800000003, 2024-09-17T22:42:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039502378999999997, 2024-09-17T22:42:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158690 2024-09-17T22:42:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241668+0000", 2024-09-17T22:42:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241669+0000", 2024-09-17T22:42:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1009999999999999e-06 2024-09-17T22:42:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241672+0000", 2024-09-17T22:42:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9059999999999998e-06 2024-09-17T22:42:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241674+0000", 2024-09-17T22:42:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4439999999999998e-06 2024-09-17T22:42:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281162+0000", 2024-09-17T22:42:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039488189 2024-09-17T22:42:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281170+0000", 2024-09-17T22:42:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5310000000000004e-06 2024-09-17T22:42:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158692 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.576098367, 2024-09-17T22:42:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039531064999999997, 2024-09-17T22:42:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158692 2024-09-17T22:42:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241678+0000", 2024-09-17T22:42:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0800000000000003e-07 2024-09-17T22:42:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241681+0000", 2024-09-17T22:42:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8729999999999999e-06 2024-09-17T22:42:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241683+0000", 2024-09-17T22:42:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.447e-06 2024-09-17T22:42:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281199+0000", 2024-09-17T22:42:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039515388999999998 2024-09-17T22:42:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281208+0000", 2024-09-17T22:42:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2399999999999996e-06 2024-09-17T22:42:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158694 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.576089195999998, 2024-09-17T22:42:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039543986000000003, 2024-09-17T22:42:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158694 2024-09-17T22:42:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:42:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241687+0000", 2024-09-17T22:42:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:42:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241691+0000", 2024-09-17T22:42:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.811e-06 2024-09-17T22:42:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241693+0000", 2024-09-17T22:42:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.385e-06 2024-09-17T22:42:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281223+0000", 2024-09-17T22:42:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039529332 2024-09-17T22:42:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281230+0000", 2024-09-17T22:42:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2139999999999999e-06 2024-09-17T22:42:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158696 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.576078807999998, 2024-09-17T22:42:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039570458000000003, 2024-09-17T22:42:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158696 2024-09-17T22:42:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:42:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:42:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:42:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.33e-07 2024-09-17T22:42:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241700+0000", 2024-09-17T22:42:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8949999999999998e-06 2024-09-17T22:42:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241703+0000", 2024-09-17T22:42:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3159999999999999e-06 2024-09-17T22:42:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281257+0000", 2024-09-17T22:42:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039553862000000002 2024-09-17T22:42:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281267+0000", 2024-09-17T22:42:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.024e-05 2024-09-17T22:42:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158698 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.576068730000003, 2024-09-17T22:42:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039576493999999997, 2024-09-17T22:42:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158698 2024-09-17T22:42:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241707+0000", 2024-09-17T22:42:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:42:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241708+0000", 2024-09-17T22:42:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5000000000000001e-07 2024-09-17T22:42:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241711+0000", 2024-09-17T22:42:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9670000000000001e-06 2024-09-17T22:42:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241713+0000", 2024-09-17T22:42:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2689999999999998e-06 2024-09-17T22:42:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281278+0000", 2024-09-17T22:42:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039565206999999998 2024-09-17T22:42:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281283+0000", 2024-09-17T22:42:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:42:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158700 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.576059518999998, 2024-09-17T22:42:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039604292999999999, 2024-09-17T22:42:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158700 2024-09-17T22:42:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241717+0000", 2024-09-17T22:42:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.78e-07 2024-09-17T22:42:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241720+0000", 2024-09-17T22:42:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8119999999999999e-06 2024-09-17T22:42:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241722+0000", 2024-09-17T22:42:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-17T22:42:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281311+0000", 2024-09-17T22:42:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039588894999999999 2024-09-17T22:42:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281320+0000", 2024-09-17T22:42:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9989999999999997e-06 2024-09-17T22:42:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158702 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.576048196999999, 2024-09-17T22:42:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039616142, 2024-09-17T22:42:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158702 2024-09-17T22:42:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4400000000000001e-07 2024-09-17T22:42:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241728+0000", 2024-09-17T22:42:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0839999999999999e-06 2024-09-17T22:42:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241731+0000", 2024-09-17T22:42:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0690000000000002e-06 2024-09-17T22:42:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241734+0000", 2024-09-17T22:42:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4159999999999998e-06 2024-09-17T22:42:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281334+0000", 2024-09-17T22:42:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039600110000000001 2024-09-17T22:42:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281343+0000", 2024-09-17T22:42:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2189999999999993e-06 2024-09-17T22:42:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158704 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.576038333, 2024-09-17T22:42:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039641485999999997, 2024-09-17T22:42:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158704 2024-09-17T22:42:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:42:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241738+0000", 2024-09-17T22:42:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0589999999999999e-06 2024-09-17T22:42:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241741+0000", 2024-09-17T22:42:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.943e-06 2024-09-17T22:42:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241744+0000", 2024-09-17T22:42:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4819999999999999e-06 2024-09-17T22:42:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281370+0000", 2024-09-17T22:42:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039625661 2024-09-17T22:42:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281378+0000", 2024-09-17T22:42:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1310000000000006e-06 2024-09-17T22:42:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158705 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.576027783000001, 2024-09-17T22:42:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039652637999999997, 2024-09-17T22:42:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158705 2024-09-17T22:42:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:42:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-07 2024-09-17T22:42:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:42:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0599999999999999e-07 2024-09-17T22:42:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241751+0000", 2024-09-17T22:42:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.926e-06 2024-09-17T22:42:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241754+0000", 2024-09-17T22:42:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4360000000000001e-06 2024-09-17T22:42:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281392+0000", 2024-09-17T22:42:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039638566 2024-09-17T22:42:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281400+0000", 2024-09-17T22:42:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.571e-06 2024-09-17T22:42:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158707 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.576006681999999, 2024-09-17T22:42:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039654809999999999, 2024-09-17T22:42:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158707 2024-09-17T22:42:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241769+0000", 2024-09-17T22:42:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4299999999999999e-07 2024-09-17T22:42:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241770+0000", 2024-09-17T22:42:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3769999999999999e-06 2024-09-17T22:42:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241773+0000", 2024-09-17T22:42:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2109999999999998e-06 2024-09-17T22:42:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241776+0000", 2024-09-17T22:42:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5069999999999999e-06 2024-09-17T22:42:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281415+0000", 2024-09-17T22:42:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039639499000000002 2024-09-17T22:42:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281423+0000", 2024-09-17T22:42:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8730000000000007e-06 2024-09-17T22:42:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158709 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 62.575996072999999, 2024-09-17T22:42:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039661937000000001, 2024-09-17T22:42:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158709 2024-09-17T22:42:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242843+0000", 2024-09-17T22:42:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010634780000000001 2024-09-17T22:42:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242845+0000", 2024-09-17T22:42:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7209999999999999e-06 2024-09-17T22:42:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242851+0000", 2024-09-17T22:42:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6949999999999997e-06 2024-09-17T22:42:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242855+0000", 2024-09-17T22:42:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7960000000000002e-06 2024-09-17T22:42:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281434+0000", 2024-09-17T22:42:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038579519 2024-09-17T22:42:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281441+0000", 2024-09-17T22:42:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7279999999999998e-06 2024-09-17T22:42:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:52.913 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:52.915 INFO:tasks.thrashosds.thrasher:j is {'epoch': 104, 'fsid': '551c5ed8-e790-485a-bc95-a9578a29275c', 'created': '2024-09-17T22:40:50.189230+0000', 'modified': '2024-09-17T22:42:52.431088+0000', 'last_up_change': '2024-09-17T22:41:53.099515+0000', 'last_in_change': '2024-09-17T22:40:52.260526+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T22:41:02.338356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8128433227539062, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.8100000023841858}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T22:41:08.097861+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.619999885559082, 'score_stable': 2.619999885559082, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8128433227539062, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.8100000023841858}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T22:41:17.501542+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 40, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.309999942779541, 'score_stable': 1.309999942779541, 'optimal_score': 1, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 0.8128433227539062, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.8100000023841858}}], 'osds': [{'osd': 0, 'uuid': '3bd5c11f-e7c2-4968-839a-2799c00d93a6', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 51, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6824', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6825', 'nonce': 2782271783}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 2782271783}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 2782271783}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 2782271783}]}, 'public_addr': '172.21.15.115:6825/2782271783', 'cluster_addr': '172.21.15.115:6827/2782271783', 'heartbeat_back_addr': '172.21.15.115:6831/2782271783', 'heartbeat_front_addr': '172.21.15.115:6829/2782271783', 'state': ['exists']}, {'osd': 1, 'uuid': '02918203-cf2e-49fc-b994-743b5d81b4a1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 1941780390}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 1941780390}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 1941780390}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 1941780390}]}, 'public_addr': '172.21.15.115:6817/1941780390', 'cluster_addr': '172.21.15.115:6819/1941780390', 'heartbeat_back_addr': '172.21.15.115:6823/1941780390', 'heartbeat_front_addr': '172.21.15.115:6821/1941780390', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '22ee34ef-522e-4121-85b7-38bbafeca5de', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 689014698}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 689014698}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 689014698}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 689014698}]}, 'public_addr': '172.21.15.115:6809/689014698', 'cluster_addr': '172.21.15.115:6811/689014698', 'heartbeat_back_addr': '172.21.15.115:6815/689014698', 'heartbeat_front_addr': '172.21.15.115:6813/689014698', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5d464921-5696-43c5-b89d-430a54ddefb5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.50274658203125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 453192890}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 453192890}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 453192890}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 453192890}]}, 'public_addr': '172.21.15.115:6801/453192890', 'cluster_addr': '172.21.15.115:6803/453192890', 'heartbeat_back_addr': '172.21.15.115:6807/453192890', 'heartbeat_front_addr': '172.21.15.115:6805/453192890', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '4cb6973a-63ae-4f48-af94-37610aee3c06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6800', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6801', 'nonce': 380388860}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6802', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6803', 'nonce': 380388860}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6806', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6807', 'nonce': 380388860}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6804', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6805', 'nonce': 380388860}]}, 'public_addr': '172.21.15.129:6801/380388860', 'cluster_addr': '172.21.15.129:6803/380388860', 'heartbeat_back_addr': '172.21.15.129:6807/380388860', 'heartbeat_front_addr': '172.21.15.129:6805/380388860', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ef9fe11b-b719-48f4-9c6c-e1b1b355acdc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6808', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6809', 'nonce': 943488949}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6810', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6811', 'nonce': 943488949}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6814', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6815', 'nonce': 943488949}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6812', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6813', 'nonce': 943488949}]}, 'public_addr': '172.21.15.129:6809/943488949', 'cluster_addr': '172.21.15.129:6811/943488949', 'heartbeat_back_addr': '172.21.15.129:6815/943488949', 'heartbeat_front_addr': '172.21.15.129:6813/943488949', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e4045aec-22c1-401b-bfbf-64f115bc32af', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6816', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6817', 'nonce': 1873501077}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6818', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6819', 'nonce': 1873501077}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6822', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6823', 'nonce': 1873501077}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6820', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6821', 'nonce': 1873501077}]}, 'public_addr': '172.21.15.129:6817/1873501077', 'cluster_addr': '172.21.15.129:6819/1873501077', 'heartbeat_back_addr': '172.21.15.129:6823/1873501077', 'heartbeat_front_addr': '172.21.15.129:6821/1873501077', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '4a05a5d7-8281-4572-a9d0-6095ee5e168b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6824', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6825', 'nonce': 3609357666}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6826', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6827', 'nonce': 3609357666}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6830', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6831', 'nonce': 3609357666}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6828', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6829', 'nonce': 3609357666}]}, 'public_addr': '172.21.15.129:6825/3609357666', 'cluster_addr': '172.21.15.129:6827/3609357666', 'heartbeat_back_addr': '172.21.15.129:6831/3609357666', 'heartbeat_front_addr': '172.21.15.129:6829/3609357666', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:59.351758+0000', 'dead_epoch': 51}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:59.058915+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.922383+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.802379+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.881681+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.616649+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.589555+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:59.047783+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1c', 'osds': [1, 3]}], 'pg_upmap_items': [{'pgid': '3.12', 'mappings': [{'from': 3, 'to': 0}]}, {'pgid': '3.16', 'mappings': [{'from': 2, 'to': 0}]}, {'pgid': '3.1d', 'mappings': [{'from': 3, 'to': 7}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.12', 'osds': [6, 3]}, {'pgid': '3.16', 'osds': [3, 2]}, {'pgid': '3.1d', 'osds': [7, 3]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 27, 'length': 14}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 27, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T22:42:52.915 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:42:52.915 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:52.824+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:52.916 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:42:52.960 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:52.958+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:53.020 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:53.020 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:53.020 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:53.020 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:53.027 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:53.025+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:53.029 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:42:53.128 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:53.126+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:53.172 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:53.172 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:53.172 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:53.172 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:53.172 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:53.181 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:42:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:42:53.232 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:42:53.243 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-17T22:42:52.435544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1057501113,"num_objects":1582,"num_object_clones":356,"num_object_copies":3164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":400,"num_objects_misplaced":132,"num_objects_unfound":0,"num_objects_dirty":1582,"num_whiteouts":832,"num_read":261522,"num_read_kb":796608,"num_write":17035,"num_write_kb":295801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":10513915,"num_keys_recovered":74,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14661,"ondisk_log_size":14661,"up":70,"acting":73,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":351772,"kb_used_data":36876,"kb_used_omap":12,"kb_used_meta":314547,"kb_avail":754622948,"statfs":{"total":773094113280,"available":772733898752,"internally_reserved":0,"allocated":37761024,"data_stored":239070950,"data_compressed":7710289,"data_compressed_allocated":20545536,"data_compressed_original":229961595,"omap_allocated":12750,"internal_metadata":322096690},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-96968206,"num_objects":-134,"num_object_clones":-51,"num_object_copies":-268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-6,"num_objects_misplaced":-16,"num_objects_unfound":0,"num_objects_dirty":-134,"num_whiteouts":-38,"num_read":1382,"num_read_kb":1177,"num_write":188,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.024536"},"pg_stats":[{"pgid":"3.1f","version":"99'803","reported_seq":6941,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T22:42:50.427887+0000","last_change":"2024-09-17T22:42:50.428029+0000","last_active":"2024-09-17T22:42:50.427887+0000","last_peered":"2024-09-17T22:42:50.427887+0000","last_clean":"2024-09-17T22:42:50.427887+0000","last_became_active":"2024-09-17T22:41:19.443658+0000","last_became_peered":"2024-09-17T22:41:19.443658+0000","last_unstale":"2024-09-17T22:42:50.427887+0000","last_undegraded":"2024-09-17T22:42:50.427887+0000","last_fullsized":"2024-09-17T22:42:50.427887+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:14:29.109484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034006299999999999,"stat_sum":{"num_bytes":32931362,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":20,"num_read":8614,"num_read_kb":14735,"num_write":1062,"num_write_kb":4537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"c"}]},{"pgid":"3.1e","version":"102'751","reported_seq":7146,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T22:42:50.493442+0000","last_change":"2024-09-17T22:42:50.428929+0000","last_active":"2024-09-17T22:42:50.493442+0000","last_peered":"2024-09-17T22:42:50.493442+0000","last_clean":"2024-09-17T22:42:50.493442+0000","last_became_active":"2024-09-17T22:41:19.449357+0000","last_became_peered":"2024-09-17T22:41:19.449357+0000","last_unstale":"2024-09-17T22:42:50.493442+0000","last_undegraded":"2024-09-17T22:42:50.493442+0000","last_fullsized":"2024-09-17T22:42:50.493442+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:56.997080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001303394,"stat_sum":{"num_bytes":54559417,"num_objects":54,"num_object_clones":10,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":29,"num_read":8851,"num_read_kb":26531,"num_write":1111,"num_write_kb":10958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"c"}]},{"pgid":"3.1d","version":"99'389","reported_seq":2006,"reported_epoch":102,"state":"active+clean+remapped","last_fresh":"2024-09-17T22:42:50.425766+0000","last_change":"2024-09-17T22:42:50.426248+0000","last_active":"2024-09-17T22:42:50.425766+0000","last_peered":"2024-09-17T22:42:50.425766+0000","last_clean":"2024-09-17T22:42:50.425766+0000","last_became_active":"2024-09-17T22:41:57.133365+0000","last_became_peered":"2024-09-17T22:41:57.133365+0000","last_unstale":"2024-09-17T22:42:50.425766+0000","last_undegraded":"2024-09-17T22:42:50.425766+0000","last_fullsized":"2024-09-17T22:42:50.425766+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:56:07.961009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077212799999999996,"stat_sum":{"num_bytes":29059604,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":34,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":17,"num_read":17371,"num_read_kb":48211,"num_write":351,"num_write_kb":8981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":3161143,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[{"shards":"3,7","objects":34}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"c"}]},{"pgid":"3.1c","version":"103'442","reported_seq":2446,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T22:42:51.505473+0000","last_change":"2024-09-17T22:42:51.438491+0000","last_active":"2024-09-17T22:42:51.505473+0000","last_peered":"2024-09-17T22:42:51.505473+0000","last_clean":"2024-09-17T22:42:51.505473+0000","last_became_active":"2024-09-17T22:42:34.310411+0000","last_became_peered":"2024-09-17T22:42:34.310411+0000","last_unstale":"2024-09-17T22:42:51.505473+0000","last_undegraded":"2024-09-17T22:42:51.505473+0000","last_fullsized":"2024-09-17T22:42:51.505473+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:07:26.543213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082252900000000001,"stat_sum":{"num_bytes":9625026,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":19,"num_read":2461,"num_read_kb":14063,"num_write":544,"num_write_kb":12172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":7352772,"num_keys_recovered":74,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"d"}]},{"pgid":"3.1b","version":"101'207","reported_seq":7666,"reported_epoch":102,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:50.422802+0000","last_change":"2024-09-17T22:41:54.118027+0000","last_active":"2024-09-17T22:42:50.422802+0000","last_peered":"2024-09-17T22:42:50.422802+0000","last_clean":"2024-09-17T22:41:52.368521+0000","last_became_active":"2024-09-17T22:41:54.118027+0000","last_became_peered":"2024-09-17T22:41:54.118027+0000","last_unstale":"2024-09-17T22:42:50.422802+0000","last_undegraded":"2024-09-17T22:41:54.116518+0000","last_fullsized":"2024-09-17T22:41:54.116411+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:48:01.323462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055813,"stat_sum":{"num_bytes":30373227,"num_objects":49,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":26,"num_read":7315,"num_read_kb":22822,"num_write":177,"num_write_kb":6215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":49}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"101'296","reported_seq":4699,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T22:42:51.559917+0000","last_change":"2024-09-17T22:42:51.446479+0000","last_active":"2024-09-17T22:42:51.559917+0000","last_peered":"2024-09-17T22:42:51.559917+0000","last_clean":"2024-09-17T22:42:51.559917+0000","last_became_active":"2024-09-17T22:41:19.446541+0000","last_became_peered":"2024-09-17T22:41:19.446541+0000","last_unstale":"2024-09-17T22:42:51.559917+0000","last_undegraded":"2024-09-17T22:42:51.559917+0000","last_fullsized":"2024-09-17T22:42:51.559917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:15:40.230249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084327020000000003,"stat_sum":{"num_bytes":33193194,"num_objects":60,"num_object_clones":9,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":36,"num_read":4710,"num_read_kb":19338,"num_write":287,"num_write_kb":4169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"d"}]},{"pgid":"3.19","version":"99'330","reported_seq":6456,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T22:42:50.422953+0000","last_change":"2024-09-17T22:42:50.423348+0000","last_active":"2024-09-17T22:42:50.422953+0000","last_peered":"2024-09-17T22:42:50.422953+0000","last_clean":"2024-09-17T22:42:50.422953+0000","last_became_active":"2024-09-17T22:41:19.449940+0000","last_became_peered":"2024-09-17T22:41:19.449940+0000","last_unstale":"2024-09-17T22:42:50.422953+0000","last_undegraded":"2024-09-17T22:42:50.422953+0000","last_fullsized":"2024-09-17T22:42:50.422953+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:18:45.803904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058703399999999995,"stat_sum":{"num_bytes":33774074,"num_objects":57,"num_object_clones":12,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":28,"num_read":6605,"num_read_kb":22100,"num_write":307,"num_write_kb":9030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"c"}]},{"pgid":"3.18","version":"101'328","reported_seq":2849,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T22:42:50.428987+0000","last_change":"2024-09-17T22:42:50.429166+0000","last_active":"2024-09-17T22:42:50.428987+0000","last_peered":"2024-09-17T22:42:50.428987+0000","last_clean":"2024-09-17T22:42:50.428987+0000","last_became_active":"2024-09-17T22:41:19.446287+0000","last_became_peered":"2024-09-17T22:41:19.446287+0000","last_unstale":"2024-09-17T22:42:50.428987+0000","last_undegraded":"2024-09-17T22:42:50.428987+0000","last_fullsized":"2024-09-17T22:42:50.428987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:20.115987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001432263,"stat_sum":{"num_bytes":75832695,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":26,"num_read":3230,"num_read_kb":19901,"num_write":295,"num_write_kb":10924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"c"}]},{"pgid":"3.17","version":"102'465","reported_seq":2466,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T22:42:50.431315+0000","last_change":"2024-09-17T22:42:50.431315+0000","last_active":"2024-09-17T22:42:50.431315+0000","last_peered":"2024-09-17T22:42:50.431315+0000","last_clean":"2024-09-17T22:42:50.431315+0000","last_became_active":"2024-09-17T22:41:19.445923+0000","last_became_peered":"2024-09-17T22:41:19.445923+0000","last_unstale":"2024-09-17T22:42:50.431315+0000","last_undegraded":"2024-09-17T22:42:50.431315+0000","last_fullsized":"2024-09-17T22:42:50.431315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:55:25.283454+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0036187089999999999,"stat_sum":{"num_bytes":20889184,"num_objects":51,"num_object_clones":12,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":26,"num_read":3412,"num_read_kb":15080,"num_write":488,"num_write_kb":8710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"c"}]},{"pgid":"3.16","version":"101'487","reported_seq":22028,"reported_epoch":101,"state":"active+clean+remapped","last_fresh":"2024-09-17T22:42:49.421506+0000","last_change":"2024-09-17T22:42:49.421506+0000","last_active":"2024-09-17T22:42:49.421506+0000","last_peered":"2024-09-17T22:42:49.421506+0000","last_clean":"2024-09-17T22:42:49.421506+0000","last_became_active":"2024-09-17T22:41:57.132018+0000","last_became_peered":"2024-09-17T22:41:57.132018+0000","last_unstale":"2024-09-17T22:42:49.421506+0000","last_undegraded":"2024-09-17T22:42:49.421506+0000","last_fullsized":"2024-09-17T22:42:49.421506+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:19:05.536209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001455935,"stat_sum":{"num_bytes":47255584,"num_objects":58,"num_object_clones":17,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":58,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":32,"num_read":22628,"num_read_kb":63836,"num_write":567,"num_write_kb":12641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":58}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"b"}]},{"pgid":"3.15","version":"96'340","reported_seq":6946,"reported_epoch":102,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:50.524808+0000","last_change":"2024-09-17T22:41:54.119655+0000","last_active":"2024-09-17T22:42:50.524808+0000","last_peered":"2024-09-17T22:42:50.524808+0000","last_clean":"2024-09-17T22:41:52.245886+0000","last_became_active":"2024-09-17T22:41:54.119655+0000","last_became_peered":"2024-09-17T22:41:54.119655+0000","last_unstale":"2024-09-17T22:42:50.524808+0000","last_undegraded":"2024-09-17T22:41:54.119161+0000","last_fullsized":"2024-09-17T22:41:54.119071+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:40:54.561004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041055799999999999,"stat_sum":{"num_bytes":62197187,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":27,"num_read":8219,"num_read_kb":19879,"num_write":285,"num_write_kb":4804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":48}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"99'543","reported_seq":7556,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-17T22:42:49.421593+0000","last_change":"2024-09-17T22:42:49.422083+0000","last_active":"2024-09-17T22:42:49.421593+0000","last_peered":"2024-09-17T22:42:49.421593+0000","last_clean":"2024-09-17T22:42:49.421593+0000","last_became_active":"2024-09-17T22:41:19.451631+0000","last_became_peered":"2024-09-17T22:41:19.451631+0000","last_unstale":"2024-09-17T22:42:49.421593+0000","last_undegraded":"2024-09-17T22:42:49.421593+0000","last_fullsized":"2024-09-17T22:42:49.421593+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:44:30.216208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014612200000000001,"stat_sum":{"num_bytes":21610776,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":17,"num_read":8274,"num_read_kb":19410,"num_write":706,"num_write_kb":19536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"b"}]},{"pgid":"2.7","version":"0'0","reported_seq":181,"reported_epoch":101,"state":"active+undersized","last_fresh":"2024-09-17T22:42:49.420349+0000","last_change":"2024-09-17T22:41:54.118987+0000","last_active":"2024-09-17T22:42:49.420349+0000","last_peered":"2024-09-17T22:42:49.420349+0000","last_clean":"2024-09-17T22:41:52.098057+0000","last_became_active":"2024-09-17T22:41:54.118987+0000","last_became_peered":"2024-09-17T22:41:54.118987+0000","last_unstale":"2024-09-17T22:42:49.420349+0000","last_undegraded":"2024-09-17T22:42:49.420349+0000","last_fullsized":"2024-09-17T22:41:54.114557+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.941682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"102'499","reported_seq":5021,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T22:42:50.425199+0000","last_change":"2024-09-17T22:42:50.425199+0000","last_active":"2024-09-17T22:42:50.425199+0000","last_peered":"2024-09-17T22:42:50.425199+0000","last_clean":"2024-09-17T22:42:50.425199+0000","last_became_active":"2024-09-17T22:41:19.448966+0000","last_became_peered":"2024-09-17T22:41:19.448966+0000","last_unstale":"2024-09-17T22:42:50.425199+0000","last_undegraded":"2024-09-17T22:42:50.425199+0000","last_fullsized":"2024-09-17T22:42:50.425199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:31:23.205627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021432000000000001,"stat_sum":{"num_bytes":25622777,"num_objects":51,"num_object_clones":14,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":22,"num_read":6223,"num_read_kb":29894,"num_write":511,"num_write_kb":6238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"c"}]},{"pgid":"2.6","version":"0'0","reported_seq":182,"reported_epoch":102,"state":"active+undersized","last_fresh":"2024-09-17T22:42:50.422969+0000","last_change":"2024-09-17T22:41:54.118378+0000","last_active":"2024-09-17T22:42:50.422969+0000","last_peered":"2024-09-17T22:42:50.422969+0000","last_clean":"2024-09-17T22:41:52.097696+0000","last_became_active":"2024-09-17T22:41:54.118378+0000","last_became_peered":"2024-09-17T22:41:54.118378+0000","last_unstale":"2024-09-17T22:42:50.422969+0000","last_undegraded":"2024-09-17T22:42:50.422969+0000","last_fullsized":"2024-09-17T22:41:54.116433+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:48:40.138581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"99'338","reported_seq":1481,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T22:42:50.424308+0000","last_change":"2024-09-17T22:42:50.424787+0000","last_active":"2024-09-17T22:42:50.424308+0000","last_peered":"2024-09-17T22:42:50.424308+0000","last_clean":"2024-09-17T22:42:50.424308+0000","last_became_active":"2024-09-17T22:41:19.451637+0000","last_became_peered":"2024-09-17T22:41:19.451637+0000","last_unstale":"2024-09-17T22:42:50.424308+0000","last_undegraded":"2024-09-17T22:42:50.424308+0000","last_fullsized":"2024-09-17T22:42:50.424308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:09:15.621471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018643869999999999,"stat_sum":{"num_bytes":46829265,"num_objects":62,"num_object_clones":17,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":32,"num_read":1443,"num_read_kb":9951,"num_write":382,"num_write_kb":7019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"c"}]},{"pgid":"2.5","version":"0'0","reported_seq":179,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-17T22:42:49.420339+0000","last_change":"2024-09-17T22:41:11.397753+0000","last_active":"2024-09-17T22:42:49.420339+0000","last_peered":"2024-09-17T22:42:49.420339+0000","last_clean":"2024-09-17T22:42:49.420339+0000","last_became_active":"2024-09-17T22:41:09.387651+0000","last_became_peered":"2024-09-17T22:41:09.387651+0000","last_unstale":"2024-09-17T22:42:49.420339+0000","last_undegraded":"2024-09-17T22:42:49.420339+0000","last_fullsized":"2024-09-17T22:42:49.420339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:14:30.915047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"100'459","reported_seq":1179,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T22:42:50.507555+0000","last_change":"2024-09-17T22:42:50.507929+0000","last_active":"2024-09-17T22:42:50.507555+0000","last_peered":"2024-09-17T22:42:50.507555+0000","last_clean":"2024-09-17T22:42:50.507555+0000","last_became_active":"2024-09-17T22:41:19.444893+0000","last_became_peered":"2024-09-17T22:41:19.444893+0000","last_unstale":"2024-09-17T22:42:50.507555+0000","last_undegraded":"2024-09-17T22:42:50.507555+0000","last_fullsized":"2024-09-17T22:42:50.507555+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:54:56.685421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076767600000000001,"stat_sum":{"num_bytes":22014935,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":21,"num_read":1718,"num_read_kb":9540,"num_write":626,"num_write_kb":7900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1d","length":"c"}]},{"pgid":"2.4","version":"0'0","reported_seq":180,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T22:42:50.427557+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:42:50.427557+0000","last_peered":"2024-09-17T22:42:50.427557+0000","last_clean":"2024-09-17T22:42:50.427557+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:42:50.427557+0000","last_undegraded":"2024-09-17T22:42:50.427557+0000","last_fullsized":"2024-09-17T22:42:50.427557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"101'336","reported_seq":15301,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-17T22:42:49.472997+0000","last_change":"2024-09-17T22:42:49.423594+0000","last_active":"2024-09-17T22:42:49.472997+0000","last_peered":"2024-09-17T22:42:49.472997+0000","last_clean":"2024-09-17T22:42:49.472997+0000","last_became_active":"2024-09-17T22:41:19.451723+0000","last_became_peered":"2024-09-17T22:41:19.451723+0000","last_unstale":"2024-09-17T22:42:49.472997+0000","last_undegraded":"2024-09-17T22:42:49.472997+0000","last_fullsized":"2024-09-17T22:42:49.472997+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:28:12.680815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096017599999999998,"stat_sum":{"num_bytes":40926266,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":19,"num_read":15294,"num_read_kb":33603,"num_write":391,"num_write_kb":8744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1e","length":"b"}]},{"pgid":"3.3","version":"101'610","reported_seq":10959,"reported_epoch":101,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:49.506629+0000","last_change":"2024-09-17T22:41:54.119266+0000","last_active":"2024-09-17T22:42:49.506629+0000","last_peered":"2024-09-17T22:42:49.506629+0000","last_clean":"2024-09-17T22:41:52.262287+0000","last_became_active":"2024-09-17T22:41:54.119266+0000","last_became_peered":"2024-09-17T22:41:54.119266+0000","last_unstale":"2024-09-17T22:42:49.506629+0000","last_undegraded":"2024-09-17T22:41:54.116326+0000","last_fullsized":"2024-09-17T22:41:54.116242+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":11,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:06:21.166792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063898899999999996,"stat_sum":{"num_bytes":47214852,"num_objects":81,"num_object_clones":21,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":81,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":46,"num_read":14037,"num_read_kb":32562,"num_write":647,"num_write_kb":8593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":81}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":34,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T22:42:50.507324+0000","last_change":"2024-09-17T22:42:48.420232+0000","last_active":"2024-09-17T22:42:50.507324+0000","last_peered":"2024-09-17T22:42:50.507324+0000","last_clean":"2024-09-17T22:42:50.507324+0000","last_became_active":"2024-09-17T22:42:48.419946+0000","last_became_peered":"2024-09-17T22:42:48.419946+0000","last_unstale":"2024-09-17T22:42:50.507324+0000","last_undegraded":"2024-09-17T22:42:50.507324+0000","last_fullsized":"2024-09-17T22:42:50.507324+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:11:29.985673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"102'322","reported_seq":3813,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T22:42:50.433411+0000","last_change":"2024-09-17T22:42:50.433605+0000","last_active":"2024-09-17T22:42:50.433411+0000","last_peered":"2024-09-17T22:42:50.433411+0000","last_clean":"2024-09-17T22:42:50.433411+0000","last_became_active":"2024-09-17T22:41:19.444314+0000","last_became_peered":"2024-09-17T22:41:19.444314+0000","last_unstale":"2024-09-17T22:42:50.433411+0000","last_undegraded":"2024-09-17T22:42:50.433411+0000","last_fullsized":"2024-09-17T22:42:50.433411+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:41:30.237081+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0058478860000000001,"stat_sum":{"num_bytes":31286915,"num_objects":47,"num_object_clones":8,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":24,"num_read":3801,"num_read_kb":22182,"num_write":314,"num_write_kb":9342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1d","length":"c"}]},{"pgid":"2.1","version":"0'0","reported_seq":181,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T22:42:50.425264+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:42:50.425264+0000","last_peered":"2024-09-17T22:42:50.425264+0000","last_clean":"2024-09-17T22:42:50.425264+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:42:50.425264+0000","last_undegraded":"2024-09-17T22:42:50.425264+0000","last_fullsized":"2024-09-17T22:42:50.425264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"99'434","reported_seq":7164,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T22:42:50.507512+0000","last_change":"2024-09-17T22:42:50.507512+0000","last_active":"2024-09-17T22:42:50.507512+0000","last_peered":"2024-09-17T22:42:50.507512+0000","last_clean":"2024-09-17T22:42:50.507512+0000","last_became_active":"2024-09-17T22:41:19.449791+0000","last_became_peered":"2024-09-17T22:41:19.449791+0000","last_unstale":"2024-09-17T22:42:50.507512+0000","last_undegraded":"2024-09-17T22:42:50.507512+0000","last_fullsized":"2024-09-17T22:42:50.507512+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:18:51.103359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051382299999999995,"stat_sum":{"num_bytes":15838814,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":20,"num_read":8045,"num_read_kb":9215,"num_write":491,"num_write_kb":4601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1d","length":"c"}]},{"pgid":"2.0","version":"0'0","reported_seq":179,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-17T22:42:49.419757+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:42:49.419757+0000","last_peered":"2024-09-17T22:42:49.419757+0000","last_clean":"2024-09-17T22:42:49.419757+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:42:49.419757+0000","last_undegraded":"2024-09-17T22:42:49.419757+0000","last_fullsized":"2024-09-17T22:42:49.419757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"101'690","reported_seq":8320,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T22:42:50.423012+0000","last_change":"2024-09-17T22:42:50.423380+0000","last_active":"2024-09-17T22:42:50.423012+0000","last_peered":"2024-09-17T22:42:50.423012+0000","last_clean":"2024-09-17T22:42:50.423012+0000","last_became_active":"2024-09-17T22:41:19.449279+0000","last_became_peered":"2024-09-17T22:41:19.449279+0000","last_unstale":"2024-09-17T22:42:50.423012+0000","last_undegraded":"2024-09-17T22:42:50.423012+0000","last_fullsized":"2024-09-17T22:42:50.423012+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:13:39.382625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060825699999999996,"stat_sum":{"num_bytes":20790558,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":27,"num_read":10958,"num_read_kb":22667,"num_write":922,"num_write_kb":2854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"c"}]},{"pgid":"2.3","version":"19'1","reported_seq":19,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-17T22:42:49.422804+0000","last_change":"2024-09-17T22:42:48.420535+0000","last_active":"2024-09-17T22:42:49.422804+0000","last_peered":"2024-09-17T22:42:49.422804+0000","last_clean":"2024-09-17T22:42:49.422804+0000","last_became_active":"2024-09-17T22:42:48.420210+0000","last_became_peered":"2024-09-17T22:42:48.420210+0000","last_unstale":"2024-09-17T22:42:49.422804+0000","last_undegraded":"2024-09-17T22:42:49.422804+0000","last_fullsized":"2024-09-17T22:42:49.422804+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:36.249795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":334,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T22:42:51.437460+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:42:51.437460+0000","last_peered":"2024-09-17T22:42:51.437460+0000","last_clean":"2024-09-17T22:42:51.437460+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:42:51.437460+0000","last_undegraded":"2024-09-17T22:42:51.437460+0000","last_fullsized":"2024-09-17T22:42:51.437460+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"102'573","reported_seq":11669,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T22:42:50.497027+0000","last_change":"2024-09-17T22:42:50.424824+0000","last_active":"2024-09-17T22:42:50.497027+0000","last_peered":"2024-09-17T22:42:50.497027+0000","last_clean":"2024-09-17T22:42:50.497027+0000","last_became_active":"2024-09-17T22:41:19.448956+0000","last_became_peered":"2024-09-17T22:41:19.448956+0000","last_unstale":"2024-09-17T22:42:50.497027+0000","last_undegraded":"2024-09-17T22:42:50.497027+0000","last_fullsized":"2024-09-17T22:42:50.497027+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:51:01.606349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019664470000000001,"stat_sum":{"num_bytes":29743706,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":24,"num_read":13213,"num_read_kb":46884,"num_write":629,"num_write_kb":25170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1d","length":"c"}]},{"pgid":"3.9","version":"100'694","reported_seq":14340,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-17T22:42:49.420176+0000","last_change":"2024-09-17T22:42:49.420799+0000","last_active":"2024-09-17T22:42:49.420176+0000","last_peered":"2024-09-17T22:42:49.420176+0000","last_clean":"2024-09-17T22:42:49.420176+0000","last_became_active":"2024-09-17T22:41:19.449003+0000","last_became_peered":"2024-09-17T22:41:19.449003+0000","last_unstale":"2024-09-17T22:42:49.420176+0000","last_undegraded":"2024-09-17T22:42:49.420176+0000","last_fullsized":"2024-09-17T22:42:49.420176+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:06:33.185660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077437399999999998,"stat_sum":{"num_bytes":13847910,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":21,"num_read":15961,"num_read_kb":43029,"num_write":977,"num_write_kb":9827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"b"}]},{"pgid":"3.a","version":"103'408","reported_seq":8219,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T22:42:51.529599+0000","last_change":"2024-09-17T22:42:51.446588+0000","last_active":"2024-09-17T22:42:51.529599+0000","last_peered":"2024-09-17T22:42:51.529599+0000","last_clean":"2024-09-17T22:42:51.529599+0000","last_became_active":"2024-09-17T22:42:48.419427+0000","last_became_peered":"2024-09-17T22:42:48.419427+0000","last_unstale":"2024-09-17T22:42:51.529599+0000","last_undegraded":"2024-09-17T22:42:51.529599+0000","last_fullsized":"2024-09-17T22:42:51.529599+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:52:46.712779+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0089299029999999995,"stat_sum":{"num_bytes":41986812,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":26,"num_read":8152,"num_read_kb":41049,"num_write":434,"num_write_kb":9769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"d"}]},{"pgid":"3.b","version":"101'390","reported_seq":2465,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-17T22:42:50.425872+0000","last_change":"2024-09-17T22:42:50.426202+0000","last_active":"2024-09-17T22:42:50.425872+0000","last_peered":"2024-09-17T22:42:50.425872+0000","last_clean":"2024-09-17T22:42:50.425872+0000","last_became_active":"2024-09-17T22:41:19.447215+0000","last_became_peered":"2024-09-17T22:41:19.447215+0000","last_unstale":"2024-09-17T22:42:50.425872+0000","last_undegraded":"2024-09-17T22:42:50.425872+0000","last_fullsized":"2024-09-17T22:42:50.425872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:25:47.573596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064189,"stat_sum":{"num_bytes":24909824,"num_objects":47,"num_object_clones":9,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":23,"num_read":3403,"num_read_kb":14478,"num_write":389,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1d","length":"c"}]},{"pgid":"3.c","version":"101'296","reported_seq":21189,"reported_epoch":102,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:50.425432+0000","last_change":"2024-09-17T22:41:54.119555+0000","last_active":"2024-09-17T22:42:50.425432+0000","last_peered":"2024-09-17T22:42:50.425432+0000","last_clean":"2024-09-17T22:41:52.249181+0000","last_became_active":"2024-09-17T22:41:54.119555+0000","last_became_peered":"2024-09-17T22:41:54.119555+0000","last_unstale":"2024-09-17T22:42:50.425432+0000","last_undegraded":"2024-09-17T22:41:54.118897+0000","last_fullsized":"2024-09-17T22:41:54.118785+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:16:05.493979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000395712,"stat_sum":{"num_bytes":88562262,"num_objects":55,"num_object_clones":11,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":55,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":30,"num_read":21316,"num_read_kb":63555,"num_write":310,"num_write_kb":14934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":55}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"103'462","reported_seq":1816,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T22:42:51.525517+0000","last_change":"2024-09-17T22:42:51.445997+0000","last_active":"2024-09-17T22:42:51.525517+0000","last_peered":"2024-09-17T22:42:51.525517+0000","last_clean":"2024-09-17T22:42:51.525517+0000","last_became_active":"2024-09-17T22:42:48.419360+0000","last_became_peered":"2024-09-17T22:42:48.419360+0000","last_unstale":"2024-09-17T22:42:51.525517+0000","last_undegraded":"2024-09-17T22:42:51.525517+0000","last_fullsized":"2024-09-17T22:42:51.525517+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:57.800696+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0083810259999999998,"stat_sum":{"num_bytes":19637323,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":21,"num_read":2214,"num_read_kb":10203,"num_write":564,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"d"}]},{"pgid":"3.e","version":"97'291","reported_seq":8184,"reported_epoch":101,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:49.422747+0000","last_change":"2024-09-17T22:41:54.119016+0000","last_active":"2024-09-17T22:42:49.422747+0000","last_peered":"2024-09-17T22:42:49.422747+0000","last_clean":"2024-09-17T22:41:52.398679+0000","last_became_active":"2024-09-17T22:41:54.119016+0000","last_became_peered":"2024-09-17T22:41:54.119016+0000","last_unstale":"2024-09-17T22:42:49.422747+0000","last_undegraded":"2024-09-17T22:41:54.116346+0000","last_fullsized":"2024-09-17T22:41:54.116297+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":11,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:41:30.063769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038538199999999998,"stat_sum":{"num_bytes":23415453,"num_objects":55,"num_object_clones":15,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":55,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":31,"num_read":8353,"num_read_kb":22721,"num_write":250,"num_write_kb":11321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":55}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"102'313","reported_seq":1466,"reported_epoch":103,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:51.437748+0000","last_change":"2024-09-17T22:41:54.113116+0000","last_active":"2024-09-17T22:42:51.437748+0000","last_peered":"2024-09-17T22:42:51.437748+0000","last_clean":"2024-09-17T22:41:52.346319+0000","last_became_active":"2024-09-17T22:41:54.113116+0000","last_became_peered":"2024-09-17T22:41:54.113116+0000","last_unstale":"2024-09-17T22:42:51.437748+0000","last_undegraded":"2024-09-17T22:41:54.112692+0000","last_fullsized":"2024-09-17T22:41:54.112614+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":13,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:47:28.109419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000602379,"stat_sum":{"num_bytes":22757743,"num_objects":60,"num_object_clones":15,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":60,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":35,"num_read":1623,"num_read_kb":5496,"num_write":370,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":60}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"99'260","reported_seq":1638,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-17T22:42:51.437662+0000","last_change":"2024-09-17T22:42:51.438134+0000","last_active":"2024-09-17T22:42:51.437662+0000","last_peered":"2024-09-17T22:42:51.437662+0000","last_clean":"2024-09-17T22:42:51.437662+0000","last_became_active":"2024-09-17T22:41:19.447419+0000","last_became_peered":"2024-09-17T22:41:19.447419+0000","last_unstale":"2024-09-17T22:42:51.437662+0000","last_undegraded":"2024-09-17T22:42:51.437662+0000","last_fullsized":"2024-09-17T22:42:51.437662+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:14:47.299107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062809599999999997,"stat_sum":{"num_bytes":11086817,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":25,"num_read":2598,"num_read_kb":8045,"num_write":227,"num_write_kb":3143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"d"}]},{"pgid":"3.11","version":"101'511","reported_seq":3153,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-17T22:42:49.528643+0000","last_change":"2024-09-17T22:42:49.425327+0000","last_active":"2024-09-17T22:42:49.528643+0000","last_peered":"2024-09-17T22:42:49.528643+0000","last_clean":"2024-09-17T22:42:49.528643+0000","last_became_active":"2024-09-17T22:41:19.454151+0000","last_became_peered":"2024-09-17T22:41:19.454151+0000","last_unstale":"2024-09-17T22:42:49.528643+0000","last_undegraded":"2024-09-17T22:42:49.528643+0000","last_fullsized":"2024-09-17T22:42:49.528643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:42:15.029420+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0054752810000000002,"stat_sum":{"num_bytes":16007633,"num_objects":55,"num_object_clones":15,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":29,"num_read":4758,"num_read_kb":13115,"num_write":590,"num_write_kb":3710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"b"}]},{"pgid":"3.12","version":"102'823","reported_seq":11012,"reported_epoch":102,"state":"active+clean+remapped","last_fresh":"2024-09-17T22:42:50.539166+0000","last_change":"2024-09-17T22:42:50.508122+0000","last_active":"2024-09-17T22:42:50.539166+0000","last_peered":"2024-09-17T22:42:50.539166+0000","last_clean":"2024-09-17T22:42:50.539166+0000","last_became_active":"2024-09-17T22:41:57.131666+0000","last_became_peered":"2024-09-17T22:41:57.131666+0000","last_unstale":"2024-09-17T22:42:50.539166+0000","last_undegraded":"2024-09-17T22:42:50.539166+0000","last_fullsized":"2024-09-17T22:42:50.539166+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:36:52.138152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010618279999999999,"stat_sum":{"num_bytes":46686463,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":40,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":20,"num_read":14170,"num_read_kb":43663,"num_write":1088,"num_write_kb":22148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6,3],"avail_no_missing":["6","3"],"object_location_counts":[{"shards":"3,6","objects":40}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1d","length":"c"}]},{"pgid":"3.13","version":"95'451","reported_seq":2176,"reported_epoch":101,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:42:49.420313+0000","last_change":"2024-09-17T22:41:54.118972+0000","last_active":"2024-09-17T22:42:49.420313+0000","last_peered":"2024-09-17T22:42:49.420313+0000","last_clean":"2024-09-17T22:41:52.267782+0000","last_became_active":"2024-09-17T22:41:54.118972+0000","last_became_peered":"2024-09-17T22:41:54.118972+0000","last_unstale":"2024-09-17T22:42:49.420313+0000","last_undegraded":"2024-09-17T22:41:54.114427+0000","last_fullsized":"2024-09-17T22:41:54.114366+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":11,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:41:58.937727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062146200000000003,"stat_sum":{"num_bytes":16443068,"num_objects":52,"num_object_clones":10,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":33,"num_read":2436,"num_read_kb":8750,"num_write":582,"num_write_kb":8374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":52}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1056910726,"num_objects":1578,"num_object_clones":356,"num_object_copies":3156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":400,"num_objects_misplaced":132,"num_objects_unfound":0,"num_objects_dirty":1578,"num_whiteouts":832,"num_read":261406,"num_read_kb":796508,"num_write":16874,"num_write_kb":293383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":10513915,"num_keys_recovered":74,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28463104,"data_stored":230993451,"data_compressed":7595848,"data_compressed_allocated":20299776,"data_compressed_original":226144123,"omap_allocated":0,"internal_metadata":0},"log_size":14541,"ondisk_log_size":14541,"up":54,"acting":57,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":14,"acting":14,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542170,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43580,"kb_used_data":4068,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328260,"statfs":{"total":96636764160,"available":96592138240,"internally_reserved":0,"allocated":4165632,"data_stored":22651919,"data_compressed":686696,"data_compressed_allocated":2023424,"data_compressed_original":21522217,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542156,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34156,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337684,"statfs":{"total":96636764160,"available":96601788416,"internally_reserved":0,"allocated":3043328,"data_stored":23926169,"data_compressed":437861,"data_compressed_allocated":1896448,"data_compressed_original":23315877,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542168,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44216,"kb_used_data":4708,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327624,"statfs":{"total":96636764160,"available":96591486976,"internally_reserved":0,"allocated":4820992,"data_stored":35296088,"data_compressed":986358,"data_compressed_allocated":2801664,"data_compressed_original":34227119,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542168,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44224,"kb_used_data":4708,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327616,"statfs":{"total":96636764160,"available":96591478784,"internally_reserved":0,"allocated":4820992,"data_stored":27329560,"data_compressed":1096891,"data_compressed_allocated":2523136,"data_compressed_original":26099119,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":4932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":5050368,"data_stored":32417258,"data_compressed":1085098,"data_compressed_allocated":2826240,"data_compressed_original":31274525,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43640,"kb_used_data":4132,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328200,"statfs":{"total":96636764160,"available":96592076800,"internally_reserved":0,"allocated":4231168,"data_stored":23242545,"data_compressed":886970,"data_compressed_allocated":1982464,"data_compressed_original":22030310,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55436,"kb_used_data":6192,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94316404,"statfs":{"total":96636764160,"available":96579997696,"internally_reserved":0,"allocated":6340608,"data_stored":42502307,"data_compressed":1464944,"data_compressed_allocated":3698688,"data_compressed_original":41099108,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54404,"kb_used_data":5164,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317436,"statfs":{"total":96636764160,"available":96581054464,"internally_reserved":0,"allocated":5287936,"data_stored":31705104,"data_compressed":1065471,"data_compressed_allocated":2793472,"data_compressed_original":30393320,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2514944,"data_stored":23924031,"data_compressed":438892,"data_compressed_allocated":1921024,"data_compressed_original":23549349,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2785280,"data_stored":19692703,"data_compressed":612524,"data_compressed_allocated":1884160,"data_compressed_original":19158825,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4091904,"data_stored":31140792,"data_compressed":1065471,"data_compressed_allocated":2793472,"data_compressed_original":30393320,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5140480,"data_stored":41937976,"data_compressed":1464944,"data_compressed_allocated":3698688,"data_compressed_original":41099108,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3022848,"data_stored":22671803,"data_compressed":886970,"data_compressed_allocated":1982464,"data_compressed_original":22030310,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3686400,"data_stored":30142001,"data_compressed":1043798,"data_compressed_allocated":2695168,"data_compressed_original":29586973,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3608576,"data_stored":26758799,"data_compressed":1096891,"data_compressed_allocated":2523136,"data_compressed_original":26099119,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3612672,"data_stored":34725346,"data_compressed":986358,"data_compressed_allocated":2801664,"data_compressed_original":34227119,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:42:53.245 INFO:tasks.thrashosds.thrasher:Setting 3.1d to [1, 6] 2024-09-17T22:42:53.246 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.1d', '1', '6'] 2024-09-17T22:42:53.246 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.1d 1 6 2024-09-17T22:42:53.264 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:53.262+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:53.325 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:53.325 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:53.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:53.325 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:53.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.325 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157229 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4073472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.325 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:53.325 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.175790800000001, 2024-09-17T22:42:53.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041957959000000003, 2024-09-17T22:42:53.326 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.326 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.326 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.326 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.326 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.326 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.326 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157229 2024-09-17T22:42:53.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.326 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:53.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:42:53.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146207+0000", 2024-09-17T22:42:53.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0299999999999999e-07 2024-09-17T22:42:53.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146215+0000", 2024-09-17T22:42:53.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3059999999999998e-06 2024-09-17T22:42:53.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146222+0000", 2024-09-17T22:42:53.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1180000000000003e-06 2024-09-17T22:42:53.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146227+0000", 2024-09-17T22:42:53.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1749999999999998e-06 2024-09-17T22:42:53.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.187985+0000", 2024-09-17T22:42:53.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041758391999999998 2024-09-17T22:42:53.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188065+0000", 2024-09-17T22:42:53.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0290000000000005e-05 2024-09-17T22:42:53.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188164+0000", 2024-09-17T22:42:53.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9074999999999996e-05 2024-09-17T22:42:53.330 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.330 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.330 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.330 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157231 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4079616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.330 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:53.330 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.175653175999997, 2024-09-17T22:42:53.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042034581000000001, 2024-09-17T22:42:53.330 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.330 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.330 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.331 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.331 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.331 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.331 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157231 2024-09-17T22:42:53.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.331 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:53.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:42:53.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146345+0000", 2024-09-17T22:42:53.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:42:53.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146347+0000", 2024-09-17T22:42:53.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.357e-06 2024-09-17T22:42:53.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146352+0000", 2024-09-17T22:42:53.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T22:42:53.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146361+0000", 2024-09-17T22:42:53.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0219999999999993e-06 2024-09-17T22:42:53.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188199+0000", 2024-09-17T22:42:53.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041838130000000001 2024-09-17T22:42:53.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188285+0000", 2024-09-17T22:42:53.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5160000000000005e-05 2024-09-17T22:42:53.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188379+0000", 2024-09-17T22:42:53.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3949000000000005e-05 2024-09-17T22:42:53.334 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.334 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.335 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.335 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157233 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4085760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.335 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:53.335 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.175437027000001, 2024-09-17T22:42:53.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042008960999999997, 2024-09-17T22:42:53.335 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.335 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.335 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.335 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.335 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.335 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.336 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157233 2024-09-17T22:42:53.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.336 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:53.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:42:53.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146561+0000", 2024-09-17T22:42:53.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6300000000000005e-07 2024-09-17T22:42:53.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.337 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.337 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146564+0000", 2024-09-17T22:42:53.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-17T22:42:53.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146575+0000", 2024-09-17T22:42:53.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0891e-05 2024-09-17T22:42:53.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146579+0000", 2024-09-17T22:42:53.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.284e-06 2024-09-17T22:42:53.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188407+0000", 2024-09-17T22:42:53.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041828390999999999 2024-09-17T22:42:53.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188476+0000", 2024-09-17T22:42:53.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9129999999999997e-05 2024-09-17T22:42:53.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188569+0000", 2024-09-17T22:42:53.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2669000000000003e-05 2024-09-17T22:42:53.339 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.339 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.339 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.340 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157235 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4091904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.340 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:53.340 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.175243029999997, 2024-09-17T22:42:53.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042047517, 2024-09-17T22:42:53.340 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.340 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.340 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.340 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.340 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.340 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.341 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157235 2024-09-17T22:42:53.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.341 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:53.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.341 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.341 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:42:53.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146755+0000", 2024-09-17T22:42:53.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0900000000000001e-07 2024-09-17T22:42:53.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146758+0000", 2024-09-17T22:42:53.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.399e-06 2024-09-17T22:42:53.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146768+0000", 2024-09-17T22:42:53.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4730000000000006e-06 2024-09-17T22:42:53.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146774+0000", 2024-09-17T22:42:53.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6939999999999998e-06 2024-09-17T22:42:53.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188616+0000", 2024-09-17T22:42:53.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041841481999999999 2024-09-17T22:42:53.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188700+0000", 2024-09-17T22:42:53.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4227999999999994e-05 2024-09-17T22:42:53.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188802+0000", 2024-09-17T22:42:53.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101532 2024-09-17T22:42:53.344 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.344 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.344 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.345 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157237 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4098048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.345 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:53.345 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.175212758999997, 2024-09-17T22:42:53.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165452999999999, 2024-09-17T22:42:53.345 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.345 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.345 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.345 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.345 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.345 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.345 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157237 2024-09-17T22:42:53.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.346 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:53.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:42:53.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146785+0000", 2024-09-17T22:42:53.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6800000000000001e-07 2024-09-17T22:42:53.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146787+0000", 2024-09-17T22:42:53.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.046e-06 2024-09-17T22:42:53.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146794+0000", 2024-09-17T22:42:53.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9249999999999998e-06 2024-09-17T22:42:53.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146799+0000", 2024-09-17T22:42:53.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9289999999999997e-06 2024-09-17T22:42:53.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188844+0000", 2024-09-17T22:42:53.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042045386999999997 2024-09-17T22:42:53.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188888+0000", 2024-09-17T22:42:53.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3760000000000001e-05 2024-09-17T22:42:53.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188950+0000", 2024-09-17T22:42:53.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1938000000000002e-05 2024-09-17T22:42:53.349 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.349 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.349 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.350 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157239 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4104192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.350 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:53.350 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.175184373, 2024-09-17T22:42:53.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042347193999999998, 2024-09-17T22:42:53.350 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.350 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.350 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.350 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157239 2024-09-17T22:42:53.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.350 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:53.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:53.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:42:53.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:42:53.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146816+0000", 2024-09-17T22:42:53.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4700000000000001e-06 2024-09-17T22:42:53.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146822+0000", 2024-09-17T22:42:53.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.579e-06 2024-09-17T22:42:53.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146826+0000", 2024-09-17T22:42:53.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3780000000000001e-06 2024-09-17T22:42:53.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188991+0000", 2024-09-17T22:42:53.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165553000000001 2024-09-17T22:42:53.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189068+0000", 2024-09-17T22:42:53.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6548e-05 2024-09-17T22:42:53.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189160+0000", 2024-09-17T22:42:53.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2163000000000005e-05 2024-09-17T22:42:53.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.354 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.354 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157242 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4110336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.354 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:53.354 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.175146587, 2024-09-17T22:42:53.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042499783999999999, 2024-09-17T22:42:53.354 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.355 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.355 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.355 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157242 2024-09-17T22:42:53.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.355 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:53.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:42:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:42:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146852+0000", 2024-09-17T22:42:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.37e-06 2024-09-17T22:42:53.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146856+0000", 2024-09-17T22:42:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1690000000000002e-06 2024-09-17T22:42:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146863+0000", 2024-09-17T22:42:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9920000000000001e-06 2024-09-17T22:42:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189193+0000", 2024-09-17T22:42:53.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042329355999999999 2024-09-17T22:42:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189246+0000", 2024-09-17T22:42:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3643000000000001e-05 2024-09-17T22:42:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189350+0000", 2024-09-17T22:42:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010394 2024-09-17T22:42:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157244 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4116480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.175127375999999, 2024-09-17T22:42:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042675684999999998, 2024-09-17T22:42:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157244 2024-09-17T22:42:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.359 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:42:53.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1e-07 2024-09-17T22:42:53.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146871+0000", 2024-09-17T22:42:53.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.189e-06 2024-09-17T22:42:53.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146877+0000", 2024-09-17T22:42:53.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8100000000000003e-06 2024-09-17T22:42:53.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146880+0000", 2024-09-17T22:42:53.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.83e-06 2024-09-17T22:42:53.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189388+0000", 2024-09-17T22:42:53.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042507637000000001 2024-09-17T22:42:53.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189443+0000", 2024-09-17T22:42:53.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5742999999999998e-05 2024-09-17T22:42:53.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189545+0000", 2024-09-17T22:42:53.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102166 2024-09-17T22:42:53.363 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.363 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.363 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.363 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157246 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4122624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.363 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:53.363 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.175098323999997, 2024-09-17T22:42:53.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042847083000000001, 2024-09-17T22:42:53.364 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.364 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.364 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.364 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.364 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.364 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.364 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157246 2024-09-17T22:42:53.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.364 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:53.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:53.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:42:53.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0100000000000001e-07 2024-09-17T22:42:53.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146901+0000", 2024-09-17T22:42:53.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.531e-06 2024-09-17T22:42:53.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146905+0000", 2024-09-17T22:42:53.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0759999999999996e-06 2024-09-17T22:42:53.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146908+0000", 2024-09-17T22:42:53.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.083e-06 2024-09-17T22:42:53.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.366 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.366 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189585+0000", 2024-09-17T22:42:53.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042677009000000002 2024-09-17T22:42:53.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189656+0000", 2024-09-17T22:42:53.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0712999999999999e-05 2024-09-17T22:42:53.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189746+0000", 2024-09-17T22:42:53.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0370000000000004e-05 2024-09-17T22:42:53.367 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.367 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.367 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.368 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157247 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4128768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.368 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:53.368 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.175026488, 2024-09-17T22:42:53.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043016916000000002, 2024-09-17T22:42:53.368 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.368 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.368 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.368 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.368 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.368 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.368 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157247 2024-09-17T22:42:53.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.369 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:53.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:42:53.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147071+0000", 2024-09-17T22:42:53.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100769 2024-09-17T22:42:53.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147081+0000", 2024-09-17T22:42:53.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9229999999999997e-06 2024-09-17T22:42:53.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147090+0000", 2024-09-17T22:42:53.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.935e-06 2024-09-17T22:42:53.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147097+0000", 2024-09-17T22:42:53.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T22:42:53.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189787+0000", 2024-09-17T22:42:53.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042690525 2024-09-17T22:42:53.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189884+0000", 2024-09-17T22:42:53.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6858e-05 2024-09-17T22:42:53.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.372 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.372 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189988+0000", 2024-09-17T22:42:53.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103336 2024-09-17T22:42:53.372 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.372 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.372 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.372 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.372 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157250 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4134912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.373 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:53.373 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.174883027, 2024-09-17T22:42:53.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043099446, 2024-09-17T22:42:53.373 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.373 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.373 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.373 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.373 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.373 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.373 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157250 2024-09-17T22:42:53.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.373 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:53.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:42:53.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147235+0000", 2024-09-17T22:42:53.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120975 2024-09-17T22:42:53.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147238+0000", 2024-09-17T22:42:53.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5289999999999998e-06 2024-09-17T22:42:53.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147244+0000", 2024-09-17T22:42:53.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6050000000000003e-06 2024-09-17T22:42:53.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147249+0000", 2024-09-17T22:42:53.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4939999999999997e-06 2024-09-17T22:42:53.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190020+0000", 2024-09-17T22:42:53.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042771300999999998 2024-09-17T22:42:53.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190102+0000", 2024-09-17T22:42:53.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1600000000000005e-05 2024-09-17T22:42:53.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.377 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.377 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190214+0000", 2024-09-17T22:42:53.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011194200000000001 2024-09-17T22:42:53.377 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.377 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.377 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.377 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157252 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4141056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.377 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:53.377 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.174740358000001, 2024-09-17T22:42:53.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043165508999999998, 2024-09-17T22:42:53.378 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.378 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.378 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.378 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157252 2024-09-17T22:42:53.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.378 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:53.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:42:53.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147402+0000", 2024-09-17T22:42:53.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014475600000000001 2024-09-17T22:42:53.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147411+0000", 2024-09-17T22:42:53.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4350000000000003e-06 2024-09-17T22:42:53.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147420+0000", 2024-09-17T22:42:53.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4790000000000002e-06 2024-09-17T22:42:53.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147427+0000", 2024-09-17T22:42:53.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5400000000000001e-06 2024-09-17T22:42:53.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190254+0000", 2024-09-17T22:42:53.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042826731 2024-09-17T22:42:53.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190326+0000", 2024-09-17T22:42:53.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1798999999999995e-05 2024-09-17T22:42:53.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190422+0000", 2024-09-17T22:42:53.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6768999999999995e-05 2024-09-17T22:42:53.382 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.382 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.382 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.382 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157253 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4147200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.382 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:53.382 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.174553992, 2024-09-17T22:42:53.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043157752000000001, 2024-09-17T22:42:53.382 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.382 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.382 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.382 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.383 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157253 2024-09-17T22:42:53.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.383 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:53.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.383 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.383 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:42:53.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147590+0000", 2024-09-17T22:42:53.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014709 2024-09-17T22:42:53.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147593+0000", 2024-09-17T22:42:53.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5840000000000002e-06 2024-09-17T22:42:53.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147598+0000", 2024-09-17T22:42:53.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2469999999999999e-06 2024-09-17T22:42:53.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147602+0000", 2024-09-17T22:42:53.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0049999999999998e-06 2024-09-17T22:42:53.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190475+0000", 2024-09-17T22:42:53.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872767999999999 2024-09-17T22:42:53.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190528+0000", 2024-09-17T22:42:53.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2889999999999997e-05 2024-09-17T22:42:53.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190601+0000", 2024-09-17T22:42:53.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3168000000000001e-05 2024-09-17T22:42:53.386 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.386 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.386 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.387 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157255 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4153344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.387 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:53.387 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.174388010999998, 2024-09-17T22:42:53.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043132251000000003, 2024-09-17T22:42:53.387 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.387 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.387 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.387 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.387 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.387 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.387 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157255 2024-09-17T22:42:53.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.387 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:53.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:42:53.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.388 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.388 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147820+0000", 2024-09-17T22:42:53.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021123600000000001 2024-09-17T22:42:53.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147824+0000", 2024-09-17T22:42:53.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6780000000000002e-06 2024-09-17T22:42:53.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147833+0000", 2024-09-17T22:42:53.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4250000000000004e-06 2024-09-17T22:42:53.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147842+0000", 2024-09-17T22:42:53.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0679999999999997e-06 2024-09-17T22:42:53.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190632+0000", 2024-09-17T22:42:53.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042790863999999998 2024-09-17T22:42:53.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190674+0000", 2024-09-17T22:42:53.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1804000000000002e-05 2024-09-17T22:42:53.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190741+0000", 2024-09-17T22:42:53.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7175999999999995e-05 2024-09-17T22:42:53.391 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.391 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.391 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.391 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157257 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4159488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.391 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:53.391 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.174147521000002, 2024-09-17T22:42:53.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.04307035, 2024-09-17T22:42:53.391 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.391 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.391 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.392 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.392 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.392 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.392 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157257 2024-09-17T22:42:53.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.392 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:53.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:42:53.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148844+0000", 2024-09-17T22:42:53.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00099455599999999996 2024-09-17T22:42:53.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148848+0000", 2024-09-17T22:42:53.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0180000000000003e-06 2024-09-17T22:42:53.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148858+0000", 2024-09-17T22:42:53.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.825e-06 2024-09-17T22:42:53.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148865+0000", 2024-09-17T22:42:53.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8290000000000002e-06 2024-09-17T22:42:53.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.394 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.394 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.394 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190789+0000", 2024-09-17T22:42:53.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041924630999999997 2024-09-17T22:42:53.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190860+0000", 2024-09-17T22:42:53.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0557999999999998e-05 2024-09-17T22:42:53.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190920+0000", 2024-09-17T22:42:53.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9932999999999997e-05 2024-09-17T22:42:53.395 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.395 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.396 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.396 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157259 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4165632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.396 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:53.396 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.173121502000001, 2024-09-17T22:42:53.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042261411999999998, 2024-09-17T22:42:53.396 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.396 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.396 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.396 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.396 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.396 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.397 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157259 2024-09-17T22:42:53.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.397 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:53.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:53.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:42:53.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:42:53.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148878+0000", 2024-09-17T22:42:53.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:42:53.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148885+0000", 2024-09-17T22:42:53.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7290000000000003e-06 2024-09-17T22:42:53.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148893+0000", 2024-09-17T22:42:53.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8110000000000003e-06 2024-09-17T22:42:53.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.399 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190967+0000", 2024-09-17T22:42:53.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042073948999999999 2024-09-17T22:42:53.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.399 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191069+0000", 2024-09-17T22:42:53.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102075 2024-09-17T22:42:53.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191137+0000", 2024-09-17T22:42:53.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8281000000000006e-05 2024-09-17T22:42:53.400 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.400 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.400 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.400 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157261 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4171776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.401 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:53.401 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.173071895, 2024-09-17T22:42:53.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042437255, 2024-09-17T22:42:53.401 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.401 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.401 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.401 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.401 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157261 2024-09-17T22:42:53.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.401 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:53.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:42:53.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148926+0000", 2024-09-17T22:42:53.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T22:42:53.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148928+0000", 2024-09-17T22:42:53.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4329999999999998e-06 2024-09-17T22:42:53.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148936+0000", 2024-09-17T22:42:53.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6969999999999993e-06 2024-09-17T22:42:53.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148942+0000", 2024-09-17T22:42:53.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9479999999999996e-06 2024-09-17T22:42:53.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191185+0000", 2024-09-17T22:42:53.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042243147000000002 2024-09-17T22:42:53.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191266+0000", 2024-09-17T22:42:53.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1186000000000005e-05 2024-09-17T22:42:53.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.405 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191362+0000", 2024-09-17T22:42:53.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6384000000000004e-05 2024-09-17T22:42:53.405 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.405 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.405 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.405 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157263 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4177920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.405 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:53.405 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.173047496000002, 2024-09-17T22:42:53.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042636395000000001, 2024-09-17T22:42:53.405 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.405 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.406 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.406 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.406 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157263 2024-09-17T22:42:53.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.406 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:53.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:53.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:42:53.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.37e-07 2024-09-17T22:42:53.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148952+0000", 2024-09-17T22:42:53.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.88e-06 2024-09-17T22:42:53.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148959+0000", 2024-09-17T22:42:53.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5649999999999998e-06 2024-09-17T22:42:53.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148963+0000", 2024-09-17T22:42:53.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8570000000000003e-06 2024-09-17T22:42:53.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191408+0000", 2024-09-17T22:42:53.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042444707999999998 2024-09-17T22:42:53.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191489+0000", 2024-09-17T22:42:53.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0612999999999995e-05 2024-09-17T22:42:53.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191586+0000", 2024-09-17T22:42:53.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7235e-05 2024-09-17T22:42:53.409 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.409 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.410 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.410 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157265 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4184064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.410 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:53.410 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.173025946999999, 2024-09-17T22:42:53.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042791830000000003, 2024-09-17T22:42:53.410 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.410 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.410 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.410 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.410 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.410 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.410 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157265 2024-09-17T22:42:53.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.411 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:53.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:42:53.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148972+0000", 2024-09-17T22:42:53.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:42:53.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148973+0000", 2024-09-17T22:42:53.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.894e-06 2024-09-17T22:42:53.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148985+0000", 2024-09-17T22:42:53.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1995e-05 2024-09-17T22:42:53.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148990+0000", 2024-09-17T22:42:53.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8269999999999997e-06 2024-09-17T22:42:53.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191630+0000", 2024-09-17T22:42:53.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042639639 2024-09-17T22:42:53.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191686+0000", 2024-09-17T22:42:53.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6066000000000002e-05 2024-09-17T22:42:53.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191763+0000", 2024-09-17T22:42:53.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7035999999999999e-05 2024-09-17T22:42:53.414 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.415 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.415 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.415 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157267 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4190208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:53.415 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:53.415 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.173000133999999, 2024-09-17T22:42:53.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043028813999999999, 2024-09-17T22:42:53.415 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.415 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.415 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.415 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.415 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.416 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.416 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157267 2024-09-17T22:42:53.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.416 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:53.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.416 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:53.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:42:53.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:42:53.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148999+0000", 2024-09-17T22:42:53.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.432e-06 2024-09-17T22:42:53.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149002+0000", 2024-09-17T22:42:53.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7850000000000002e-06 2024-09-17T22:42:53.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149005+0000", 2024-09-17T22:42:53.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:42:53.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191878+0000", 2024-09-17T22:42:53.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872312000000003 2024-09-17T22:42:53.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191961+0000", 2024-09-17T22:42:53.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3418000000000007e-05 2024-09-17T22:42:53.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.192026+0000", 2024-09-17T22:42:53.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4689000000000004e-05 2024-09-17T22:42:53.419 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.419 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.419 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.419 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.419 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.420 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:53.421 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:42:53.421 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:53.419+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:53.522 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:53.519+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:53.571 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:53.571 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:53.571 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:53.571 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:53.580 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:42:53.622 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:53.621+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:53.723 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:53.723 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:53.723 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:53.723 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:53.724 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:53.724 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:53.721+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:53.732 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:42:53.824 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:53.822+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:53.877 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:53.877 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:53.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:53.877 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:53.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.877 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188693 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5881856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:53.878 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:53.878 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.046916549000002, 2024-09-17T22:42:53.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028193223, 2024-09-17T22:42:53.878 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.878 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.878 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.878 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.878 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.878 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.878 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188693 2024-09-17T22:42:53.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.878 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:53.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:53.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:42:53.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:42:53.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827090+0000", 2024-09-17T22:42:53.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2170000000000001e-06 2024-09-17T22:42:53.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827098+0000", 2024-09-17T22:42:53.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1510000000000004e-06 2024-09-17T22:42:53.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827103+0000", 2024-09-17T22:42:53.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0839999999999999e-06 2024-09-17T22:42:53.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855158+0000", 2024-09-17T22:42:53.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028055217 2024-09-17T22:42:53.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855201+0000", 2024-09-17T22:42:53.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3272000000000002e-05 2024-09-17T22:42:53.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855280+0000", 2024-09-17T22:42:53.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8845999999999997e-05 2024-09-17T22:42:53.882 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.882 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.882 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.882 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188694 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5971968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:53.882 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:53.882 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.046892661999998, 2024-09-17T22:42:53.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028312278, 2024-09-17T22:42:53.882 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.882 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.882 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.883 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.883 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.883 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.883 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188694 2024-09-17T22:42:53.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.883 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:53.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:53.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:42:53.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:42:53.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827114+0000", 2024-09-17T22:42:53.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3029999999999998e-06 2024-09-17T22:42:53.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827121+0000", 2024-09-17T22:42:53.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3699999999999997e-06 2024-09-17T22:42:53.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827126+0000", 2024-09-17T22:42:53.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7350000000000004e-06 2024-09-17T22:42:53.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855307+0000", 2024-09-17T22:42:53.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028180851999999999 2024-09-17T22:42:53.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855346+0000", 2024-09-17T22:42:53.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9274999999999999e-05 2024-09-17T22:42:53.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855423+0000", 2024-09-17T22:42:53.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7294999999999998e-05 2024-09-17T22:42:53.886 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.886 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.886 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.887 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188695 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:53.887 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:53.887 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.046871883999998, 2024-09-17T22:42:53.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028435876999999998, 2024-09-17T22:42:53.887 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.887 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.887 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.887 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.887 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.887 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.887 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188695 2024-09-17T22:42:53.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.887 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:53.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:53.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.888 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.888 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:42:53.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:42:53.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827138+0000", 2024-09-17T22:42:53.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2580000000000001e-06 2024-09-17T22:42:53.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827145+0000", 2024-09-17T22:42:53.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0129999999999996e-06 2024-09-17T22:42:53.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827150+0000", 2024-09-17T22:42:53.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3749999999999996e-06 2024-09-17T22:42:53.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855447+0000", 2024-09-17T22:42:53.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028297224999999999 2024-09-17T22:42:53.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855487+0000", 2024-09-17T22:42:53.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0454000000000003e-05 2024-09-17T22:42:53.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855567+0000", 2024-09-17T22:42:53.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0149999999999997e-05 2024-09-17T22:42:53.891 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.891 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.891 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.891 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188696 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:53.891 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:53.891 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.046848099999998, 2024-09-17T22:42:53.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028543570000000001, 2024-09-17T22:42:53.891 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.891 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.892 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.892 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.892 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188696 2024-09-17T22:42:53.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.892 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:53.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:42:53.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827156+0000", 2024-09-17T22:42:53.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T22:42:53.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827158+0000", 2024-09-17T22:42:53.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.875e-06 2024-09-17T22:42:53.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827164+0000", 2024-09-17T22:42:53.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7950000000000002e-06 2024-09-17T22:42:53.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827169+0000", 2024-09-17T22:42:53.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2880000000000003e-06 2024-09-17T22:42:53.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855582+0000", 2024-09-17T22:42:53.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028412929 2024-09-17T22:42:53.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855624+0000", 2024-09-17T22:42:53.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2484000000000002e-05 2024-09-17T22:42:53.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855699+0000", 2024-09-17T22:42:53.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4869000000000004e-05 2024-09-17T22:42:53.895 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.895 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.896 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.896 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188697 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6242304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:53.896 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:53.896 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.046816571000001, 2024-09-17T22:42:53.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028658598, 2024-09-17T22:42:53.896 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.896 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.896 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.896 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.896 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188697 2024-09-17T22:42:53.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.897 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:53.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:53.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:42:53.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8800000000000003e-07 2024-09-17T22:42:53.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827190+0000", 2024-09-17T22:42:53.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.351e-06 2024-09-17T22:42:53.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827198+0000", 2024-09-17T22:42:53.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.909e-06 2024-09-17T22:42:53.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827203+0000", 2024-09-17T22:42:53.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4020000000000001e-06 2024-09-17T22:42:53.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855720+0000", 2024-09-17T22:42:53.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028516909999999999 2024-09-17T22:42:53.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855762+0000", 2024-09-17T22:42:53.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2184999999999997e-05 2024-09-17T22:42:53.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855845+0000", 2024-09-17T22:42:53.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3353e-05 2024-09-17T22:42:53.900 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.900 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.900 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.900 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202883 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2342912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:53.900 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:42:53.900 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.149180190000003, 2024-09-17T22:42:53.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028098636, 2024-09-17T22:42:53.900 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.900 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.901 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.901 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.901 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202883 2024-09-17T22:42:53.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.901 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:42:53.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:42:53.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724824+0000", 2024-09-17T22:42:53.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:42:53.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724826+0000", 2024-09-17T22:42:53.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.086e-06 2024-09-17T22:42:53.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724832+0000", 2024-09-17T22:42:53.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9100000000000002e-06 2024-09-17T22:42:53.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724837+0000", 2024-09-17T22:42:53.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9439999999999996e-06 2024-09-17T22:42:53.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752790+0000", 2024-09-17T22:42:53.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027953657999999999 2024-09-17T22:42:53.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752848+0000", 2024-09-17T22:42:53.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7677999999999999e-05 2024-09-17T22:42:53.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.904 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.904 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752922+0000", 2024-09-17T22:42:53.904 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3957999999999999e-05 2024-09-17T22:42:53.904 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.904 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.904 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.904 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.904 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202884 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2473984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:53.905 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:42:53.905 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.149151965000001, 2024-09-17T22:42:53.905 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028206208999999999, 2024-09-17T22:42:53.905 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.905 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.905 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.905 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.905 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202884 2024-09-17T22:42:53.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.905 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.905 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:42:53.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:42:53.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.906 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.906 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.906 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724852+0000", 2024-09-17T22:42:53.906 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2599999999999998e-07 2024-09-17T22:42:53.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724854+0000", 2024-09-17T22:42:53.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.26e-06 2024-09-17T22:42:53.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724871+0000", 2024-09-17T22:42:53.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6589999999999999e-05 2024-09-17T22:42:53.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.907 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.907 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.907 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724877+0000", 2024-09-17T22:42:53.907 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.054e-06 2024-09-17T22:42:53.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752951+0000", 2024-09-17T22:42:53.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028073747 2024-09-17T22:42:53.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.908 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752993+0000", 2024-09-17T22:42:53.908 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2082000000000001e-05 2024-09-17T22:42:53.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.908 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.908 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.909 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753058+0000", 2024-09-17T22:42:53.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5049999999999996e-05 2024-09-17T22:42:53.909 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.909 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.909 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.909 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.909 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202885 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:53.909 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:53.909 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.149118661999999, 2024-09-17T22:42:53.909 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028333028, 2024-09-17T22:42:53.909 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.909 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.909 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.910 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.910 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202885 2024-09-17T22:42:53.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.910 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.910 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:53.910 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.910 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.910 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:53.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:42:53.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3300000000000002e-07 2024-09-17T22:42:53.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.911 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.911 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.911 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724892+0000", 2024-09-17T22:42:53.911 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2779999999999999e-06 2024-09-17T22:42:53.911 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724899+0000", 2024-09-17T22:42:53.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9110000000000004e-06 2024-09-17T22:42:53.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724904+0000", 2024-09-17T22:42:53.912 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2360000000000004e-06 2024-09-17T22:42:53.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.912 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.912 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.912 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753094+0000", 2024-09-17T22:42:53.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02819033 2024-09-17T22:42:53.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753142+0000", 2024-09-17T22:42:53.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.829e-05 2024-09-17T22:42:53.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.913 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.913 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.913 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753218+0000", 2024-09-17T22:42:53.913 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5450000000000004e-05 2024-09-17T22:42:53.913 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.914 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.914 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.914 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.914 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202886 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:53.914 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:53.914 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.149093464000003, 2024-09-17T22:42:53.914 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028442512, 2024-09-17T22:42:53.914 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.914 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.914 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.914 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.915 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.915 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202886 2024-09-17T22:42:53.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.915 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:53.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.915 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.915 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.915 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:53.915 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:42:53.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:42:53.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724912+0000", 2024-09-17T22:42:53.916 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.967e-06 2024-09-17T22:42:53.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.916 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.916 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.916 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724918+0000", 2024-09-17T22:42:53.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8069999999999996e-06 2024-09-17T22:42:53.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724923+0000", 2024-09-17T22:42:53.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.386e-06 2024-09-17T22:42:53.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.917 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753244+0000", 2024-09-17T22:42:53.917 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028321567999999998 2024-09-17T22:42:53.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.917 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.917 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753289+0000", 2024-09-17T22:42:53.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4620000000000003e-05 2024-09-17T22:42:53.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.918 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.918 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753353+0000", 2024-09-17T22:42:53.918 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3795999999999997e-05 2024-09-17T22:42:53.918 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.918 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.918 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.918 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.918 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202887 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:53.918 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:53.919 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.149075551000003, 2024-09-17T22:42:53.919 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028565838999999999, 2024-09-17T22:42:53.919 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.919 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.919 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.919 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.919 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.919 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.919 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202887 2024-09-17T22:42:53.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.919 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.919 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.919 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.919 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:53.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:53.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.920 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.920 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:42:53.920 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:42:53.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.920 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724930+0000", 2024-09-17T22:42:53.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8980000000000001e-06 2024-09-17T22:42:53.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724936+0000", 2024-09-17T22:42:53.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7849999999999997e-06 2024-09-17T22:42:53.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.921 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.921 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.921 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724942+0000", 2024-09-17T22:42:53.921 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9649999999999996e-06 2024-09-17T22:42:53.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753380+0000", 2024-09-17T22:42:53.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028438049 2024-09-17T22:42:53.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.922 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753425+0000", 2024-09-17T22:42:53.922 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5163000000000001e-05 2024-09-17T22:42:53.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.922 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.923 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753494+0000", 2024-09-17T22:42:53.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8588000000000001e-05 2024-09-17T22:42:53.923 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.923 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.923 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.923 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.923 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202888 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:53.923 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:42:53.923 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.149056053000002, 2024-09-17T22:42:53.923 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028703314000000001, 2024-09-17T22:42:53.923 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.923 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.923 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.923 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.924 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.924 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.924 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202888 2024-09-17T22:42:53.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.924 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.924 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:42:53.924 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.924 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.924 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:42:53.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724948+0000", 2024-09-17T22:42:53.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:42:53.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.925 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.925 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.925 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724950+0000", 2024-09-17T22:42:53.925 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8029999999999999e-06 2024-09-17T22:42:53.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724955+0000", 2024-09-17T22:42:53.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8760000000000002e-06 2024-09-17T22:42:53.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724960+0000", 2024-09-17T22:42:53.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5979999999999999e-06 2024-09-17T22:42:53.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.926 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.926 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.926 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753527+0000", 2024-09-17T22:42:53.926 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028566864000000001 2024-09-17T22:42:53.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753568+0000", 2024-09-17T22:42:53.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0775000000000002e-05 2024-09-17T22:42:53.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.927 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.927 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.927 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753651+0000", 2024-09-17T22:42:53.927 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3039000000000002e-05 2024-09-17T22:42:53.927 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.927 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.927 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.928 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.928 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202889 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:53.928 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:53.928 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.149038480999998, 2024-09-17T22:42:53.928 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028820964000000001, 2024-09-17T22:42:53.928 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.928 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.928 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.928 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.928 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.928 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.928 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202889 2024-09-17T22:42:53.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.929 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:53.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.929 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.929 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.929 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:53.929 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:42:53.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2599999999999998e-07 2024-09-17T22:42:53.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724967+0000", 2024-09-17T22:42:53.930 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T22:42:53.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.930 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.930 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.930 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724973+0000", 2024-09-17T22:42:53.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6060000000000002e-06 2024-09-17T22:42:53.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724978+0000", 2024-09-17T22:42:53.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8230000000000002e-06 2024-09-17T22:42:53.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.931 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753687+0000", 2024-09-17T22:42:53.931 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028709495000000002 2024-09-17T22:42:53.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.931 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.931 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753727+0000", 2024-09-17T22:42:53.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9953999999999997e-05 2024-09-17T22:42:53.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.932 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.932 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753786+0000", 2024-09-17T22:42:53.932 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8941999999999999e-05 2024-09-17T22:42:53.932 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.932 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.932 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.932 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.932 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202890 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:53.933 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:53.933 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.149020913000001, 2024-09-17T22:42:53.933 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028933838, 2024-09-17T22:42:53.933 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.933 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.933 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.933 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.933 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.933 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.933 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202890 2024-09-17T22:42:53.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.933 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:53.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:53.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.934 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.934 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.934 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:42:53.934 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5999999999999999e-07 2024-09-17T22:42:53.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724985+0000", 2024-09-17T22:42:53.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8169999999999999e-06 2024-09-17T22:42:53.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.935 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724991+0000", 2024-09-17T22:42:53.935 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:42:53.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.935 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.935 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724995+0000", 2024-09-17T22:42:53.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4530000000000004e-06 2024-09-17T22:42:53.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753811+0000", 2024-09-17T22:42:53.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028815584000000002 2024-09-17T22:42:53.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.936 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.936 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.936 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753854+0000", 2024-09-17T22:42:53.936 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3356999999999998e-05 2024-09-17T22:42:53.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.937 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.937 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753916+0000", 2024-09-17T22:42:53.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2488000000000002e-05 2024-09-17T22:42:53.937 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.937 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.937 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.937 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202891 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:53.937 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:53.937 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.149003456000003, 2024-09-17T22:42:53.937 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029067554999999998, 2024-09-17T22:42:53.937 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.937 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.938 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.938 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.938 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.938 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.938 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202891 2024-09-17T22:42:53.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.938 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.938 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:53.938 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.938 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.938 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:53.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:42:53.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:42:53.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.939 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.939 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.939 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725002+0000", 2024-09-17T22:42:53.939 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:42:53.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725008+0000", 2024-09-17T22:42:53.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6629999999999998e-06 2024-09-17T22:42:53.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725016+0000", 2024-09-17T22:42:53.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8269999999999998e-06 2024-09-17T22:42:53.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.940 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.940 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.940 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753949+0000", 2024-09-17T22:42:53.940 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028932625999999999 2024-09-17T22:42:53.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753994+0000", 2024-09-17T22:42:53.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5717000000000003e-05 2024-09-17T22:42:53.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.941 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.941 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.941 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754068+0000", 2024-09-17T22:42:53.941 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3312000000000006e-05 2024-09-17T22:42:53.941 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.941 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.941 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.942 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.942 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202892 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:53.942 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:53.942 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.148960748999997, 2024-09-17T22:42:53.942 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029159939999999999, 2024-09-17T22:42:53.942 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.942 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.942 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.942 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.942 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202892 2024-09-17T22:42:53.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.943 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:53.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.943 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.943 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.943 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:53.943 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:42:53.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7999999999999995e-07 2024-09-17T22:42:53.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.944 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.944 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.944 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725045+0000", 2024-09-17T22:42:53.944 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-17T22:42:53.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725052+0000", 2024-09-17T22:42:53.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3629999999999999e-06 2024-09-17T22:42:53.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725056+0000", 2024-09-17T22:42:53.945 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3250000000000001e-06 2024-09-17T22:42:53.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.945 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.945 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.945 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754094+0000", 2024-09-17T22:42:53.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029038270000000001 2024-09-17T22:42:53.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754134+0000", 2024-09-17T22:42:53.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9722999999999999e-05 2024-09-17T22:42:53.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.946 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.946 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754203+0000", 2024-09-17T22:42:53.946 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8524999999999999e-05 2024-09-17T22:42:53.946 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.946 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.946 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.947 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.947 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202893 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:53.947 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:42:53.947 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.148941172000001, 2024-09-17T22:42:53.947 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029341651, 2024-09-17T22:42:53.947 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.947 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.947 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.947 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.947 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.947 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.947 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202893 2024-09-17T22:42:53.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.948 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:42:53.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.948 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:42:53.948 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.948 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.948 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725063+0000", 2024-09-17T22:42:53.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6699999999999999e-07 2024-09-17T22:42:53.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725065+0000", 2024-09-17T22:42:53.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2120000000000002e-06 2024-09-17T22:42:53.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.949 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.949 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.949 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725071+0000", 2024-09-17T22:42:53.949 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5169999999999996e-06 2024-09-17T22:42:53.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725076+0000", 2024-09-17T22:42:53.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.78e-06 2024-09-17T22:42:53.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754233+0000", 2024-09-17T22:42:53.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029156277000000001 2024-09-17T22:42:53.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.950 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.950 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.950 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754298+0000", 2024-09-17T22:42:53.950 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5320000000000005e-05 2024-09-17T22:42:53.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.951 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.951 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754404+0000", 2024-09-17T22:42:53.951 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106078 2024-09-17T22:42:53.951 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.951 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.951 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.951 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.951 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202894 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:53.951 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:42:53.951 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.148921293000001, 2024-09-17T22:42:53.952 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029456572, 2024-09-17T22:42:53.952 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.952 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.952 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.952 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.952 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.952 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.952 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202894 2024-09-17T22:42:53.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.952 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.952 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.952 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.952 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:42:53.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:42:53.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.953 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725083+0000", 2024-09-17T22:42:53.953 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5900000000000002e-07 2024-09-17T22:42:53.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.953 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.953 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725085+0000", 2024-09-17T22:42:53.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.979e-06 2024-09-17T22:42:53.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725091+0000", 2024-09-17T22:42:53.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1480000000000003e-06 2024-09-17T22:42:53.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.954 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.954 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.954 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725095+0000", 2024-09-17T22:42:53.954 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1339999999999997e-06 2024-09-17T22:42:53.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754429+0000", 2024-09-17T22:42:53.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029333758000000001 2024-09-17T22:42:53.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754471+0000", 2024-09-17T22:42:53.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2274999999999998e-05 2024-09-17T22:42:53.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.955 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.955 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.955 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754539+0000", 2024-09-17T22:42:53.955 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7818999999999996e-05 2024-09-17T22:42:53.955 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.956 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.956 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.956 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.956 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202895 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:53.956 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:53.956 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.148902896999999, 2024-09-17T22:42:53.956 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029635578999999999, 2024-09-17T22:42:53.956 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.956 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.956 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.956 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.956 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.957 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.957 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202895 2024-09-17T22:42:53.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.957 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:53.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.957 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.957 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.957 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:53.957 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:42:53.958 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.08e-07 2024-09-17T22:42:53.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.958 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.958 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.958 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725107+0000", 2024-09-17T22:42:53.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9020000000000004e-06 2024-09-17T22:42:53.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725113+0000", 2024-09-17T22:42:53.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7400000000000001e-06 2024-09-17T22:42:53.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.959 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.959 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725117+0000", 2024-09-17T22:42:53.959 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:42:53.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754567+0000", 2024-09-17T22:42:53.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029449411000000002 2024-09-17T22:42:53.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754638+0000", 2024-09-17T22:42:53.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1432000000000003e-05 2024-09-17T22:42:53.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.960 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.960 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.960 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754736+0000", 2024-09-17T22:42:53.960 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8066000000000006e-05 2024-09-17T22:42:53.960 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.961 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.961 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.961 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.961 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202896 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:53.961 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:42:53.961 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.148881166000002, 2024-09-17T22:42:53.961 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029760534000000002, 2024-09-17T22:42:53.961 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.961 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.961 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.961 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.961 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.961 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.962 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202896 2024-09-17T22:42:53.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.962 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:42:53.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.962 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.962 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:42:53.962 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.962 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725123+0000", 2024-09-17T22:42:53.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:42:53.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725125+0000", 2024-09-17T22:42:53.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.863e-06 2024-09-17T22:42:53.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.963 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.963 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725130+0000", 2024-09-17T22:42:53.963 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7250000000000002e-06 2024-09-17T22:42:53.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725135+0000", 2024-09-17T22:42:53.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:42:53.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754763+0000", 2024-09-17T22:42:53.964 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029628202999999999 2024-09-17T22:42:53.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.964 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.964 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.964 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754805+0000", 2024-09-17T22:42:53.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2370000000000003e-05 2024-09-17T22:42:53.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.965 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.965 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754883+0000", 2024-09-17T22:42:53.965 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7565999999999996e-05 2024-09-17T22:42:53.965 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.965 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.965 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.965 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.965 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202897 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:53.965 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:53.965 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.148863806000001, 2024-09-17T22:42:53.966 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029947972999999999, 2024-09-17T22:42:53.966 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:53.966 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:53.966 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:53.966 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:53.966 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:53.966 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:53.966 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202897 2024-09-17T22:42:53.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.966 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:53.966 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.966 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:53.966 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:53.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:53.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:53.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:53.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:53.967 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:42:53.967 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:42:53.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.967 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:53.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725146+0000", 2024-09-17T22:42:53.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6030000000000004e-06 2024-09-17T22:42:53.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:53.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725152+0000", 2024-09-17T22:42:53.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9970000000000004e-06 2024-09-17T22:42:53.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.968 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:53.968 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725156+0000", 2024-09-17T22:42:53.968 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-06 2024-09-17T22:42:53.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:53.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754909+0000", 2024-09-17T22:42:53.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029752958999999999 2024-09-17T22:42:53.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:53.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754987+0000", 2024-09-17T22:42:53.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8208999999999996e-05 2024-09-17T22:42:53.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:53.969 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:53.969 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:53.969 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.755088+0000", 2024-09-17T22:42:53.969 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100348 2024-09-17T22:42:53.969 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.970 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.970 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.970 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:53.970 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:53.970 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:53.971 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:42:53.972 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:53.969+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:54.072 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:54.070+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:54.076 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:54.076 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:54.076 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:54.076 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:54.085 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:42:54.173 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:54.171+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:54.228 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:54.228 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:54.228 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:54.228 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:54.228 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:54.237 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:42:54.274 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:54.273+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:54.343 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:54.343 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:54.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:54.343 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:54.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.344 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202977 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:54.344 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:42:54.344 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.614104361999999, 2024-09-17T22:42:54.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023068750999999998, 2024-09-17T22:42:54.344 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.344 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.345 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:54.345 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:54.345 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:54.345 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:54.345 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202977 2024-09-17T22:42:54.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.345 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:42:54.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:42:54.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725980+0000", 2024-09-17T22:42:54.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:42:54.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725982+0000", 2024-09-17T22:42:54.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0669999999999999e-06 2024-09-17T22:42:54.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725988+0000", 2024-09-17T22:42:54.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6610000000000003e-06 2024-09-17T22:42:54.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725993+0000", 2024-09-17T22:42:54.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:42:54.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748888+0000", 2024-09-17T22:42:54.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022894930000000001 2024-09-17T22:42:54.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748951+0000", 2024-09-17T22:42:54.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3638000000000003e-05 2024-09-17T22:42:54.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749048+0000", 2024-09-17T22:42:54.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6343999999999998e-05 2024-09-17T22:42:54.349 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.349 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.349 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.349 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202978 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2211840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:54.350 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:54.350 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.61408385, 2024-09-17T22:42:54.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023238050999999999, 2024-09-17T22:42:54.350 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.350 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.350 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:54.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:54.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:54.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:54.350 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202978 2024-09-17T22:42:54.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.350 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:54.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:54.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:42:54.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.58e-07 2024-09-17T22:42:54.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726002+0000", 2024-09-17T22:42:54.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.193e-06 2024-09-17T22:42:54.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726009+0000", 2024-09-17T22:42:54.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8000000000000001e-06 2024-09-17T22:42:54.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726014+0000", 2024-09-17T22:42:54.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9840000000000001e-06 2024-09-17T22:42:54.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749090+0000", 2024-09-17T22:42:54.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023075917000000001 2024-09-17T22:42:54.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749163+0000", 2024-09-17T22:42:54.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3219999999999994e-05 2024-09-17T22:42:54.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749238+0000", 2024-09-17T22:42:54.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4479000000000003e-05 2024-09-17T22:42:54.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.355 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.355 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.355 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202979 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2342912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:54.355 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:42:54.355 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.614062398999998, 2024-09-17T22:42:54.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023393536999999999, 2024-09-17T22:42:54.355 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.355 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.356 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:54.356 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:54.356 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:54.356 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:54.356 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202979 2024-09-17T22:42:54.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.356 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:42:54.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:42:54.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726022+0000", 2024-09-17T22:42:54.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:42:54.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726024+0000", 2024-09-17T22:42:54.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1969999999999999e-06 2024-09-17T22:42:54.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726030+0000", 2024-09-17T22:42:54.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.691e-06 2024-09-17T22:42:54.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726035+0000", 2024-09-17T22:42:54.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8029999999999996e-06 2024-09-17T22:42:54.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749280+0000", 2024-09-17T22:42:54.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023244668 2024-09-17T22:42:54.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749344+0000", 2024-09-17T22:42:54.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4418000000000006e-05 2024-09-17T22:42:54.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749415+0000", 2024-09-17T22:42:54.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0272999999999996e-05 2024-09-17T22:42:54.360 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.360 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.361 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.361 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202980 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2473984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:54.361 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:54.361 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.614041630000003, 2024-09-17T22:42:54.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023562221000000001, 2024-09-17T22:42:54.361 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.361 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.361 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:54.361 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:54.361 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:54.362 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:54.362 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202980 2024-09-17T22:42:54.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.362 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:54.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:54.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:42:54.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:42:54.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.363 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.363 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726044+0000", 2024-09-17T22:42:54.363 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9489999999999999e-06 2024-09-17T22:42:54.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.363 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726066+0000", 2024-09-17T22:42:54.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1508000000000001e-05 2024-09-17T22:42:54.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726072+0000", 2024-09-17T22:42:54.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3450000000000002e-06 2024-09-17T22:42:54.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.364 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.364 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.364 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749455+0000", 2024-09-17T22:42:54.364 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023382824999999999 2024-09-17T22:42:54.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749527+0000", 2024-09-17T22:42:54.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1728000000000002e-05 2024-09-17T22:42:54.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.365 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.365 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.365 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749604+0000", 2024-09-17T22:42:54.365 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7428999999999993e-05 2024-09-17T22:42:54.366 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.366 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.366 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.366 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.366 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202981 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:54.366 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:42:54.366 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.614003355000001, 2024-09-17T22:42:54.366 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023677800999999998, 2024-09-17T22:42:54.366 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.366 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.366 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:54.367 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:54.367 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:54.367 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:54.367 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202981 2024-09-17T22:42:54.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.367 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.367 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.367 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.367 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:42:54.367 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:42:54.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.368 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.368 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726081+0000", 2024-09-17T22:42:54.368 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5499999999999998e-07 2024-09-17T22:42:54.368 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.368 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726083+0000", 2024-09-17T22:42:54.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1780000000000002e-06 2024-09-17T22:42:54.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.369 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726090+0000", 2024-09-17T22:42:54.369 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8510000000000001e-06 2024-09-17T22:42:54.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.369 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.369 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726101+0000", 2024-09-17T22:42:54.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.133e-05 2024-09-17T22:42:54.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749639+0000", 2024-09-17T22:42:54.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023537679999999998 2024-09-17T22:42:54.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.370 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.370 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.370 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749688+0000", 2024-09-17T22:42:54.370 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9548000000000001e-05 2024-09-17T22:42:54.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.371 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.371 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749758+0000", 2024-09-17T22:42:54.371 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9758999999999994e-05 2024-09-17T22:42:54.371 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.371 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.371 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.371 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.371 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202982 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:54.371 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:54.372 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.613975772000003, 2024-09-17T22:42:54.372 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023841469000000001, 2024-09-17T22:42:54.372 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.372 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.372 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:54.372 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:54.372 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:54.372 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:54.372 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202982 2024-09-17T22:42:54.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.372 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:54.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.373 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.373 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:54.373 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.373 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:42:54.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:42:54.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.374 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726110+0000", 2024-09-17T22:42:54.374 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.902e-06 2024-09-17T22:42:54.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.374 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.374 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726117+0000", 2024-09-17T22:42:54.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6189999999999996e-06 2024-09-17T22:42:54.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726122+0000", 2024-09-17T22:42:54.375 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.198e-06 2024-09-17T22:42:54.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.375 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.375 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.375 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749795+0000", 2024-09-17T22:42:54.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023673397999999998 2024-09-17T22:42:54.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749877+0000", 2024-09-17T22:42:54.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1910999999999996e-05 2024-09-17T22:42:54.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.376 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.376 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.376 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749949+0000", 2024-09-17T22:42:54.376 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2001000000000004e-05 2024-09-17T22:42:54.377 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.377 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.377 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.377 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.377 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202983 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:54.377 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:54.377 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.613954712999998, 2024-09-17T22:42:54.377 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024014081, 2024-09-17T22:42:54.377 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.377 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.377 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:54.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:54.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:54.378 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:54.378 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202983 2024-09-17T22:42:54.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.378 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.378 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.378 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.378 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:54.378 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:54.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.379 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.379 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.379 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:42:54.379 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:42:54.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726131+0000", 2024-09-17T22:42:54.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9429999999999999e-06 2024-09-17T22:42:54.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.380 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.380 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726138+0000", 2024-09-17T22:42:54.380 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T22:42:54.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.380 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726143+0000", 2024-09-17T22:42:54.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0629999999999996e-06 2024-09-17T22:42:54.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.381 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.381 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749991+0000", 2024-09-17T22:42:54.381 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023847671000000001 2024-09-17T22:42:54.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.381 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750065+0000", 2024-09-17T22:42:54.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4569999999999999e-05 2024-09-17T22:42:54.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.382 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.382 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.382 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750143+0000", 2024-09-17T22:42:54.382 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7649000000000001e-05 2024-09-17T22:42:54.382 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.382 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.382 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.383 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.383 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202984 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:54.383 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:42:54.383 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.613934012000001, 2024-09-17T22:42:54.383 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024177605000000001, 2024-09-17T22:42:54.383 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.383 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.383 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:54.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:54.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:54.383 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:54.384 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202984 2024-09-17T22:42:54.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.384 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:42:54.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.384 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.384 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.384 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:42:54.384 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726150+0000", 2024-09-17T22:42:54.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T22:42:54.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.385 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.385 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726152+0000", 2024-09-17T22:42:54.385 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:42:54.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.385 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726162+0000", 2024-09-17T22:42:54.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7839999999999998e-06 2024-09-17T22:42:54.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726167+0000", 2024-09-17T22:42:54.386 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5690000000000004e-06 2024-09-17T22:42:54.386 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.386 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.386 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.386 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750184+0000", 2024-09-17T22:42:54.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024016909999999999 2024-09-17T22:42:54.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750255+0000", 2024-09-17T22:42:54.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1137999999999996e-05 2024-09-17T22:42:54.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.387 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.387 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.387 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750327+0000", 2024-09-17T22:42:54.387 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1903000000000006e-05 2024-09-17T22:42:54.388 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.388 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.388 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.388 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.388 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202985 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:54.388 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:54.388 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.613908625000001, 2024-09-17T22:42:54.388 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024280500999999999, 2024-09-17T22:42:54.388 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.388 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.389 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:54.389 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:54.389 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:54.389 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:54.389 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202985 2024-09-17T22:42:54.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.389 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.389 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.389 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.389 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:54.389 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:54.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.390 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.390 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.390 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:42:54.390 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8100000000000003e-07 2024-09-17T22:42:54.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726182+0000", 2024-09-17T22:42:54.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:42:54.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726190+0000", 2024-09-17T22:42:54.391 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:42:54.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.391 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.391 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.391 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726195+0000", 2024-09-17T22:42:54.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2809999999999999e-06 2024-09-17T22:42:54.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750366+0000", 2024-09-17T22:42:54.392 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024171635 2024-09-17T22:42:54.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.392 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.392 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.392 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750410+0000", 2024-09-17T22:42:54.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3823999999999999e-05 2024-09-17T22:42:54.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.393 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.393 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750455+0000", 2024-09-17T22:42:54.393 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5039e-05 2024-09-17T22:42:54.393 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.393 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.393 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.393 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.394 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202986 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:54.394 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:42:54.394 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.613881044999999, 2024-09-17T22:42:54.394 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024453077, 2024-09-17T22:42:54.394 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.394 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.394 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:54.394 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:54.394 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:54.394 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:54.394 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202986 2024-09-17T22:42:54.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.395 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:42:54.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.395 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.395 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:42:54.395 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.395 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726203+0000", 2024-09-17T22:42:54.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T22:42:54.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.396 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726205+0000", 2024-09-17T22:42:54.396 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0250000000000001e-06 2024-09-17T22:42:54.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.396 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.396 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726211+0000", 2024-09-17T22:42:54.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1340000000000001e-06 2024-09-17T22:42:54.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726222+0000", 2024-09-17T22:42:54.397 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1105e-05 2024-09-17T22:42:54.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.397 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.397 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.397 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750520+0000", 2024-09-17T22:42:54.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024297462999999998 2024-09-17T22:42:54.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750585+0000", 2024-09-17T22:42:54.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5940999999999998e-05 2024-09-17T22:42:54.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.398 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.398 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.398 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750655+0000", 2024-09-17T22:42:54.398 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0012000000000007e-05 2024-09-17T22:42:54.398 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.399 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.399 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.399 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.399 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202987 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:54.399 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:54.399 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.613854664999998, 2024-09-17T22:42:54.399 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024622569, 2024-09-17T22:42:54.399 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.399 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.399 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:54.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:54.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:54.400 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:54.400 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202987 2024-09-17T22:42:54.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.400 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.400 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.400 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:54.400 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.400 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.400 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:54.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:42:54.401 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:42:54.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.401 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.401 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.401 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726238+0000", 2024-09-17T22:42:54.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7430000000000008e-06 2024-09-17T22:42:54.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726245+0000", 2024-09-17T22:42:54.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6640000000000001e-06 2024-09-17T22:42:54.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.402 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.402 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.402 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726249+0000", 2024-09-17T22:42:54.402 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9119999999999997e-06 2024-09-17T22:42:54.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750694+0000", 2024-09-17T22:42:54.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024444105000000001 2024-09-17T22:42:54.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.403 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.403 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750767+0000", 2024-09-17T22:42:54.403 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3577000000000004e-05 2024-09-17T22:42:54.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.404 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.404 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750851+0000", 2024-09-17T22:42:54.404 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4189000000000003e-05 2024-09-17T22:42:54.404 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.404 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.404 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.404 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.404 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.404 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202988 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:54.404 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:54.404 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.613827803, 2024-09-17T22:42:54.405 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024808883, 2024-09-17T22:42:54.405 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.405 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.405 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:54.405 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:54.405 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:54.405 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:54.405 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202988 2024-09-17T22:42:54.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.405 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.405 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:54.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:54.406 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.406 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.406 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.406 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:42:54.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5499999999999999e-07 2024-09-17T22:42:54.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726258+0000", 2024-09-17T22:42:54.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8020000000000001e-06 2024-09-17T22:42:54.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.407 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.407 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.407 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726272+0000", 2024-09-17T22:42:54.407 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3828e-05 2024-09-17T22:42:54.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726277+0000", 2024-09-17T22:42:54.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2379999999999997e-06 2024-09-17T22:42:54.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.408 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.408 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.408 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750895+0000", 2024-09-17T22:42:54.408 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024618543 2024-09-17T22:42:54.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750978+0000", 2024-09-17T22:42:54.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3030999999999998e-05 2024-09-17T22:42:54.409 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.409 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.409 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.409 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751065+0000", 2024-09-17T22:42:54.409 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6086000000000002e-05 2024-09-17T22:42:54.410 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.410 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.410 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.410 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.410 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202989 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:54.410 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:42:54.410 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.613798930000002, 2024-09-17T22:42:54.410 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024976788, 2024-09-17T22:42:54.410 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.410 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.410 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:54.410 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:54.411 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:54.411 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:54.411 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202989 2024-09-17T22:42:54.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.411 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.411 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.411 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:42:54.411 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:42:54.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726285+0000", 2024-09-17T22:42:54.412 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T22:42:54.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.412 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.412 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.412 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726287+0000", 2024-09-17T22:42:54.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9860000000000001e-06 2024-09-17T22:42:54.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726294+0000", 2024-09-17T22:42:54.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2849999999999997e-06 2024-09-17T22:42:54.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.413 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.413 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.413 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726299+0000", 2024-09-17T22:42:54.413 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0150000000000003e-06 2024-09-17T22:42:54.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751109+0000", 2024-09-17T22:42:54.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024810175 2024-09-17T22:42:54.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.414 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.414 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751184+0000", 2024-09-17T22:42:54.414 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4640000000000004e-05 2024-09-17T22:42:54.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.414 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.415 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.415 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751261+0000", 2024-09-17T22:42:54.415 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7317000000000005e-05 2024-09-17T22:42:54.415 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.415 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.415 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.415 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.415 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202990 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:54.415 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:54.415 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.613778715000002, 2024-09-17T22:42:54.416 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025156767999999999, 2024-09-17T22:42:54.416 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.416 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.416 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:54.416 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:54.416 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:54.416 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:54.416 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202990 2024-09-17T22:42:54.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.416 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.416 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.416 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:54.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:54.417 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.417 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.417 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.417 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.417 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:42:54.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:42:54.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726307+0000", 2024-09-17T22:42:54.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7090000000000001e-06 2024-09-17T22:42:54.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.418 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.418 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.418 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726312+0000", 2024-09-17T22:42:54.418 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4909999999999996e-06 2024-09-17T22:42:54.418 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726317+0000", 2024-09-17T22:42:54.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3800000000000004e-06 2024-09-17T22:42:54.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.419 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751308+0000", 2024-09-17T22:42:54.419 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024991143 2024-09-17T22:42:54.419 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.419 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.419 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751385+0000", 2024-09-17T22:42:54.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6793999999999997e-05 2024-09-17T22:42:54.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.420 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.420 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751461+0000", 2024-09-17T22:42:54.420 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6891999999999995e-05 2024-09-17T22:42:54.420 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.420 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.420 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.420 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.421 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202991 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:54.421 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:54.421 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.613760681000002, 2024-09-17T22:42:54.421 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025289477000000001, 2024-09-17T22:42:54.421 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.421 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.421 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:54.421 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:54.421 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:54.421 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:54.421 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202991 2024-09-17T22:42:54.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.422 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:54.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.422 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.422 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.422 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:54.422 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:42:54.423 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.27e-07 2024-09-17T22:42:54.423 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.423 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.423 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.423 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726325+0000", 2024-09-17T22:42:54.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1459999999999999e-06 2024-09-17T22:42:54.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726333+0000", 2024-09-17T22:42:54.424 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1289999999999996e-06 2024-09-17T22:42:54.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.424 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.424 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.424 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726339+0000", 2024-09-17T22:42:54.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2530000000000004e-06 2024-09-17T22:42:54.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751499+0000", 2024-09-17T22:42:54.425 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025160109 2024-09-17T22:42:54.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.425 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.425 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.425 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751543+0000", 2024-09-17T22:42:54.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4317000000000003e-05 2024-09-17T22:42:54.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.426 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.426 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.426 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751612+0000", 2024-09-17T22:42:54.426 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9096000000000004e-05 2024-09-17T22:42:54.426 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.426 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.427 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.427 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.427 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202992 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:54.427 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:54.427 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.613738617000003, 2024-09-17T22:42:54.427 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025420585999999998, 2024-09-17T22:42:54.427 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.427 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.428 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:54.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:54.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:54.428 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:54.428 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202992 2024-09-17T22:42:54.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.428 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.428 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.428 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:42:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5199999999999998e-07 2024-09-17T22:42:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726347+0000", 2024-09-17T22:42:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9939999999999999e-06 2024-09-17T22:42:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726358+0000", 2024-09-17T22:42:54.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0421999999999999e-05 2024-09-17T22:42:54.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.431 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.431 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.431 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726363+0000", 2024-09-17T22:42:54.431 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:42:54.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751644+0000", 2024-09-17T22:42:54.432 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025280727999999999 2024-09-17T22:42:54.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.432 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.432 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.432 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751712+0000", 2024-09-17T22:42:54.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8207999999999995e-05 2024-09-17T22:42:54.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.433 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.433 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751765+0000", 2024-09-17T22:42:54.433 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3533999999999999e-05 2024-09-17T22:42:54.433 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.433 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.433 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.434 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.434 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202993 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:42:54.434 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:54.434 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 36.613714625999997, 2024-09-17T22:42:54.434 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025516259999999999, 2024-09-17T22:42:54.434 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.434 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.434 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:54.435 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:54.435 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:54.435 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:54.435 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202993 2024-09-17T22:42:54.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.435 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.435 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.435 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.435 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:54.435 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:54.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.436 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.436 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:42:54.436 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:42:54.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.436 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726371+0000", 2024-09-17T22:42:54.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T22:42:54.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726380+0000", 2024-09-17T22:42:54.437 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9109999999999999e-06 2024-09-17T22:42:54.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.437 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.437 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.437 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726385+0000", 2024-09-17T22:42:54.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7380000000000002e-06 2024-09-17T22:42:54.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751785+0000", 2024-09-17T22:42:54.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025400526 2024-09-17T22:42:54.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.438 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.438 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.438 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751824+0000", 2024-09-17T22:42:54.438 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8989999999999998e-05 2024-09-17T22:42:54.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.439 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.439 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751885+0000", 2024-09-17T22:42:54.439 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0937000000000003e-05 2024-09-17T22:42:54.439 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.439 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.439 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.439 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.439 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:215737 3.b 3:d5ee24db:::rbd_data.115f2f67ca35.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-17T22:42:54.439 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:42:54.440 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.038945931000001, 2024-09-17T22:42:54.440 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025642133000000001, 2024-09-17T22:42:54.440 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.440 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.440 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:54.440 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:54.440 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:54.440 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:54.440 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 215737 2024-09-17T22:42:54.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.440 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.440 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:42:54.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.441 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.441 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:42:54.441 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.441 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301138+0000", 2024-09-17T22:42:54.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8100000000000002e-07 2024-09-17T22:42:54.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.442 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301143+0000", 2024-09-17T22:42:54.442 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6999999999999999e-06 2024-09-17T22:42:54.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.442 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.442 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301152+0000", 2024-09-17T22:42:54.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4979999999999995e-06 2024-09-17T22:42:54.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301159+0000", 2024-09-17T22:42:54.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9430000000000003e-06 2024-09-17T22:42:54.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.443 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.443 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.443 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301201+0000", 2024-09-17T22:42:54.443 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1219e-05 2024-09-17T22:42:54.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301247+0000", 2024-09-17T22:42:54.444 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5924999999999998e-05 2024-09-17T22:42:54.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.444 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.444 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:42:54.444 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301870+0000", 2024-09-17T22:42:54.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00062316100000000001 2024-09-17T22:42:54.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:54.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326658+0000", 2024-09-17T22:42:54.445 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024788719000000001 2024-09-17T22:42:54.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.445 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.445 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:42:54.445 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326694+0000", 2024-09-17T22:42:54.446 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5858999999999997e-05 2024-09-17T22:42:54.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.446 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.446 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:54.446 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326727+0000", 2024-09-17T22:42:54.448 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3176999999999998e-05 2024-09-17T22:42:54.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.448 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.448 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.448 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326780+0000", 2024-09-17T22:42:54.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2151000000000003e-05 2024-09-17T22:42:54.449 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.449 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.449 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.449 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.449 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:215738 3.b 3:d5937cde:::rbd_data.115f2f67ca35.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-17T22:42:54.449 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:42:54.449 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.038909399000001, 2024-09-17T22:42:54.449 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025650267000000001, 2024-09-17T22:42:54.449 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.450 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.450 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:54.450 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:54.450 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:54.450 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:54.450 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 215738 2024-09-17T22:42:54.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.450 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.450 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.450 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.450 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:42:54.450 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:42:54.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.451 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.451 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.451 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301175+0000", 2024-09-17T22:42:54.451 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2600000000000002e-07 2024-09-17T22:42:54.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301178+0000", 2024-09-17T22:42:54.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2049999999999998e-06 2024-09-17T22:42:54.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.452 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.452 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301185+0000", 2024-09-17T22:42:54.452 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8900000000000001e-06 2024-09-17T22:42:54.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.452 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301190+0000", 2024-09-17T22:42:54.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6910000000000002e-06 2024-09-17T22:42:54.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.453 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.304605+0000", 2024-09-17T22:42:54.453 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003414594 2024-09-17T22:42:54.453 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.453 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.453 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.304658+0000", 2024-09-17T22:42:54.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3334000000000001e-05 2024-09-17T22:42:54.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:42:54.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.308265+0000", 2024-09-17T22:42:54.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036068680000000001 2024-09-17T22:42:54.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.454 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.454 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:54.454 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326688+0000", 2024-09-17T22:42:54.454 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018422851000000001 2024-09-17T22:42:54.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:42:54.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326785+0000", 2024-09-17T22:42:54.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7050000000000001e-05 2024-09-17T22:42:54.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.455 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.455 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:42:54.455 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326806+0000", 2024-09-17T22:42:54.455 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1299999999999999e-05 2024-09-17T22:42:54.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.456 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.456 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326824+0000", 2024-09-17T22:42:54.456 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7958e-05 2024-09-17T22:42:54.456 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.456 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.456 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.456 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.456 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop_reply(client.4447.0:215737 3.b e87/23 ondisk, result = 0)", 2024-09-17T22:42:54.456 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:42:54.457 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 19.036454188, 2024-09-17T22:42:54.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023035287000000002, 2024-09-17T22:42:54.457 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:54.457 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:54.457 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:42:54.457 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:54.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.457 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:54.457 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:42:54.457 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.457 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:54.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:42:54.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:54.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:54.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303630+0000", 2024-09-17T22:42:54.458 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2e-07 2024-09-17T22:42:54.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.458 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.458 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:54.458 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303633+0000", 2024-09-17T22:42:54.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7470000000000001e-06 2024-09-17T22:42:54.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:54.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303641+0000", 2024-09-17T22:42:54.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7979999999999995e-06 2024-09-17T22:42:54.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.459 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.459 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:54.459 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303646+0000", 2024-09-17T22:42:54.459 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9910000000000002e-06 2024-09-17T22:42:54.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:54.460 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326647+0000", 2024-09-17T22:42:54.460 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023000936999999999 2024-09-17T22:42:54.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.460 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.460 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:42:54.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326654+0000", 2024-09-17T22:42:54.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2529999999999998e-06 2024-09-17T22:42:54.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:54.461 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:54.461 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:54.461 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326665+0000", 2024-09-17T22:42:54.461 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0040999999999999e-05 2024-09-17T22:42:54.461 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.461 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.461 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.461 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:54.461 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:54.462 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:54.462 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:42:54.463 INFO:teuthology.orchestra.run.smithi115.stderr:set 3.1d pg_upmap mapping to [1,6] 2024-09-17T22:42:54.464 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:54.460+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:54.465 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:42:54.564 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:54.562+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:54.584 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:54.584 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:54.584 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:54.584 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:54.593 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:42:54.619 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:42:54.677 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:54.676+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:54.741 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:54.744 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:54.745 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:54.745 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:54.745 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:54.745 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:54.757 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:42:54.765 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:54.765 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:54.766 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:54.763+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:54.866 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:54.864+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:54.867 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:54.867 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:54.876 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:42:54.968 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:54.965+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:54.993 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:54.993 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:54.994 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:54.994 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:54.994 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is set 2024-09-17T22:42:55.001 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:55.001 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:55.006 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:42:55.019 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:55.019 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:55.036 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:55.036 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:55.053 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:55.053 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:55.068 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:55.066+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:55.070 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:55.070 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:55.087 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:55.087 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:55.090 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:55.090 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:55.122 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:55.122 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:55.131 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:42:55.170 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:55.167+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:55.270 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:55.268+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:55.274 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:55.274 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:55.282 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:42:55.371 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:55.369+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:55.386 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:55.386 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:55.395 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:42:55.473 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:55.470+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:55.538 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:55.539 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:55.548 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:42:55.574 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:55.572+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:55.650 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:55.650 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:55.659 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:42:55.702 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:55.700+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:55.772 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:55.772 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:55.775 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:55.773+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:55.782 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:42:55.876 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:55.874+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:55.927 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:55.928 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:55.937 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:42:55.978 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:55.975+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:56.079 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:56.076+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:56.081 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:56.081 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:56.090 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:42:56.180 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:56.178+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:56.234 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:56.234 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:56.243 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:42:56.282 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:56.279+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:56.382 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:56.380+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:56.387 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:56.388 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:56.397 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:42:56.483 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:56.481+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:56.561 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:56.561 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:56.571 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:42:56.584 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:56.582+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:56.674 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:56.674 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:56.683 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:42:56.686 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:56.683+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:56.786 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:56.786 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:56.786 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:56.784+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:56.795 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:42:56.887 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:56.885+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:56.898 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:56.898 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:56.907 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:42:56.989 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:42:56.987+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:42:57.010 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:42:57.017 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:57.017 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:57.026 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:42:57.090 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:57.088+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:57.091 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:42:57.133 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:57.133 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:57.142 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:42:57.196 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:57.213 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:57.213 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:57.225 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:57.223+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:57.230 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:57.230 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:57.246 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:57.246 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:57.263 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:57.263 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:57.280 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:57.280 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:57.286 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:57.286 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:57.292 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:57.289+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:57.295 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:42:57.296 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:57.296 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:57.312 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:57.313 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:57.316 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:57.316 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:57.393 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:57.391+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:57.399 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:57.399 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:57.399 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:57.399 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:57.407 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:42:57.494 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:57.491+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:57.511 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:57.511 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:57.511 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:57.511 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:57.511 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:57.520 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:42:57.595 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:57.592+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:57.665 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:57.665 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:57.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:57.666 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:57.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.666 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:42:57.666 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:57.666 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 114.303197775, 2024-09-17T22:42:57.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.249158453, 2024-09-17T22:42:57.666 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.666 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.666 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:42:57.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:42:57.667 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:42:57.667 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-17T22:42:57.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.667 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:57.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:42:57.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358680+0000", 2024-09-17T22:42:57.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.592e-06 2024-09-17T22:42:57.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358688+0000", 2024-09-17T22:42:57.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1610000000000003e-06 2024-09-17T22:42:57.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358697+0000", 2024-09-17T22:42:57.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2580000000000001e-06 2024-09-17T22:42:57.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358706+0000", 2024-09-17T22:42:57.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1640000000000007e-06 2024-09-17T22:42:57.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358766+0000", 2024-09-17T22:42:57.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9893999999999999e-05 2024-09-17T22:42:57.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:42:57.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358778+0000", 2024-09-17T22:42:57.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1148999999999999e-05 2024-09-17T22:42:57.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606347+0000", 2024-09-17T22:42:57.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.247569971 2024-09-17T22:42:57.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606533+0000", 2024-09-17T22:42:57.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018599400000000001 2024-09-17T22:42:57.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:42:57.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606752+0000", 2024-09-17T22:42:57.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000218373 2024-09-17T22:42:57.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:42:57.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607455+0000", 2024-09-17T22:42:57.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070311799999999997 2024-09-17T22:42:57.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:42:57.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607735+0000", 2024-09-17T22:42:57.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027972300000000001 2024-09-17T22:42:57.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:42:57.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607793+0000", 2024-09-17T22:42:57.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8529999999999997e-05 2024-09-17T22:42:57.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607837+0000", 2024-09-17T22:42:57.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3526000000000003e-05 2024-09-17T22:42:57.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.674 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.674 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186749 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 11960320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:57.674 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:57.674 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.238556453000001, 2024-09-17T22:42:57.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029226166000000001, 2024-09-17T22:42:57.675 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.675 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.675 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:57.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:57.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:57.675 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186749 2024-09-17T22:42:57.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.676 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:57.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:57.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:42:57.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-09-17T22:42:57.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423328+0000", 2024-09-17T22:42:57.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6790000000000005e-06 2024-09-17T22:42:57.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423334+0000", 2024-09-17T22:42:57.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0009999999999998e-06 2024-09-17T22:42:57.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423338+0000", 2024-09-17T22:42:57.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6419999999999998e-06 2024-09-17T22:42:57.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452464+0000", 2024-09-17T22:42:57.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029125389000000002 2024-09-17T22:42:57.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452501+0000", 2024-09-17T22:42:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7656e-05 2024-09-17T22:42:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452546+0000", 2024-09-17T22:42:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4468999999999997e-05 2024-09-17T22:42:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.680 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186750 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12050432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:57.680 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:57.680 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.238530662999999, 2024-09-17T22:42:57.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029307863999999999, 2024-09-17T22:42:57.680 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.680 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.680 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:57.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:57.681 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:57.681 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186750 2024-09-17T22:42:57.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.681 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:57.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:42:57.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423346+0000", 2024-09-17T22:42:57.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.84e-07 2024-09-17T22:42:57.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423348+0000", 2024-09-17T22:42:57.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7540000000000001e-06 2024-09-17T22:42:57.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423353+0000", 2024-09-17T22:42:57.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:42:57.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423362+0000", 2024-09-17T22:42:57.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9220000000000003e-06 2024-09-17T22:42:57.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452571+0000", 2024-09-17T22:42:57.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029209174000000001 2024-09-17T22:42:57.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452612+0000", 2024-09-17T22:42:57.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1363999999999999e-05 2024-09-17T22:42:57.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452653+0000", 2024-09-17T22:42:57.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0893999999999998e-05 2024-09-17T22:42:57.685 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.685 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.685 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.685 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186751 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12140544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:57.685 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:57.686 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.238508173, 2024-09-17T22:42:57.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029387488, 2024-09-17T22:42:57.686 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.686 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.686 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:57.686 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.686 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:57.686 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:57.686 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186751 2024-09-17T22:42:57.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.686 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:57.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:57.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:42:57.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1399999999999997e-07 2024-09-17T22:42:57.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423375+0000", 2024-09-17T22:42:57.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8889999999999996e-06 2024-09-17T22:42:57.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423381+0000", 2024-09-17T22:42:57.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5690000000000004e-06 2024-09-17T22:42:57.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423386+0000", 2024-09-17T22:42:57.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9559999999999996e-06 2024-09-17T22:42:57.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452679+0000", 2024-09-17T22:42:57.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029292820000000001 2024-09-17T22:42:57.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452718+0000", 2024-09-17T22:42:57.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8961000000000002e-05 2024-09-17T22:42:57.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452755+0000", 2024-09-17T22:42:57.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7879000000000001e-05 2024-09-17T22:42:57.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.691 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.691 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186752 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12230656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:57.691 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:57.691 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.238476921999997, 2024-09-17T22:42:57.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029463269, 2024-09-17T22:42:57.691 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.691 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.691 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:57.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:57.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:57.692 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186752 2024-09-17T22:42:57.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.692 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:57.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:42:57.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423400+0000", 2024-09-17T22:42:57.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:42:57.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423401+0000", 2024-09-17T22:42:57.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3599999999999999e-06 2024-09-17T22:42:57.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423404+0000", 2024-09-17T22:42:57.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5999999999999998e-06 2024-09-17T22:42:57.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423407+0000", 2024-09-17T22:42:57.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6869999999999999e-06 2024-09-17T22:42:57.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452782+0000", 2024-09-17T22:42:57.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029375107000000001 2024-09-17T22:42:57.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452821+0000", 2024-09-17T22:42:57.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9122999999999998e-05 2024-09-17T22:42:57.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452862+0000", 2024-09-17T22:42:57.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1019000000000001e-05 2024-09-17T22:42:57.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.696 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.697 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186753 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:57.697 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:57.697 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.238464899999997, 2024-09-17T22:42:57.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029553297999999999, 2024-09-17T22:42:57.697 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.697 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.697 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:57.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:57.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:57.697 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186753 2024-09-17T22:42:57.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.698 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:57.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:57.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:42:57.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.98e-07 2024-09-17T22:42:57.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423412+0000", 2024-09-17T22:42:57.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8700000000000004e-07 2024-09-17T22:42:57.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423415+0000", 2024-09-17T22:42:57.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:42:57.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423418+0000", 2024-09-17T22:42:57.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.283e-06 2024-09-17T22:42:57.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452893+0000", 2024-09-17T22:42:57.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029475881999999998 2024-09-17T22:42:57.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452928+0000", 2024-09-17T22:42:57.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4109000000000002e-05 2024-09-17T22:42:57.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452964+0000", 2024-09-17T22:42:57.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6937000000000002e-05 2024-09-17T22:42:57.702 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.702 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.702 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.702 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186754 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12410880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:57.702 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:57.703 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.238455762999997, 2024-09-17T22:42:57.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029656021000000001, 2024-09-17T22:42:57.703 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.703 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.703 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:57.703 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.703 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:57.703 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:57.703 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186754 2024-09-17T22:42:57.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.703 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:57.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:42:57.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423421+0000", 2024-09-17T22:42:57.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9999999999999999e-07 2024-09-17T22:42:57.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423425+0000", 2024-09-17T22:42:57.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6519999999999997e-06 2024-09-17T22:42:57.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423428+0000", 2024-09-17T22:42:57.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1370000000000002e-06 2024-09-17T22:42:57.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423431+0000", 2024-09-17T22:42:57.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4090000000000001e-06 2024-09-17T22:42:57.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452992+0000", 2024-09-17T22:42:57.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029561545000000002 2024-09-17T22:42:57.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453031+0000", 2024-09-17T22:42:57.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8834000000000001e-05 2024-09-17T22:42:57.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453076+0000", 2024-09-17T22:42:57.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5244000000000003e-05 2024-09-17T22:42:57.708 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.708 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.708 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.708 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186755 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12500992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:57.708 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:57.708 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.238442482000004, 2024-09-17T22:42:57.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029745649999999998, 2024-09-17T22:42:57.708 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.709 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.709 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:57.709 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.709 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:57.709 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:57.709 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186755 2024-09-17T22:42:57.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.709 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:57.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:57.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:42:57.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.17e-07 2024-09-17T22:42:57.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423435+0000", 2024-09-17T22:42:57.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:42:57.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423438+0000", 2024-09-17T22:42:57.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8210000000000002e-06 2024-09-17T22:42:57.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423443+0000", 2024-09-17T22:42:57.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1070000000000004e-06 2024-09-17T22:42:57.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453103+0000", 2024-09-17T22:42:57.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029660399 2024-09-17T22:42:57.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453141+0000", 2024-09-17T22:42:57.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.748e-05 2024-09-17T22:42:57.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453179+0000", 2024-09-17T22:42:57.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8671000000000003e-05 2024-09-17T22:42:57.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.713 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.713 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.714 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187439 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8437760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:57.714 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:57.714 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.121005384999997, 2024-09-17T22:42:57.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029278048000000001, 2024-09-17T22:42:57.714 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.714 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.714 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:57.714 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.714 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:57.715 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:57.715 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187439 2024-09-17T22:42:57.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.715 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:57.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:57.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:42:57.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6199999999999999e-07 2024-09-17T22:42:57.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540872+0000", 2024-09-17T22:42:57.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.09e-07 2024-09-17T22:42:57.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540875+0000", 2024-09-17T22:42:57.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8990000000000001e-06 2024-09-17T22:42:57.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540877+0000", 2024-09-17T22:42:57.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1789999999999998e-06 2024-09-17T22:42:57.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570074+0000", 2024-09-17T22:42:57.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029197358 2024-09-17T22:42:57.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570111+0000", 2024-09-17T22:42:57.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6170000000000001e-05 2024-09-17T22:42:57.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570149+0000", 2024-09-17T22:42:57.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8170999999999998e-05 2024-09-17T22:42:57.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.720 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.720 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187440 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8527872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:57.720 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:57.720 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.120989915000003, 2024-09-17T22:42:57.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029359758999999999, 2024-09-17T22:42:57.720 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.720 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.720 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:57.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:57.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:57.721 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187440 2024-09-17T22:42:57.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.721 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:57.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:57.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:42:57.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2700000000000001e-07 2024-09-17T22:42:57.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540888+0000", 2024-09-17T22:42:57.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-17T22:42:57.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540893+0000", 2024-09-17T22:42:57.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9110000000000001e-06 2024-09-17T22:42:57.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540895+0000", 2024-09-17T22:42:57.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.712e-06 2024-09-17T22:42:57.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570168+0000", 2024-09-17T22:42:57.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029273114999999999 2024-09-17T22:42:57.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570208+0000", 2024-09-17T22:42:57.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9227999999999998e-05 2024-09-17T22:42:57.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570246+0000", 2024-09-17T22:42:57.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8253000000000001e-05 2024-09-17T22:42:57.725 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.725 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.725 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.726 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187441 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8617984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:57.726 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:57.726 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.120976951000003, 2024-09-17T22:42:57.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029442211999999999, 2024-09-17T22:42:57.726 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.726 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.726 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:57.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:57.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:57.727 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187441 2024-09-17T22:42:57.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.727 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:57.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:57.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:42:57.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:42:57.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540900+0000", 2024-09-17T22:42:57.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7999999999999993e-07 2024-09-17T22:42:57.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540903+0000", 2024-09-17T22:42:57.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9129999999999999e-06 2024-09-17T22:42:57.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540909+0000", 2024-09-17T22:42:57.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4269999999999999e-06 2024-09-17T22:42:57.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570267+0000", 2024-09-17T22:42:57.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029358308 2024-09-17T22:42:57.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570304+0000", 2024-09-17T22:42:57.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6860999999999998e-05 2024-09-17T22:42:57.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570341+0000", 2024-09-17T22:42:57.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7509000000000003e-05 2024-09-17T22:42:57.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.731 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.732 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187442 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8708096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:57.732 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:57.732 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.120964244, 2024-09-17T22:42:57.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029522606999999999, 2024-09-17T22:42:57.732 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.732 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.732 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:57.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:57.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:57.733 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187442 2024-09-17T22:42:57.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.733 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:57.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:57.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:42:57.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.22e-07 2024-09-17T22:42:57.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540917+0000", 2024-09-17T22:42:57.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999998e-06 2024-09-17T22:42:57.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540920+0000", 2024-09-17T22:42:57.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4989999999999999e-06 2024-09-17T22:42:57.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540923+0000", 2024-09-17T22:42:57.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3870000000000002e-06 2024-09-17T22:42:57.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570361+0000", 2024-09-17T22:42:57.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029438345000000001 2024-09-17T22:42:57.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570397+0000", 2024-09-17T22:42:57.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5429000000000002e-05 2024-09-17T22:42:57.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570434+0000", 2024-09-17T22:42:57.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7824999999999998e-05 2024-09-17T22:42:57.737 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.737 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.737 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.737 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187443 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8798208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:57.737 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:57.737 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.120950317000002, 2024-09-17T22:42:57.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029603640000000001, 2024-09-17T22:42:57.738 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.738 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.738 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:57.738 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.738 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:57.738 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:57.738 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187443 2024-09-17T22:42:57.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.738 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:57.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:57.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:42:57.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.05e-07 2024-09-17T22:42:57.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540927+0000", 2024-09-17T22:42:57.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:42:57.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540930+0000", 2024-09-17T22:42:57.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9229999999999998e-06 2024-09-17T22:42:57.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540932+0000", 2024-09-17T22:42:57.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.294e-06 2024-09-17T22:42:57.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570454+0000", 2024-09-17T22:42:57.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029522230999999999 2024-09-17T22:42:57.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570491+0000", 2024-09-17T22:42:57.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6835999999999998e-05 2024-09-17T22:42:57.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570529+0000", 2024-09-17T22:42:57.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8167999999999998e-05 2024-09-17T22:42:57.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.743 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.743 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187444 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8888320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:57.743 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:57.743 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.120940843, 2024-09-17T22:42:57.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029687584999999999, 2024-09-17T22:42:57.743 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:57.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:57.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:57.744 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187444 2024-09-17T22:42:57.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.744 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:57.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:57.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:42:57.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-09-17T22:42:57.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540936+0000", 2024-09-17T22:42:57.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3900000000000003e-07 2024-09-17T22:42:57.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540939+0000", 2024-09-17T22:42:57.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.994e-06 2024-09-17T22:42:57.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540942+0000", 2024-09-17T22:42:57.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1739999999999999e-06 2024-09-17T22:42:57.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570549+0000", 2024-09-17T22:42:57.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029607652000000002 2024-09-17T22:42:57.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570585+0000", 2024-09-17T22:42:57.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5775000000000002e-05 2024-09-17T22:42:57.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570623+0000", 2024-09-17T22:42:57.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7839999999999997e-05 2024-09-17T22:42:57.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.748 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.749 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.749 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187445 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8978432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:57.749 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:57.749 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.120931955000003, 2024-09-17T22:42:57.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029772837999999999, 2024-09-17T22:42:57.749 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.749 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.749 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:57.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:57.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:57.750 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187445 2024-09-17T22:42:57.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.750 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:57.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:57.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:42:57.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:42:57.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540945+0000", 2024-09-17T22:42:57.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3699999999999999e-07 2024-09-17T22:42:57.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540948+0000", 2024-09-17T22:42:57.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.898e-06 2024-09-17T22:42:57.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540950+0000", 2024-09-17T22:42:57.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2249999999999999e-06 2024-09-17T22:42:57.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570643+0000", 2024-09-17T22:42:57.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029692463999999998 2024-09-17T22:42:57.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570680+0000", 2024-09-17T22:42:57.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6772e-05 2024-09-17T22:42:57.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570717+0000", 2024-09-17T22:42:57.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7438999999999998e-05 2024-09-17T22:42:57.754 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.754 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.754 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.754 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187446 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9068544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:57.754 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:57.755 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.120923216999998, 2024-09-17T22:42:57.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029856321000000002, 2024-09-17T22:42:57.755 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.755 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.755 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:57.755 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.755 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:57.755 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:57.755 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187446 2024-09-17T22:42:57.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.755 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:57.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:57.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:42:57.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:42:57.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540958+0000", 2024-09-17T22:42:57.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3449999999999996e-06 2024-09-17T22:42:57.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540962+0000", 2024-09-17T22:42:57.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3529999999999999e-06 2024-09-17T22:42:57.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540964+0000", 2024-09-17T22:42:57.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3810000000000002e-06 2024-09-17T22:42:57.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570737+0000", 2024-09-17T22:42:57.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029772824 2024-09-17T22:42:57.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570770+0000", 2024-09-17T22:42:57.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2835e-05 2024-09-17T22:42:57.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570809+0000", 2024-09-17T22:42:57.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9379999999999999e-05 2024-09-17T22:42:57.759 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.760 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.760 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187447 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9158656~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:57.760 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:57.760 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.120909249, 2024-09-17T22:42:57.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02993262, 2024-09-17T22:42:57.761 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.761 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.761 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:57.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:57.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:57.761 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187447 2024-09-17T22:42:57.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.761 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:57.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:57.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:42:57.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-09-17T22:42:57.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540968+0000", 2024-09-17T22:42:57.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6500000000000008e-07 2024-09-17T22:42:57.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540971+0000", 2024-09-17T22:42:57.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9869999999999999e-06 2024-09-17T22:42:57.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540973+0000", 2024-09-17T22:42:57.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3120000000000001e-06 2024-09-17T22:42:57.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570829+0000", 2024-09-17T22:42:57.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029855742000000001 2024-09-17T22:42:57.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570861+0000", 2024-09-17T22:42:57.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2152e-05 2024-09-17T22:42:57.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570899+0000", 2024-09-17T22:42:57.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8241999999999997e-05 2024-09-17T22:42:57.765 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.765 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.766 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.766 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187448 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9248768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:57.766 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:57.766 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.120900218999999, 2024-09-17T22:42:57.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030014865000000002, 2024-09-17T22:42:57.766 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.766 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.766 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:57.766 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.766 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:57.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:57.767 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187448 2024-09-17T22:42:57.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.767 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:57.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:57.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:42:57.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9600000000000001e-07 2024-09-17T22:42:57.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540977+0000", 2024-09-17T22:42:57.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.16e-07 2024-09-17T22:42:57.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540980+0000", 2024-09-17T22:42:57.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.892e-06 2024-09-17T22:42:57.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540982+0000", 2024-09-17T22:42:57.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3410000000000001e-06 2024-09-17T22:42:57.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570919+0000", 2024-09-17T22:42:57.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029936800999999999 2024-09-17T22:42:57.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570953+0000", 2024-09-17T22:42:57.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3939000000000002e-05 2024-09-17T22:42:57.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570991+0000", 2024-09-17T22:42:57.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7780000000000001e-05 2024-09-17T22:42:57.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.771 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.771 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187449 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9338880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:57.771 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:57.771 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.120891039999997, 2024-09-17T22:42:57.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030110238000000001, 2024-09-17T22:42:57.771 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.772 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.772 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:57.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:57.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:57.772 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187449 2024-09-17T22:42:57.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.772 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:57.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:57.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:42:57.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1500000000000001e-07 2024-09-17T22:42:57.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540986+0000", 2024-09-17T22:42:57.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9500000000000001e-07 2024-09-17T22:42:57.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540989+0000", 2024-09-17T22:42:57.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7949999999999999e-06 2024-09-17T22:42:57.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540994+0000", 2024-09-17T22:42:57.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0359999999999997e-06 2024-09-17T22:42:57.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571011+0000", 2024-09-17T22:42:57.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030016915000000002 2024-09-17T22:42:57.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571056+0000", 2024-09-17T22:42:57.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5457999999999997e-05 2024-09-17T22:42:57.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571095+0000", 2024-09-17T22:42:57.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8924000000000002e-05 2024-09-17T22:42:57.777 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.777 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.777 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.777 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187450 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9428992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:42:57.777 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:57.777 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 49.120879393000003, 2024-09-17T22:42:57.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030188204999999999, 2024-09-17T22:42:57.777 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:57.777 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:57.777 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:57.778 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:57.778 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:57.778 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:57.778 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187450 2024-09-17T22:42:57.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.778 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:57.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:57.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:57.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:57.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:57.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:57.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:57.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:42:57.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.16e-07 2024-09-17T22:42:57.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:57.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540998+0000", 2024-09-17T22:42:57.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9000000000000005e-07 2024-09-17T22:42:57.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:57.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.541001+0000", 2024-09-17T22:42:57.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0680000000000001e-06 2024-09-17T22:42:57.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:57.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.541006+0000", 2024-09-17T22:42:57.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4720000000000003e-06 2024-09-17T22:42:57.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:57.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571110+0000", 2024-09-17T22:42:57.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030103431 2024-09-17T22:42:57.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:57.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571147+0000", 2024-09-17T22:42:57.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6748999999999997e-05 2024-09-17T22:42:57.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:57.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:57.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:57.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571185+0000", 2024-09-17T22:42:57.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8278999999999997e-05 2024-09-17T22:42:57.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.782 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:57.782 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:57.782 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:57.784 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:42:57.785 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:57.781+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:57.884 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:57.882+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:57.927 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:57.927 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:57.928 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:57.928 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:57.936 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:42:57.985 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:57.982+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:58.000 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is set 2024-09-17T22:42:58.047 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:58.048 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:58.048 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:58.048 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:58.048 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:58.056 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:42:58.086 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:58.084+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:58.187 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:58.185+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:58.201 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:58.201 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:58.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:58.202 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:58.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.202 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145246 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3850240~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.202 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:58.202 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.868338092000002, 2024-09-17T22:42:58.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052644991000000002, 2024-09-17T22:42:58.202 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.202 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.202 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.202 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.202 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.202 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.203 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145246 2024-09-17T22:42:58.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.203 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:58.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:42:58.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329899+0000", 2024-09-17T22:42:58.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-09-17T22:42:58.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329901+0000", 2024-09-17T22:42:58.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1619999999999998e-06 2024-09-17T22:42:58.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329907+0000", 2024-09-17T22:42:58.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9059999999999999e-06 2024-09-17T22:42:58.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329911+0000", 2024-09-17T22:42:58.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7419999999999997e-06 2024-09-17T22:42:58.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382383+0000", 2024-09-17T22:42:58.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052471583000000002 2024-09-17T22:42:58.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382457+0000", 2024-09-17T22:42:58.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4309999999999998e-05 2024-09-17T22:42:58.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382543+0000", 2024-09-17T22:42:58.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5918e-05 2024-09-17T22:42:58.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.206 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.206 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145247 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3946496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.207 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:58.207 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.868319442000001, 2024-09-17T22:42:58.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052805863000000001, 2024-09-17T22:42:58.207 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.207 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.207 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.207 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145247 2024-09-17T22:42:58.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.207 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:58.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:58.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:42:58.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3200000000000001e-07 2024-09-17T22:42:58.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329919+0000", 2024-09-17T22:42:58.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.888e-06 2024-09-17T22:42:58.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329925+0000", 2024-09-17T22:42:58.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5369999999999997e-06 2024-09-17T22:42:58.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329929+0000", 2024-09-17T22:42:58.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.639e-06 2024-09-17T22:42:58.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382570+0000", 2024-09-17T22:42:58.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052641213999999999 2024-09-17T22:42:58.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382636+0000", 2024-09-17T22:42:58.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6008999999999998e-05 2024-09-17T22:42:58.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382723+0000", 2024-09-17T22:42:58.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6243999999999996e-05 2024-09-17T22:42:58.211 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.211 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.211 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.211 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145248 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3856384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.211 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:58.211 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.868301394, 2024-09-17T22:42:58.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053002858, 2024-09-17T22:42:58.212 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.212 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.212 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.212 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.212 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.212 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.212 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145248 2024-09-17T22:42:58.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.212 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:58.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:58.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:42:58.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.13e-07 2024-09-17T22:42:58.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329937+0000", 2024-09-17T22:42:58.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9300000000000002e-06 2024-09-17T22:42:58.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329943+0000", 2024-09-17T22:42:58.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-09-17T22:42:58.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329948+0000", 2024-09-17T22:42:58.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8579999999999999e-06 2024-09-17T22:42:58.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382763+0000", 2024-09-17T22:42:58.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052814537000000002 2024-09-17T22:42:58.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382840+0000", 2024-09-17T22:42:58.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6879999999999996e-05 2024-09-17T22:42:58.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382938+0000", 2024-09-17T22:42:58.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8159999999999995e-05 2024-09-17T22:42:58.216 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.216 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.216 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.216 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145249 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3952640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.216 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:58.216 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.868282492999995, 2024-09-17T22:42:58.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053188602000000001, 2024-09-17T22:42:58.217 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.217 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.217 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.217 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.217 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.217 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.217 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145249 2024-09-17T22:42:58.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.217 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:58.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:58.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:42:58.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3700000000000001e-07 2024-09-17T22:42:58.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329956+0000", 2024-09-17T22:42:58.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1430000000000001e-06 2024-09-17T22:42:58.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329962+0000", 2024-09-17T22:42:58.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8270000000000003e-06 2024-09-17T22:42:58.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329967+0000", 2024-09-17T22:42:58.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7249999999999997e-06 2024-09-17T22:42:58.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382969+0000", 2024-09-17T22:42:58.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053001952999999997 2024-09-17T22:42:58.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383040+0000", 2024-09-17T22:42:58.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1272999999999993e-05 2024-09-17T22:42:58.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383142+0000", 2024-09-17T22:42:58.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010234399999999999 2024-09-17T22:42:58.220 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.221 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.221 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.221 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145250 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3862528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.221 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:58.221 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.868263694000007, 2024-09-17T22:42:58.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053359537999999998, 2024-09-17T22:42:58.221 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.221 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.221 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.222 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.222 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145250 2024-09-17T22:42:58.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.222 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:58.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:58.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:42:58.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:42:58.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329975+0000", 2024-09-17T22:42:58.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2199999999999999e-06 2024-09-17T22:42:58.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329981+0000", 2024-09-17T22:42:58.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1020000000000002e-06 2024-09-17T22:42:58.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329991+0000", 2024-09-17T22:42:58.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6800000000000005e-06 2024-09-17T22:42:58.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383171+0000", 2024-09-17T22:42:58.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053180142 2024-09-17T22:42:58.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383242+0000", 2024-09-17T22:42:58.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1310000000000007e-05 2024-09-17T22:42:58.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383332+0000", 2024-09-17T22:42:58.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9703999999999994e-05 2024-09-17T22:42:58.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.225 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.225 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145251 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3958784~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.225 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:58.225 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.868237993999998, 2024-09-17T22:42:58.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053438537000000001, 2024-09-17T22:42:58.226 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.226 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.226 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.226 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145251 2024-09-17T22:42:58.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.226 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:58.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:42:58.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329999+0000", 2024-09-17T22:42:58.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-07 2024-09-17T22:42:58.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330001+0000", 2024-09-17T22:42:58.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:42:58.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330006+0000", 2024-09-17T22:42:58.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000004e-06 2024-09-17T22:42:58.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330011+0000", 2024-09-17T22:42:58.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4039999999999997e-06 2024-09-17T22:42:58.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383350+0000", 2024-09-17T22:42:58.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053339011999999998 2024-09-17T22:42:58.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383387+0000", 2024-09-17T22:42:58.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7085999999999997e-05 2024-09-17T22:42:58.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383437+0000", 2024-09-17T22:42:58.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9988000000000003e-05 2024-09-17T22:42:58.229 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.229 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.229 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.230 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145252 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3868672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.230 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:58.230 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.868219546999995, 2024-09-17T22:42:58.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053526724999999997, 2024-09-17T22:42:58.230 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.230 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.230 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.230 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.230 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.230 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.230 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145252 2024-09-17T22:42:58.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.230 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:58.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:42:58.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331054+0000", 2024-09-17T22:42:58.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010371670000000001 2024-09-17T22:42:58.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331059+0000", 2024-09-17T22:42:58.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1680000000000003e-06 2024-09-17T22:42:58.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331075+0000", 2024-09-17T22:42:58.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6028e-05 2024-09-17T22:42:58.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331084+0000", 2024-09-17T22:42:58.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3980000000000005e-06 2024-09-17T22:42:58.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383464+0000", 2024-09-17T22:42:58.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052379851999999998 2024-09-17T22:42:58.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383499+0000", 2024-09-17T22:42:58.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4610000000000002e-05 2024-09-17T22:42:58.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383543+0000", 2024-09-17T22:42:58.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4502000000000002e-05 2024-09-17T22:42:58.234 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.234 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.234 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.234 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145255 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3971072~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.234 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:58.234 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.867089454999999, 2024-09-17T22:42:58.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052677527000000002, 2024-09-17T22:42:58.235 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.235 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.235 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.235 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145255 2024-09-17T22:42:58.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.235 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:58.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:58.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:42:58.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8800000000000002e-07 2024-09-17T22:42:58.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331150+0000", 2024-09-17T22:42:58.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1160000000000002e-06 2024-09-17T22:42:58.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331156+0000", 2024-09-17T22:42:58.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7469999999999999e-06 2024-09-17T22:42:58.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331162+0000", 2024-09-17T22:42:58.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-17T22:42:58.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383750+0000", 2024-09-17T22:42:58.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052588441999999999 2024-09-17T22:42:58.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383782+0000", 2024-09-17T22:42:58.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1610999999999999e-05 2024-09-17T22:42:58.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383824+0000", 2024-09-17T22:42:58.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2512000000000002e-05 2024-09-17T22:42:58.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.239 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.239 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145256 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3880960~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.239 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:58.239 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.867068033999999, 2024-09-17T22:42:58.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052744775000000001, 2024-09-17T22:42:58.239 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.240 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.240 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.240 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145256 2024-09-17T22:42:58.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.240 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:58.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:42:58.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331169+0000", 2024-09-17T22:42:58.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:42:58.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331175+0000", 2024-09-17T22:42:58.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5660000000000003e-06 2024-09-17T22:42:58.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331182+0000", 2024-09-17T22:42:58.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0169999999999999e-06 2024-09-17T22:42:58.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331188+0000", 2024-09-17T22:42:58.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2750000000000003e-06 2024-09-17T22:42:58.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383839+0000", 2024-09-17T22:42:58.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052651728000000002 2024-09-17T22:42:58.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383872+0000", 2024-09-17T22:42:58.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2347e-05 2024-09-17T22:42:58.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383913+0000", 2024-09-17T22:42:58.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1362000000000001e-05 2024-09-17T22:42:58.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.244 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.244 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145257 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3977216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.244 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:58.244 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.867041986000004, 2024-09-17T22:42:58.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052809598999999999, 2024-09-17T22:42:58.244 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.245 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.245 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.245 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145257 2024-09-17T22:42:58.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.245 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:58.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:42:58.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333051+0000", 2024-09-17T22:42:58.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018567189999999999 2024-09-17T22:42:58.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333056+0000", 2024-09-17T22:42:58.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-17T22:42:58.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333067+0000", 2024-09-17T22:42:58.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1704e-05 2024-09-17T22:42:58.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333074+0000", 2024-09-17T22:42:58.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1280000000000002e-06 2024-09-17T22:42:58.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383927+0000", 2024-09-17T22:42:58.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050853094000000001 2024-09-17T22:42:58.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383960+0000", 2024-09-17T22:42:58.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2891000000000001e-05 2024-09-17T22:42:58.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.384004+0000", 2024-09-17T22:42:58.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3565e-05 2024-09-17T22:42:58.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.249 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.249 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146168 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2467840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.249 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:58.249 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.803553364999999, 2024-09-17T22:42:58.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052652501999999997, 2024-09-17T22:42:58.249 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.249 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.250 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.250 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146168 2024-09-17T22:42:58.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.250 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:58.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:42:58.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394989+0000", 2024-09-17T22:42:58.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030618300000000002 2024-09-17T22:42:58.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394994+0000", 2024-09-17T22:42:58.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.566e-06 2024-09-17T22:42:58.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395003+0000", 2024-09-17T22:42:58.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3619999999999995e-06 2024-09-17T22:42:58.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395009+0000", 2024-09-17T22:42:58.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3589999999999996e-06 2024-09-17T22:42:58.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447180+0000", 2024-09-17T22:42:58.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052170643000000003 2024-09-17T22:42:58.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447251+0000", 2024-09-17T22:42:58.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1382000000000002e-05 2024-09-17T22:42:58.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447335+0000", 2024-09-17T22:42:58.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4006999999999997e-05 2024-09-17T22:42:58.254 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.254 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.254 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.254 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146172 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2480128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.254 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:58.254 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.803198480000006, 2024-09-17T22:42:58.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052714629999999998, 2024-09-17T22:42:58.254 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.254 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.255 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.255 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.255 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.255 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.255 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146172 2024-09-17T22:42:58.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.255 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:58.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:42:58.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395486+0000", 2024-09-17T22:42:58.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044787000000000001 2024-09-17T22:42:58.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395491+0000", 2024-09-17T22:42:58.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8550000000000001e-06 2024-09-17T22:42:58.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395503+0000", 2024-09-17T22:42:58.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2271e-05 2024-09-17T22:42:58.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395510+0000", 2024-09-17T22:42:58.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6320000000000008e-06 2024-09-17T22:42:58.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447565+0000", 2024-09-17T22:42:58.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052055016000000003 2024-09-17T22:42:58.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447644+0000", 2024-09-17T22:42:58.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8306000000000005e-05 2024-09-17T22:42:58.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447752+0000", 2024-09-17T22:42:58.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010868 2024-09-17T22:42:58.259 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.259 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.259 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.259 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146181 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2504704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.259 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:58.259 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.802331089999996, 2024-09-17T22:42:58.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052850394000000002, 2024-09-17T22:42:58.260 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.260 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.260 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.260 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.260 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.260 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.260 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146181 2024-09-17T22:42:58.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.260 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:58.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:42:58.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396210+0000", 2024-09-17T22:42:58.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000305214 2024-09-17T22:42:58.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396214+0000", 2024-09-17T22:42:58.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9389999999999999e-06 2024-09-17T22:42:58.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396228+0000", 2024-09-17T22:42:58.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3365e-05 2024-09-17T22:42:58.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396235+0000", 2024-09-17T22:42:58.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2559999999999996e-06 2024-09-17T22:42:58.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448568+0000", 2024-09-17T22:42:58.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052333084000000002 2024-09-17T22:42:58.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448646+0000", 2024-09-17T22:42:58.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8287000000000004e-05 2024-09-17T22:42:58.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448756+0000", 2024-09-17T22:42:58.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109249 2024-09-17T22:42:58.264 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.264 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.264 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.264 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146183 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2510848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.265 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:58.265 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.801990543000002, 2024-09-17T22:42:58.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052725453999999998, 2024-09-17T22:42:58.265 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.265 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.265 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.265 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.265 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.265 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.265 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146183 2024-09-17T22:42:58.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.265 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:58.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:42:58.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396508+0000", 2024-09-17T22:42:58.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026271299999999998 2024-09-17T22:42:58.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396512+0000", 2024-09-17T22:42:58.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8039999999999999e-06 2024-09-17T22:42:58.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396522+0000", 2024-09-17T22:42:58.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9289999999999993e-06 2024-09-17T22:42:58.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396528+0000", 2024-09-17T22:42:58.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2990000000000002e-06 2024-09-17T22:42:58.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448798+0000", 2024-09-17T22:42:58.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052269985999999997 2024-09-17T22:42:58.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448871+0000", 2024-09-17T22:42:58.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2581000000000002e-05 2024-09-17T22:42:58.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448971+0000", 2024-09-17T22:42:58.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100142 2024-09-17T22:42:58.269 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.269 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.269 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.269 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146391 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.269 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:58.269 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.794263051000001, 2024-09-17T22:42:58.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052666719000000001, 2024-09-17T22:42:58.269 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.269 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.269 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.270 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.270 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.270 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.270 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146391 2024-09-17T22:42:58.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.270 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:58.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:42:58.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404098+0000", 2024-09-17T22:42:58.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000124509 2024-09-17T22:42:58.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404107+0000", 2024-09-17T22:42:58.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4220000000000006e-06 2024-09-17T22:42:58.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404117+0000", 2024-09-17T22:42:58.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011e-05 2024-09-17T22:42:58.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404123+0000", 2024-09-17T22:42:58.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1530000000000002e-06 2024-09-17T22:42:58.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456481+0000", 2024-09-17T22:42:58.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052357923000000001 2024-09-17T22:42:58.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456551+0000", 2024-09-17T22:42:58.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9869000000000005e-05 2024-09-17T22:42:58.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456640+0000", 2024-09-17T22:42:58.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8733000000000006e-05 2024-09-17T22:42:58.273 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.273 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.273 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.274 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146393 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2748416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.274 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:58.274 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.794102988000006, 2024-09-17T22:42:58.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052739239, 2024-09-17T22:42:58.274 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.274 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.274 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.274 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.274 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.274 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.274 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146393 2024-09-17T22:42:58.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.275 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:58.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:42:58.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404443+0000", 2024-09-17T22:42:58.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030977199999999999 2024-09-17T22:42:58.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404447+0000", 2024-09-17T22:42:58.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6370000000000001e-06 2024-09-17T22:42:58.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404469+0000", 2024-09-17T22:42:58.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2739999999999999e-05 2024-09-17T22:42:58.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404477+0000", 2024-09-17T22:42:58.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6559999999999992e-06 2024-09-17T22:42:58.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456734+0000", 2024-09-17T22:42:58.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052256889000000001 2024-09-17T22:42:58.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456796+0000", 2024-09-17T22:42:58.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2485999999999998e-05 2024-09-17T22:42:58.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456873+0000", 2024-09-17T22:42:58.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6058999999999998e-05 2024-09-17T22:42:58.278 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.278 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.278 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.278 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146397 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2760704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.278 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:58.279 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.793727223999994, 2024-09-17T22:42:58.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052788928999999998, 2024-09-17T22:42:58.279 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.279 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.279 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.279 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.279 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.279 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.279 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146397 2024-09-17T22:42:58.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.279 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:58.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:42:58.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404763+0000", 2024-09-17T22:42:58.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025351399999999999 2024-09-17T22:42:58.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404767+0000", 2024-09-17T22:42:58.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:42:58.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404777+0000", 2024-09-17T22:42:58.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9310000000000003e-06 2024-09-17T22:42:58.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404783+0000", 2024-09-17T22:42:58.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.336e-06 2024-09-17T22:42:58.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457126+0000", 2024-09-17T22:42:58.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052342975999999999 2024-09-17T22:42:58.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457200+0000", 2024-09-17T22:42:58.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4146000000000005e-05 2024-09-17T22:42:58.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457298+0000", 2024-09-17T22:42:58.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7807999999999996e-05 2024-09-17T22:42:58.283 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.283 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.283 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.283 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146399 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2766848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.283 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:58.284 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.793437083000001, 2024-09-17T22:42:58.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052829779, 2024-09-17T22:42:58.284 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.284 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.284 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.284 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.284 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.284 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.284 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146399 2024-09-17T22:42:58.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.284 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:58.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:42:58.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404800+0000", 2024-09-17T22:42:58.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8500000000000001e-07 2024-09-17T22:42:58.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404802+0000", 2024-09-17T22:42:58.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5349999999999999e-06 2024-09-17T22:42:58.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404811+0000", 2024-09-17T22:42:58.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9390000000000003e-06 2024-09-17T22:42:58.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404817+0000", 2024-09-17T22:42:58.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.524e-06 2024-09-17T22:42:58.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457461+0000", 2024-09-17T22:42:58.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052643784999999998 2024-09-17T22:42:58.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457539+0000", 2024-09-17T22:42:58.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8589000000000003e-05 2024-09-17T22:42:58.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457629+0000", 2024-09-17T22:42:58.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9821999999999995e-05 2024-09-17T22:42:58.288 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.288 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.288 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.288 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146401 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2772992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.288 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:58.288 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.793411836000004, 2024-09-17T22:42:58.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052910077, 2024-09-17T22:42:58.289 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.289 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.289 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.289 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146401 2024-09-17T22:42:58.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.289 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:58.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:42:58.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405106+0000", 2024-09-17T22:42:58.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000281286 2024-09-17T22:42:58.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405110+0000", 2024-09-17T22:42:58.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1459999999999998e-06 2024-09-17T22:42:58.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405120+0000", 2024-09-17T22:42:58.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0546e-05 2024-09-17T22:42:58.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405127+0000", 2024-09-17T22:42:58.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4300000000000003e-06 2024-09-17T22:42:58.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457647+0000", 2024-09-17T22:42:58.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052520229000000002 2024-09-17T22:42:58.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457684+0000", 2024-09-17T22:42:58.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6609999999999997e-05 2024-09-17T22:42:58.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457735+0000", 2024-09-17T22:42:58.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.083e-05 2024-09-17T22:42:58.293 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.293 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.293 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.293 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146402 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2779136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.293 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:58.293 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.793098423999993, 2024-09-17T22:42:58.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052689812000000003, 2024-09-17T22:42:58.293 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.293 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.293 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.294 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.294 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.294 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.294 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146402 2024-09-17T22:42:58.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.294 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:58.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:42:58.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405365+0000", 2024-09-17T22:42:58.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022744599999999999 2024-09-17T22:42:58.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405370+0000", 2024-09-17T22:42:58.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2769999999999999e-06 2024-09-17T22:42:58.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405380+0000", 2024-09-17T22:42:58.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0349e-05 2024-09-17T22:42:58.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405386+0000", 2024-09-17T22:42:58.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5169999999999996e-06 2024-09-17T22:42:58.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457750+0000", 2024-09-17T22:42:58.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052363448999999999 2024-09-17T22:42:58.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457784+0000", 2024-09-17T22:42:58.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4363999999999998e-05 2024-09-17T22:42:58.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457828+0000", 2024-09-17T22:42:58.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999999e-05 2024-09-17T22:42:58.297 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.297 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.298 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.298 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.298 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.298 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:58.299 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:42:58.300 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:58.297+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:58.400 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:58.398+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:58.403 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:58.403 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:58.403 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:42:58.403 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:58.412 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:42:58.501 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:58.499+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:58.514 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:58.514 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:42:58.514 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:42:58.514 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:42:58.514 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:58.523 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:42:58.602 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:58.600+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:58.627 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:58.627 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:42:58.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:42:58.628 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:42:58.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.628 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4229.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-17T22:42:58.628 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:58.628 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 109.229839703, 2024-09-17T22:42:58.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25708877200000002, 2024-09-17T22:42:58.628 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.628 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.628 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-17T22:42:58.628 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4229", 2024-09-17T22:42:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/3080427038", 2024-09-17T22:42:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2 2024-09-17T22:42:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:42:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394611+0000", 2024-09-17T22:42:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3149999999999999e-06 2024-09-17T22:42:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394619+0000", 2024-09-17T22:42:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240000000000003e-06 2024-09-17T22:42:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394632+0000", 2024-09-17T22:42:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2921e-05 2024-09-17T22:42:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394639+0000", 2024-09-17T22:42:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-09-17T22:42:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394703+0000", 2024-09-17T22:42:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4407999999999997e-05 2024-09-17T22:42:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:42:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394711+0000", 2024-09-17T22:42:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5939999999999996e-06 2024-09-17T22:42:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651545+0000", 2024-09-17T22:42:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25683409200000001 2024-09-17T22:42:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651698+0000", 2024-09-17T22:42:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000153678 2024-09-17T22:42:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38736 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.847243698, 2024-09-17T22:42:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051395418999999998, 2024-09-17T22:42:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38736 2024-09-17T22:42:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:42:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6899999999999998e-07 2024-09-17T22:42:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777212+0000", 2024-09-17T22:42:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1240000000000002e-06 2024-09-17T22:42:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777219+0000", 2024-09-17T22:42:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5409999999999997e-06 2024-09-17T22:42:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777224+0000", 2024-09-17T22:42:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:42:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.826131+0000", 2024-09-17T22:42:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048907369999999999 2024-09-17T22:42:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828457+0000", 2024-09-17T22:42:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023256209999999999 2024-09-17T22:42:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828601+0000", 2024-09-17T22:42:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014438799999999999 2024-09-17T22:42:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38752 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 673792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.847219627000001, 2024-09-17T22:42:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051582301999999997, 2024-09-17T22:42:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38752 2024-09-17T22:42:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:42:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3000000000000001e-07 2024-09-17T22:42:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777232+0000", 2024-09-17T22:42:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9709999999999998e-06 2024-09-17T22:42:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777238+0000", 2024-09-17T22:42:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7209999999999999e-06 2024-09-17T22:42:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777243+0000", 2024-09-17T22:42:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5210000000000004e-06 2024-09-17T22:42:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828633+0000", 2024-09-17T22:42:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051390915000000002 2024-09-17T22:42:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828711+0000", 2024-09-17T22:42:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7474999999999999e-05 2024-09-17T22:42:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828812+0000", 2024-09-17T22:42:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010116900000000001 2024-09-17T22:42:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.643 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.643 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.643 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38753 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:58.643 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:58.643 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.847200353999995, 2024-09-17T22:42:58.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051770444999999998, 2024-09-17T22:42:58.643 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.643 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.643 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38753 2024-09-17T22:42:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:42:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:42:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777251+0000", 2024-09-17T22:42:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8059999999999999e-06 2024-09-17T22:42:58.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777257+0000", 2024-09-17T22:42:58.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2480000000000001e-06 2024-09-17T22:42:58.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777262+0000", 2024-09-17T22:42:58.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000004e-06 2024-09-17T22:42:58.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828842+0000", 2024-09-17T22:42:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051580203999999998 2024-09-17T22:42:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828917+0000", 2024-09-17T22:42:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4980000000000004e-05 2024-09-17T22:42:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829019+0000", 2024-09-17T22:42:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102248 2024-09-17T22:42:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38766 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.847181368999998, 2024-09-17T22:42:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051958839999999999, 2024-09-17T22:42:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38766 2024-09-17T22:42:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:42:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777269+0000", 2024-09-17T22:42:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:42:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777271+0000", 2024-09-17T22:42:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-17T22:42:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777276+0000", 2024-09-17T22:42:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:42:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777281+0000", 2024-09-17T22:42:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:42:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829055+0000", 2024-09-17T22:42:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051773570999999997 2024-09-17T22:42:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829128+0000", 2024-09-17T22:42:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3131000000000002e-05 2024-09-17T22:42:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829227+0000", 2024-09-17T22:42:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8800000000000003e-05 2024-09-17T22:42:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38767 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.847161385999996, 2024-09-17T22:42:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137018, 2024-09-17T22:42:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38767 2024-09-17T22:42:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:42:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:42:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777290+0000", 2024-09-17T22:42:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.88e-06 2024-09-17T22:42:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777296+0000", 2024-09-17T22:42:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-09-17T22:42:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777300+0000", 2024-09-17T22:42:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4510000000000002e-06 2024-09-17T22:42:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829257+0000", 2024-09-17T22:42:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051956975000000002 2024-09-17T22:42:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829328+0000", 2024-09-17T22:42:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1026999999999996e-05 2024-09-17T22:42:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829425+0000", 2024-09-17T22:42:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.679e-05 2024-09-17T22:42:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38784 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 675840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.847142931999997, 2024-09-17T22:42:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052344522999999997, 2024-09-17T22:42:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38784 2024-09-17T22:42:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:42:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777307+0000", 2024-09-17T22:42:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:42:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777309+0000", 2024-09-17T22:42:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.063e-06 2024-09-17T22:42:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777314+0000", 2024-09-17T22:42:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-17T22:42:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777319+0000", 2024-09-17T22:42:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999996e-06 2024-09-17T22:42:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829456+0000", 2024-09-17T22:42:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137151 2024-09-17T22:42:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829527+0000", 2024-09-17T22:42:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1588000000000006e-05 2024-09-17T22:42:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829651+0000", 2024-09-17T22:42:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012365 2024-09-17T22:42:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38785 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 749568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.847124816999994, 2024-09-17T22:42:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052546572999999999, 2024-09-17T22:42:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38785 2024-09-17T22:42:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:42:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:42:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777327+0000", 2024-09-17T22:42:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:42:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777333+0000", 2024-09-17T22:42:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7309999999999998e-06 2024-09-17T22:42:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777337+0000", 2024-09-17T22:42:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:42:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829686+0000", 2024-09-17T22:42:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052349159999999999 2024-09-17T22:42:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829766+0000", 2024-09-17T22:42:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9326999999999995e-05 2024-09-17T22:42:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829871+0000", 2024-09-17T22:42:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000105466 2024-09-17T22:42:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38798 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.846491209000007, 2024-09-17T22:42:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052117564999999998, 2024-09-17T22:42:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38798 2024-09-17T22:42:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:42:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777959+0000", 2024-09-17T22:42:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9599999999999999e-07 2024-09-17T22:42:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777963+0000", 2024-09-17T22:42:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2660000000000003e-06 2024-09-17T22:42:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777971+0000", 2024-09-17T22:42:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3489999999999992e-06 2024-09-17T22:42:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777978+0000", 2024-09-17T22:42:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:42:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829906+0000", 2024-09-17T22:42:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051927635999999999 2024-09-17T22:42:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829986+0000", 2024-09-17T22:42:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0444000000000004e-05 2024-09-17T22:42:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830076+0000", 2024-09-17T22:42:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9473000000000002e-05 2024-09-17T22:42:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38799 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 882688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.846459769999996, 2024-09-17T22:42:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052359606000000003, 2024-09-17T22:42:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38799 2024-09-17T22:42:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:42:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8299999999999997e-07 2024-09-17T22:42:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777992+0000", 2024-09-17T22:42:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.176e-06 2024-09-17T22:42:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777998+0000", 2024-09-17T22:42:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.173e-06 2024-09-17T22:42:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778003+0000", 2024-09-17T22:42:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6349999999999997e-06 2024-09-17T22:42:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830112+0000", 2024-09-17T22:42:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052109366999999997 2024-09-17T22:42:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830192+0000", 2024-09-17T22:42:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9814000000000005e-05 2024-09-17T22:42:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830349+0000", 2024-09-17T22:42:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015695800000000001 2024-09-17T22:42:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38814 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.846176365999995, 2024-09-17T22:42:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052274339000000003, 2024-09-17T22:42:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38814 2024-09-17T22:42:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:42:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:42:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778276+0000", 2024-09-17T22:42:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2029999999999999e-06 2024-09-17T22:42:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778282+0000", 2024-09-17T22:42:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0059999999999998e-06 2024-09-17T22:42:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778287+0000", 2024-09-17T22:42:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0039999999999999e-06 2024-09-17T22:42:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830382+0000", 2024-09-17T22:42:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052095379999999997 2024-09-17T22:42:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830455+0000", 2024-09-17T22:42:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3127000000000006e-05 2024-09-17T22:42:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830547+0000", 2024-09-17T22:42:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2207999999999995e-05 2024-09-17T22:42:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38828 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 163840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.845993957999994, 2024-09-17T22:42:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052466039999999999, 2024-09-17T22:42:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38828 2024-09-17T22:42:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:42:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778456+0000", 2024-09-17T22:42:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7599999999999996e-07 2024-09-17T22:42:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778460+0000", 2024-09-17T22:42:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8229999999999997e-06 2024-09-17T22:42:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778470+0000", 2024-09-17T22:42:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8199999999999992e-06 2024-09-17T22:42:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778476+0000", 2024-09-17T22:42:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2589999999999997e-06 2024-09-17T22:42:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830578+0000", 2024-09-17T22:42:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052102273999999997 2024-09-17T22:42:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830667+0000", 2024-09-17T22:42:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8701000000000003e-05 2024-09-17T22:42:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830921+0000", 2024-09-17T22:42:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025438700000000002 2024-09-17T22:42:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38829 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.845962560999993, 2024-09-17T22:42:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052648882000000001, 2024-09-17T22:42:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38829 2024-09-17T22:42:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:42:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7e-07 2024-09-17T22:42:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778490+0000", 2024-09-17T22:42:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:42:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778495+0000", 2024-09-17T22:42:58.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9329999999999998e-06 2024-09-17T22:42:58.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778500+0000", 2024-09-17T22:42:58.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9509999999999997e-06 2024-09-17T22:42:58.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830958+0000", 2024-09-17T22:42:58.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05245785 2024-09-17T22:42:58.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831051+0000", 2024-09-17T22:42:58.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2989e-05 2024-09-17T22:42:58.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831136+0000", 2024-09-17T22:42:58.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4448000000000002e-05 2024-09-17T22:42:58.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.691 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.692 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38891 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:58.692 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:58.692 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.842650503000002, 2024-09-17T22:42:58.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050847934999999997, 2024-09-17T22:42:58.692 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.692 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.692 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.692 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38891 2024-09-17T22:42:58.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.692 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:58.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:58.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:42:58.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-09-17T22:42:58.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781802+0000", 2024-09-17T22:42:58.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:42:58.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781810+0000", 2024-09-17T22:42:58.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8019999999999992e-06 2024-09-17T22:42:58.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781816+0000", 2024-09-17T22:42:58.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.285e-06 2024-09-17T22:42:58.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832485+0000", 2024-09-17T22:42:58.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050668462999999997 2024-09-17T22:42:58.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832565+0000", 2024-09-17T22:42:58.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9692999999999997e-05 2024-09-17T22:42:58.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832647+0000", 2024-09-17T22:42:58.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2224000000000004e-05 2024-09-17T22:42:58.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.696 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.696 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38892 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:58.696 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:58.696 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.842622254000005, 2024-09-17T22:42:58.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050946120999999997, 2024-09-17T22:42:58.697 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.697 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.697 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.697 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38892 2024-09-17T22:42:58.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.697 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:58.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:42:58.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781828+0000", 2024-09-17T22:42:58.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5400000000000002e-07 2024-09-17T22:42:58.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781830+0000", 2024-09-17T22:42:58.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1959999999999998e-06 2024-09-17T22:42:58.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781837+0000", 2024-09-17T22:42:58.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8310000000000003e-06 2024-09-17T22:42:58.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781843+0000", 2024-09-17T22:42:58.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1e-06 2024-09-17T22:42:58.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832674+0000", 2024-09-17T22:42:58.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050831182000000003 2024-09-17T22:42:58.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832725+0000", 2024-09-17T22:42:58.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1588000000000001e-05 2024-09-17T22:42:58.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832773+0000", 2024-09-17T22:42:58.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7769999999999998e-05 2024-09-17T22:42:58.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.700 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.701 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38909 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 36864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:58.701 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:58.701 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.842599484000004, 2024-09-17T22:42:58.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051046518999999999, 2024-09-17T22:42:58.701 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.701 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.701 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.702 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38909 2024-09-17T22:42:58.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.702 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:58.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:58.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:42:58.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:42:58.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781852+0000", 2024-09-17T22:42:58.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0590000000000001e-06 2024-09-17T22:42:58.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781859+0000", 2024-09-17T22:42:58.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3380000000000001e-06 2024-09-17T22:42:58.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781864+0000", 2024-09-17T22:42:58.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:42:58.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832804+0000", 2024-09-17T22:42:58.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050939614000000001 2024-09-17T22:42:58.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832845+0000", 2024-09-17T22:42:58.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1014999999999999e-05 2024-09-17T22:42:58.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832896+0000", 2024-09-17T22:42:58.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1836999999999999e-05 2024-09-17T22:42:58.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.705 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.705 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38910 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:58.705 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:58.706 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.842579173999994, 2024-09-17T22:42:58.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051216818999999997, 2024-09-17T22:42:58.706 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.706 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.706 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.706 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38910 2024-09-17T22:42:58.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.706 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:58.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:42:58.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781871+0000", 2024-09-17T22:42:58.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:42:58.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781873+0000", 2024-09-17T22:42:58.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:42:58.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781879+0000", 2024-09-17T22:42:58.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0700000000000003e-06 2024-09-17T22:42:58.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781884+0000", 2024-09-17T22:42:58.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-09-17T22:42:58.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832932+0000", 2024-09-17T22:42:58.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051048219999999998 2024-09-17T22:42:58.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833001+0000", 2024-09-17T22:42:58.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8504999999999995e-05 2024-09-17T22:42:58.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833087+0000", 2024-09-17T22:42:58.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6464000000000004e-05 2024-09-17T22:42:58.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.710 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.710 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38923 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:58.710 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:58.710 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.842558854000004, 2024-09-17T22:42:58.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051379573999999997, 2024-09-17T22:42:58.710 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.710 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.710 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.710 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.711 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38923 2024-09-17T22:42:58.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.711 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:58.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:58.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:42:58.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:42:58.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781893+0000", 2024-09-17T22:42:58.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-09-17T22:42:58.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781900+0000", 2024-09-17T22:42:58.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3860000000000004e-06 2024-09-17T22:42:58.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781904+0000", 2024-09-17T22:42:58.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9300000000000002e-06 2024-09-17T22:42:58.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833115+0000", 2024-09-17T22:42:58.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051210845999999997 2024-09-17T22:42:58.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833188+0000", 2024-09-17T22:42:58.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2276999999999999e-05 2024-09-17T22:42:58.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833270+0000", 2024-09-17T22:42:58.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2501000000000002e-05 2024-09-17T22:42:58.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.715 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.715 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38941 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:42:58.715 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:58.715 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.841766319000001, 2024-09-17T22:42:58.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050969459000000002, 2024-09-17T22:42:58.716 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.716 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.716 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.716 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38941 2024-09-17T22:42:58.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.716 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:58.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:42:58.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782684+0000", 2024-09-17T22:42:58.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:42:58.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782686+0000", 2024-09-17T22:42:58.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.396e-06 2024-09-17T22:42:58.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782693+0000", 2024-09-17T22:42:58.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1999999999999997e-06 2024-09-17T22:42:58.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782699+0000", 2024-09-17T22:42:58.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7910000000000001e-06 2024-09-17T22:42:58.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833471+0000", 2024-09-17T22:42:58.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050772333000000003 2024-09-17T22:42:58.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833536+0000", 2024-09-17T22:42:58.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4380000000000004e-05 2024-09-17T22:42:58.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833653+0000", 2024-09-17T22:42:58.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011691799999999999 2024-09-17T22:42:58.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.720 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.720 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:151189 3.14 3:2b696b13:::rbd_data.115f14f2d1d5.0000000000000009:e [read 3127296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:58.720 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:58.720 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.911545574000002, 2024-09-17T22:42:58.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.054254700000000003, 2024-09-17T22:42:58.720 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:42:58.721 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:42:58.721 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:42:58.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:42:58.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:42:58.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:58.721 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 151189 2024-09-17T22:42:58.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.721 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:42:58.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:42:58.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:58.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:42:58.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:42:58.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:42:58.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:42:58.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713051+0000", 2024-09-17T22:42:58.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014722100000000001 2024-09-17T22:42:58.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:42:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713055+0000", 2024-09-17T22:42:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.16e-06 2024-09-17T22:42:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:42:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713065+0000", 2024-09-17T22:42:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8390000000000002e-06 2024-09-17T22:42:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:42:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713073+0000", 2024-09-17T22:42:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0390000000000005e-06 2024-09-17T22:42:58.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:42:58.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762644+0000", 2024-09-17T22:42:58.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049571160000000003 2024-09-17T22:42:58.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:42:58.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762739+0000", 2024-09-17T22:42:58.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4565e-05 2024-09-17T22:42:58.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:58.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:58.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:42:58.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.767159+0000", 2024-09-17T22:42:58.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044197159999999997 2024-09-17T22:42:58.725 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.725 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.725 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.725 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:58.725 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:58.725 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:58.726 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:42:58.727 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:58.724+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:58.827 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:58.825+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:58.841 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:58.841 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:58.841 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:58.841 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:58.850 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:42:58.929 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:42:58.926+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:42:58.954 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:58.954 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:58.955 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:58.955 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:58.963 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:42:59.030 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:59.027+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:59.116 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:59.116 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:42:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:42:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:42:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158673 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871954126000006, 2024-09-17T22:42:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039400614, 2024-09-17T22:42:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.117 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158673 2024-09-17T22:42:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:42:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.04e-07 2024-09-17T22:42:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241563+0000", 2024-09-17T22:42:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5799999999999998e-07 2024-09-17T22:42:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241566+0000", 2024-09-17T22:42:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8279999999999999e-06 2024-09-17T22:42:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241568+0000", 2024-09-17T22:42:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2709999999999999e-06 2024-09-17T22:42:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280954+0000", 2024-09-17T22:42:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039386051999999998 2024-09-17T22:42:59.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280962+0000", 2024-09-17T22:42:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3010000000000007e-06 2024-09-17T22:42:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158675 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871944834999994, 2024-09-17T22:42:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039408033000000002, 2024-09-17T22:42:59.121 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158675 2024-09-17T22:42:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:42:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.36e-07 2024-09-17T22:42:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241572+0000", 2024-09-17T22:42:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6599999999999994e-07 2024-09-17T22:42:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241575+0000", 2024-09-17T22:42:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8389999999999998e-06 2024-09-17T22:42:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241577+0000", 2024-09-17T22:42:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4119999999999999e-06 2024-09-17T22:42:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.124 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280975+0000", 2024-09-17T22:42:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039397205999999997 2024-09-17T22:42:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280979+0000", 2024-09-17T22:42:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.374e-06 2024-09-17T22:42:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158677 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871922392000002, 2024-09-17T22:42:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039409307999999997, 2024-09-17T22:42:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158677 2024-09-17T22:42:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:42:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241594+0000", 2024-09-17T22:42:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:42:59.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241595+0000", 2024-09-17T22:42:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2729999999999999e-06 2024-09-17T22:42:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241602+0000", 2024-09-17T22:42:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:42:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241606+0000", 2024-09-17T22:42:59.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3019999999999999e-06 2024-09-17T22:42:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280994+0000", 2024-09-17T22:42:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039388484000000001 2024-09-17T22:42:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281003+0000", 2024-09-17T22:42:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5459999999999992e-06 2024-09-17T22:42:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.129 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158679 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871906714000005, 2024-09-17T22:42:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039413403999999999, 2024-09-17T22:42:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.130 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158679 2024-09-17T22:42:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.131 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:42:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4999999999999999e-07 2024-09-17T22:42:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241613+0000", 2024-09-17T22:42:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.207e-06 2024-09-17T22:42:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241616+0000", 2024-09-17T22:42:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.275e-06 2024-09-17T22:42:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241618+0000", 2024-09-17T22:42:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:42:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281017+0000", 2024-09-17T22:42:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039398385000000001 2024-09-17T22:42:59.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281022+0000", 2024-09-17T22:42:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.896e-06 2024-09-17T22:42:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158681 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871894535999999, 2024-09-17T22:42:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039425292000000001, 2024-09-17T22:42:59.134 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158681 2024-09-17T22:42:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:42:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:42:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241622+0000", 2024-09-17T22:42:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:42:59.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241625+0000", 2024-09-17T22:42:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9629999999999998e-06 2024-09-17T22:42:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241628+0000", 2024-09-17T22:42:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4339999999999999e-06 2024-09-17T22:42:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281039+0000", 2024-09-17T22:42:59.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039410672000000001 2024-09-17T22:42:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281047+0000", 2024-09-17T22:42:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9969999999999998e-06 2024-09-17T22:42:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158682 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871885251999998, 2024-09-17T22:42:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039436455000000002, 2024-09-17T22:42:59.138 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158682 2024-09-17T22:42:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:42:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:42:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241632+0000", 2024-09-17T22:42:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:42:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241635+0000", 2024-09-17T22:42:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8940000000000002e-06 2024-09-17T22:42:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241637+0000", 2024-09-17T22:42:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3659999999999999e-06 2024-09-17T22:42:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281061+0000", 2024-09-17T22:42:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039423664999999997 2024-09-17T22:42:59.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281067+0000", 2024-09-17T22:42:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.285e-06 2024-09-17T22:42:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158684 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871875906, 2024-09-17T22:42:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462980000000002, 2024-09-17T22:42:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158684 2024-09-17T22:42:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:42:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241641+0000", 2024-09-17T22:42:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:42:59.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241644+0000", 2024-09-17T22:42:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8870000000000001e-06 2024-09-17T22:42:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241646+0000", 2024-09-17T22:42:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3530000000000002e-06 2024-09-17T22:42:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281093+0000", 2024-09-17T22:42:59.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039446947000000003 2024-09-17T22:42:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281103+0000", 2024-09-17T22:42:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6299999999999993e-06 2024-09-17T22:42:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158686 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871866736000001, 2024-09-17T22:42:59.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039474214000000001, 2024-09-17T22:42:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158686 2024-09-17T22:42:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:42:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:42:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241650+0000", 2024-09-17T22:42:59.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2999999999999999e-07 2024-09-17T22:42:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241653+0000", 2024-09-17T22:42:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9689999999999999e-06 2024-09-17T22:42:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241655+0000", 2024-09-17T22:42:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3460000000000001e-06 2024-09-17T22:42:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281118+0000", 2024-09-17T22:42:59.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462876000000001 2024-09-17T22:42:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281123+0000", 2024-09-17T22:42:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8840000000000002e-06 2024-09-17T22:42:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.152 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158688 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871857504999994, 2024-09-17T22:42:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039490458999999999, 2024-09-17T22:42:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158688 2024-09-17T22:42:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.153 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:42:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:42:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241659+0000", 2024-09-17T22:42:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3699999999999999e-07 2024-09-17T22:42:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241662+0000", 2024-09-17T22:42:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8499999999999998e-06 2024-09-17T22:42:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241665+0000", 2024-09-17T22:42:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.356e-06 2024-09-17T22:42:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281141+0000", 2024-09-17T22:42:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039476031000000002 2024-09-17T22:42:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281149+0000", 2024-09-17T22:42:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0709999999999995e-06 2024-09-17T22:42:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.156 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158690 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871848446000001, 2024-09-17T22:42:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039502378999999997, 2024-09-17T22:42:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158690 2024-09-17T22:42:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:42:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241668+0000", 2024-09-17T22:42:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241669+0000", 2024-09-17T22:42:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1009999999999999e-06 2024-09-17T22:42:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241672+0000", 2024-09-17T22:42:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9059999999999998e-06 2024-09-17T22:42:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241674+0000", 2024-09-17T22:42:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4439999999999998e-06 2024-09-17T22:42:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281162+0000", 2024-09-17T22:42:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039488189 2024-09-17T22:42:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281170+0000", 2024-09-17T22:42:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5310000000000004e-06 2024-09-17T22:42:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158692 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.160 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871839012999999, 2024-09-17T22:42:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039531064999999997, 2024-09-17T22:42:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158692 2024-09-17T22:42:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:42:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241678+0000", 2024-09-17T22:42:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0800000000000003e-07 2024-09-17T22:42:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241681+0000", 2024-09-17T22:42:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8729999999999999e-06 2024-09-17T22:42:59.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241683+0000", 2024-09-17T22:42:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.447e-06 2024-09-17T22:42:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281199+0000", 2024-09-17T22:42:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039515388999999998 2024-09-17T22:42:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.164 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281208+0000", 2024-09-17T22:42:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2399999999999996e-06 2024-09-17T22:42:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158694 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871829841999997, 2024-09-17T22:42:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039543986000000003, 2024-09-17T22:42:59.165 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158694 2024-09-17T22:42:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:42:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:42:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241687+0000", 2024-09-17T22:42:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:42:59.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241691+0000", 2024-09-17T22:42:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.811e-06 2024-09-17T22:42:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241693+0000", 2024-09-17T22:42:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.385e-06 2024-09-17T22:42:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281223+0000", 2024-09-17T22:42:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039529332 2024-09-17T22:42:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281230+0000", 2024-09-17T22:42:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2139999999999999e-06 2024-09-17T22:42:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158696 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:59.169 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871819454000004, 2024-09-17T22:42:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039570458000000003, 2024-09-17T22:42:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158696 2024-09-17T22:42:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:59.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:42:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:42:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:42:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:42:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.33e-07 2024-09-17T22:42:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241700+0000", 2024-09-17T22:42:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8949999999999998e-06 2024-09-17T22:42:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241703+0000", 2024-09-17T22:42:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3159999999999999e-06 2024-09-17T22:42:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.172 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281257+0000", 2024-09-17T22:42:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039553862000000002 2024-09-17T22:42:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281267+0000", 2024-09-17T22:42:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.024e-05 2024-09-17T22:42:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158698 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.173 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871809376000002, 2024-09-17T22:42:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039576493999999997, 2024-09-17T22:42:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158698 2024-09-17T22:42:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:59.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:42:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241707+0000", 2024-09-17T22:42:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:42:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241708+0000", 2024-09-17T22:42:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5000000000000001e-07 2024-09-17T22:42:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241711+0000", 2024-09-17T22:42:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9670000000000001e-06 2024-09-17T22:42:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241713+0000", 2024-09-17T22:42:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2689999999999998e-06 2024-09-17T22:42:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281278+0000", 2024-09-17T22:42:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039565206999999998 2024-09-17T22:42:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281283+0000", 2024-09-17T22:42:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:42:59.177 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158700 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871800164999996, 2024-09-17T22:42:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039604292999999999, 2024-09-17T22:42:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.178 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158700 2024-09-17T22:42:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:42:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:42:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241717+0000", 2024-09-17T22:42:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.78e-07 2024-09-17T22:42:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.180 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241720+0000", 2024-09-17T22:42:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8119999999999999e-06 2024-09-17T22:42:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241722+0000", 2024-09-17T22:42:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-17T22:42:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281311+0000", 2024-09-17T22:42:59.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039588894999999999 2024-09-17T22:42:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281320+0000", 2024-09-17T22:42:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9989999999999997e-06 2024-09-17T22:42:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158702 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871788843000004, 2024-09-17T22:42:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039616142, 2024-09-17T22:42:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158702 2024-09-17T22:42:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:59.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:59.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:42:59.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4400000000000001e-07 2024-09-17T22:42:59.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241728+0000", 2024-09-17T22:42:59.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0839999999999999e-06 2024-09-17T22:42:59.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241731+0000", 2024-09-17T22:42:59.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0690000000000002e-06 2024-09-17T22:42:59.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241734+0000", 2024-09-17T22:42:59.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4159999999999998e-06 2024-09-17T22:42:59.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281334+0000", 2024-09-17T22:42:59.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039600110000000001 2024-09-17T22:42:59.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281343+0000", 2024-09-17T22:42:59.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2189999999999993e-06 2024-09-17T22:42:59.186 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.187 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.187 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.187 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158704 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.187 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:59.187 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871778978999998, 2024-09-17T22:42:59.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039641485999999997, 2024-09-17T22:42:59.187 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.187 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.187 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.187 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.188 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158704 2024-09-17T22:42:59.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.188 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:59.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:59.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:42:59.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:42:59.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241738+0000", 2024-09-17T22:42:59.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0589999999999999e-06 2024-09-17T22:42:59.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.189 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.189 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241741+0000", 2024-09-17T22:42:59.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.943e-06 2024-09-17T22:42:59.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241744+0000", 2024-09-17T22:42:59.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4819999999999999e-06 2024-09-17T22:42:59.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281370+0000", 2024-09-17T22:42:59.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039625661 2024-09-17T22:42:59.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.190 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.190 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.190 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281378+0000", 2024-09-17T22:42:59.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1310000000000006e-06 2024-09-17T22:42:59.191 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.191 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.191 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.191 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.191 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158705 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.191 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:59.191 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871768428999999, 2024-09-17T22:42:59.191 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039652637999999997, 2024-09-17T22:42:59.191 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.191 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.191 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.191 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.191 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.192 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.192 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158705 2024-09-17T22:42:59.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.192 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:59.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:42:59.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:42:59.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-07 2024-09-17T22:42:59.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:42:59.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0599999999999999e-07 2024-09-17T22:42:59.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241751+0000", 2024-09-17T22:42:59.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.926e-06 2024-09-17T22:42:59.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241754+0000", 2024-09-17T22:42:59.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4360000000000001e-06 2024-09-17T22:42:59.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281392+0000", 2024-09-17T22:42:59.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039638566 2024-09-17T22:42:59.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281400+0000", 2024-09-17T22:42:59.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.571e-06 2024-09-17T22:42:59.195 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.195 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.195 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.195 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158707 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.195 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:59.195 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871747327999998, 2024-09-17T22:42:59.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039654809999999999, 2024-09-17T22:42:59.195 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.195 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.196 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.196 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.196 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.196 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.196 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158707 2024-09-17T22:42:59.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.196 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:59.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:42:59.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241769+0000", 2024-09-17T22:42:59.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4299999999999999e-07 2024-09-17T22:42:59.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241770+0000", 2024-09-17T22:42:59.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3769999999999999e-06 2024-09-17T22:42:59.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241773+0000", 2024-09-17T22:42:59.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2109999999999998e-06 2024-09-17T22:42:59.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241776+0000", 2024-09-17T22:42:59.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5069999999999999e-06 2024-09-17T22:42:59.198 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281415+0000", 2024-09-17T22:42:59.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039639499000000002 2024-09-17T22:42:59.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.199 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.199 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281423+0000", 2024-09-17T22:42:59.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8730000000000007e-06 2024-09-17T22:42:59.199 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.199 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.199 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.199 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158709 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:42:59.199 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:59.199 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 68.871736718999998, 2024-09-17T22:42:59.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039661937000000001, 2024-09-17T22:42:59.200 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:42:59.200 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:42:59.200 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:42:59.200 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:42:59.200 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:42:59.200 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:42:59.200 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158709 2024-09-17T22:42:59.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.200 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:42:59.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.200 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:42:59.200 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:59.200 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:42:59.201 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:42:59.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:42:59.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:42:59.201 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242843+0000", 2024-09-17T22:42:59.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010634780000000001 2024-09-17T22:42:59.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:42:59.201 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242845+0000", 2024-09-17T22:42:59.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7209999999999999e-06 2024-09-17T22:42:59.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:42:59.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242851+0000", 2024-09-17T22:42:59.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6949999999999997e-06 2024-09-17T22:42:59.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:42:59.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242855+0000", 2024-09-17T22:42:59.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7960000000000002e-06 2024-09-17T22:42:59.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:42:59.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281434+0000", 2024-09-17T22:42:59.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038579519 2024-09-17T22:42:59.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:42:59.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:42:59.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:42:59.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281441+0000", 2024-09-17T22:42:59.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7279999999999998e-06 2024-09-17T22:42:59.203 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.203 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.203 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.203 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:42:59.203 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:42:59.203 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:59.205 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:42:59.205 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:42:59.202+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:42:59.305 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:59.303+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:59.317 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:42:59.349 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:59.349 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:59.349 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:42:59.349 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:59.358 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:42:59.421 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:42:59.438 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:42:59.438 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:42:59.442 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:59.440+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:59.455 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:42:59.455 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:42:59.470 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:42:59.471 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:42:59.481 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:42:59.481 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:42:59.481 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T22:42:59.481 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T22:42:59.487 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:42:59.488 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:42:59.503 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:42:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:42:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:42:59.503 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:42:59.503 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:42:59.505 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:42:59.505 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:42:59.506 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:59.504+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:59.513 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:42:59.522 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:42:59.522 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:42:59.539 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:42:59.539 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:42:59.543 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:42:59.543 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:42:59.607 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:42:59.604+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:42:59.656 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:59.657 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:59.665 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:42:59.709 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:42:59.707+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:42:59.759 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:42:59.759 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:42:59.759 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 111, 2024-09-17T22:42:59.759 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "551c5ed8-e790-485a-bc95-a9578a29275c", 2024-09-17T22:42:59.759 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-09-17T22:40:50.189230+0000", 2024-09-17T22:42:59.759 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-09-17T22:42:59.003297+0000", 2024-09-17T22:42:59.759 INFO:teuthology.orchestra.run.smithi115.stdout: "last_up_change": "2024-09-17T22:41:53.099515+0000", 2024-09-17T22:42:59.759 INFO:teuthology.orchestra.run.smithi115.stdout: "last_in_change": "2024-09-17T22:40:52.260526+0000", 2024-09-17T22:42:59.759 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T22:42:59.760 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_num": 5806080, 2024-09-17T22:42:59.760 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_set": [ 2024-09-17T22:42:59.760 INFO:teuthology.orchestra.run.smithi115.stdout: "nodeep-scrub", 2024-09-17T22:42:59.760 INFO:teuthology.orchestra.run.smithi115.stdout: "noscrub", 2024-09-17T22:42:59.760 INFO:teuthology.orchestra.run.smithi115.stdout: "pglog_hardlimit", 2024-09-17T22:42:59.760 INFO:teuthology.orchestra.run.smithi115.stdout: "purged_snapdirs", 2024-09-17T22:42:59.760 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_deletes", 2024-09-17T22:42:59.760 INFO:teuthology.orchestra.run.smithi115.stdout: "sortbitwise" 2024-09-17T22:42:59.760 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:59.760 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_version": 5, 2024-09-17T22:42:59.760 INFO:teuthology.orchestra.run.smithi115.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T22:42:59.760 INFO:teuthology.orchestra.run.smithi115.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T22:42:59.760 INFO:teuthology.orchestra.run.smithi115.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T22:42:59.760 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_snapshot": "", 2024-09-17T22:42:59.761 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_max": 3, 2024-09-17T22:42:59.761 INFO:teuthology.orchestra.run.smithi115.stdout: "max_osd": 8, 2024-09-17T22:42:59.761 INFO:teuthology.orchestra.run.smithi115.stdout: "require_min_compat_client": "luminous", 2024-09-17T22:42:59.761 INFO:teuthology.orchestra.run.smithi115.stdout: "min_compat_client": "luminous", 2024-09-17T22:42:59.761 INFO:teuthology.orchestra.run.smithi115.stdout: "require_osd_release": "squid", 2024-09-17T22:42:59.761 INFO:teuthology.orchestra.run.smithi115.stdout: "allow_crimson": false, 2024-09-17T22:42:59.761 INFO:teuthology.orchestra.run.smithi115.stdout: "pools": [ 2024-09-17T22:42:59.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.761 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 1, 2024-09-17T22:42:59.761 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_name": ".mgr", 2024-09-17T22:42:59.761 INFO:teuthology.orchestra.run.smithi115.stdout: "create_time": "2024-09-17T22:41:02.338356+0000", 2024-09-17T22:42:59.761 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-09-17T22:42:59.761 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_names": "hashpspool", 2024-09-17T22:42:59.761 INFO:teuthology.orchestra.run.smithi115.stdout: "type": 1, 2024-09-17T22:42:59.762 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 2, 2024-09-17T22:42:59.762 INFO:teuthology.orchestra.run.smithi115.stdout: "min_size": 1, 2024-09-17T22:42:59.762 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_rule": 0, 2024-09-17T22:42:59.762 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:42:59.762 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:42:59.762 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:42:59.762 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:42:59.762 INFO:teuthology.orchestra.run.smithi115.stdout: "is_stretch_pool": false, 2024-09-17T22:42:59.762 INFO:teuthology.orchestra.run.smithi115.stdout: "object_hash": 2, 2024-09-17T22:42:59.762 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:42:59.762 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num": 1, 2024-09-17T22:42:59.762 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num": 1, 2024-09-17T22:42:59.762 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num_target": 1, 2024-09-17T22:42:59.763 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_target": 1, 2024-09-17T22:42:59.763 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_pending": 1, 2024-09-17T22:42:59.763 INFO:teuthology.orchestra.run.smithi115.stdout: "last_pg_merge_meta": { 2024-09-17T22:42:59.763 INFO:teuthology.orchestra.run.smithi115.stdout: "source_pgid": "0.0", 2024-09-17T22:42:59.763 INFO:teuthology.orchestra.run.smithi115.stdout: "ready_epoch": 0, 2024-09-17T22:42:59.763 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_started": 0, 2024-09-17T22:42:59.763 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_clean": 0, 2024-09-17T22:42:59.763 INFO:teuthology.orchestra.run.smithi115.stdout: "source_version": "0'0", 2024-09-17T22:42:59.763 INFO:teuthology.orchestra.run.smithi115.stdout: "target_version": "0'0" 2024-09-17T22:42:59.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.763 INFO:teuthology.orchestra.run.smithi115.stdout: "last_change": "17", 2024-09-17T22:42:59.763 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend": "0", 2024-09-17T22:42:59.763 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:42:59.763 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:42:59.764 INFO:teuthology.orchestra.run.smithi115.stdout: "auid": 0, 2024-09-17T22:42:59.764 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:42:59.764 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_seq": 0, 2024-09-17T22:42:59.764 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_epoch": 0, 2024-09-17T22:42:59.764 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_snaps": [], 2024-09-17T22:42:59.764 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps": "[]", 2024-09-17T22:42:59.764 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_bytes": 0, 2024-09-17T22:42:59.764 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_objects": 0, 2024-09-17T22:42:59.764 INFO:teuthology.orchestra.run.smithi115.stdout: "tiers": [], 2024-09-17T22:42:59.764 INFO:teuthology.orchestra.run.smithi115.stdout: "tier_of": -1, 2024-09-17T22:42:59.765 INFO:teuthology.orchestra.run.smithi115.stdout: "read_tier": -1, 2024-09-17T22:42:59.765 INFO:teuthology.orchestra.run.smithi115.stdout: "write_tier": -1, 2024-09-17T22:42:59.765 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_mode": "none", 2024-09-17T22:42:59.765 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_bytes": 0, 2024-09-17T22:42:59.765 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_objects": 0, 2024-09-17T22:42:59.765 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:42:59.765 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:42:59.765 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:42:59.765 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_flush_age": 0, 2024-09-17T22:42:59.765 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_evict_age": 0, 2024-09-17T22:42:59.765 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profile": "", 2024-09-17T22:42:59.765 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_params": { 2024-09-17T22:42:59.765 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "none" 2024-09-17T22:42:59.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.766 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_period": 0, 2024-09-17T22:42:59.766 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_count": 0, 2024-09-17T22:42:59.766 INFO:teuthology.orchestra.run.smithi115.stdout: "use_gmt_hitset": true, 2024-09-17T22:42:59.766 INFO:teuthology.orchestra.run.smithi115.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:42:59.766 INFO:teuthology.orchestra.run.smithi115.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:42:59.766 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:42:59.766 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:42:59.766 INFO:teuthology.orchestra.run.smithi115.stdout: "grade_table": [], 2024-09-17T22:42:59.766 INFO:teuthology.orchestra.run.smithi115.stdout: "stripe_width": 0, 2024-09-17T22:42:59.766 INFO:teuthology.orchestra.run.smithi115.stdout: "expected_num_objects": 0, 2024-09-17T22:42:59.766 INFO:teuthology.orchestra.run.smithi115.stdout: "fast_read": false, 2024-09-17T22:42:59.766 INFO:teuthology.orchestra.run.smithi115.stdout: "options": { 2024-09-17T22:42:59.766 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_max": 32, 2024-09-17T22:42:59.766 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_min": 1 2024-09-17T22:42:59.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.767 INFO:teuthology.orchestra.run.smithi115.stdout: "application_metadata": { 2024-09-17T22:42:59.767 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": {} 2024-09-17T22:42:59.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.767 INFO:teuthology.orchestra.run.smithi115.stdout: "read_balance": { 2024-09-17T22:42:59.767 INFO:teuthology.orchestra.run.smithi115.stdout: "score_type": "Fair distribution", 2024-09-17T22:42:59.767 INFO:teuthology.orchestra.run.smithi115.stdout: "score_acting": 6.9000000953674316, 2024-09-17T22:42:59.767 INFO:teuthology.orchestra.run.smithi115.stdout: "score_stable": 6.9000000953674316, 2024-09-17T22:42:59.767 INFO:teuthology.orchestra.run.smithi115.stdout: "optimal_score": 0.28999999165534973, 2024-09-17T22:42:59.767 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_acting": 2, 2024-09-17T22:42:59.767 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_stable": 2, 2024-09-17T22:42:59.767 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity_weighted": 0.81284332275390625, 2024-09-17T22:42:59.767 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-17T22:42:59.767 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity_weighted": 0.81000000238418579 2024-09-17T22:42:59.768 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.768 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-17T22:42:59.768 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_name": "rbd", 2024-09-17T22:42:59.768 INFO:teuthology.orchestra.run.smithi115.stdout: "create_time": "2024-09-17T22:41:08.097861+0000", 2024-09-17T22:42:59.768 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 8193, 2024-09-17T22:42:59.768 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T22:42:59.768 INFO:teuthology.orchestra.run.smithi115.stdout: "type": 1, 2024-09-17T22:42:59.768 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 2, 2024-09-17T22:42:59.769 INFO:teuthology.orchestra.run.smithi115.stdout: "min_size": 1, 2024-09-17T22:42:59.769 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_rule": 0, 2024-09-17T22:42:59.769 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:42:59.769 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:42:59.769 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:42:59.769 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:42:59.769 INFO:teuthology.orchestra.run.smithi115.stdout: "is_stretch_pool": false, 2024-09-17T22:42:59.769 INFO:teuthology.orchestra.run.smithi115.stdout: "object_hash": 2, 2024-09-17T22:42:59.769 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:42:59.769 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num": 8, 2024-09-17T22:42:59.769 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num": 8, 2024-09-17T22:42:59.769 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num_target": 8, 2024-09-17T22:42:59.769 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_target": 8, 2024-09-17T22:42:59.769 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_pending": 8, 2024-09-17T22:42:59.769 INFO:teuthology.orchestra.run.smithi115.stdout: "last_pg_merge_meta": { 2024-09-17T22:42:59.770 INFO:teuthology.orchestra.run.smithi115.stdout: "source_pgid": "0.0", 2024-09-17T22:42:59.770 INFO:teuthology.orchestra.run.smithi115.stdout: "ready_epoch": 0, 2024-09-17T22:42:59.770 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_started": 0, 2024-09-17T22:42:59.770 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_clean": 0, 2024-09-17T22:42:59.770 INFO:teuthology.orchestra.run.smithi115.stdout: "source_version": "0'0", 2024-09-17T22:42:59.770 INFO:teuthology.orchestra.run.smithi115.stdout: "target_version": "0'0" 2024-09-17T22:42:59.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.770 INFO:teuthology.orchestra.run.smithi115.stdout: "last_change": "21", 2024-09-17T22:42:59.770 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend": "0", 2024-09-17T22:42:59.770 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:42:59.770 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:42:59.770 INFO:teuthology.orchestra.run.smithi115.stdout: "auid": 0, 2024-09-17T22:42:59.770 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:42:59.770 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_seq": 2, 2024-09-17T22:42:59.770 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_epoch": 21, 2024-09-17T22:42:59.771 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_snaps": [], 2024-09-17T22:42:59.771 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps": "[]", 2024-09-17T22:42:59.771 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_bytes": 0, 2024-09-17T22:42:59.771 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_objects": 0, 2024-09-17T22:42:59.771 INFO:teuthology.orchestra.run.smithi115.stdout: "tiers": [], 2024-09-17T22:42:59.771 INFO:teuthology.orchestra.run.smithi115.stdout: "tier_of": -1, 2024-09-17T22:42:59.771 INFO:teuthology.orchestra.run.smithi115.stdout: "read_tier": -1, 2024-09-17T22:42:59.771 INFO:teuthology.orchestra.run.smithi115.stdout: "write_tier": -1, 2024-09-17T22:42:59.771 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_mode": "none", 2024-09-17T22:42:59.771 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_bytes": 0, 2024-09-17T22:42:59.771 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_objects": 0, 2024-09-17T22:42:59.771 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:42:59.771 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:42:59.771 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:42:59.771 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_flush_age": 0, 2024-09-17T22:42:59.772 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_evict_age": 0, 2024-09-17T22:42:59.772 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profile": "", 2024-09-17T22:42:59.772 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_params": { 2024-09-17T22:42:59.772 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "none" 2024-09-17T22:42:59.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.772 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_period": 0, 2024-09-17T22:42:59.772 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_count": 0, 2024-09-17T22:42:59.772 INFO:teuthology.orchestra.run.smithi115.stdout: "use_gmt_hitset": true, 2024-09-17T22:42:59.772 INFO:teuthology.orchestra.run.smithi115.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:42:59.772 INFO:teuthology.orchestra.run.smithi115.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:42:59.772 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:42:59.772 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:42:59.772 INFO:teuthology.orchestra.run.smithi115.stdout: "grade_table": [], 2024-09-17T22:42:59.772 INFO:teuthology.orchestra.run.smithi115.stdout: "stripe_width": 0, 2024-09-17T22:42:59.773 INFO:teuthology.orchestra.run.smithi115.stdout: "expected_num_objects": 0, 2024-09-17T22:42:59.773 INFO:teuthology.orchestra.run.smithi115.stdout: "fast_read": false, 2024-09-17T22:42:59.773 INFO:teuthology.orchestra.run.smithi115.stdout: "options": {}, 2024-09-17T22:42:59.773 INFO:teuthology.orchestra.run.smithi115.stdout: "application_metadata": { 2024-09-17T22:42:59.773 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd": {} 2024-09-17T22:42:59.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.773 INFO:teuthology.orchestra.run.smithi115.stdout: "read_balance": { 2024-09-17T22:42:59.773 INFO:teuthology.orchestra.run.smithi115.stdout: "score_type": "Fair distribution", 2024-09-17T22:42:59.773 INFO:teuthology.orchestra.run.smithi115.stdout: "score_acting": 2.619999885559082, 2024-09-17T22:42:59.773 INFO:teuthology.orchestra.run.smithi115.stdout: "score_stable": 2.619999885559082, 2024-09-17T22:42:59.773 INFO:teuthology.orchestra.run.smithi115.stdout: "optimal_score": 0.86000001430511475, 2024-09-17T22:42:59.773 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_acting": 2.25, 2024-09-17T22:42:59.773 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_stable": 2.25, 2024-09-17T22:42:59.773 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity_weighted": 0.81284332275390625, 2024-09-17T22:42:59.774 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-17T22:42:59.774 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity_weighted": 0.81000000238418579 2024-09-17T22:42:59.774 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.774 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 3, 2024-09-17T22:42:59.774 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_name": "pool_client.0", 2024-09-17T22:42:59.774 INFO:teuthology.orchestra.run.smithi115.stdout: "create_time": "2024-09-17T22:41:17.501542+0000", 2024-09-17T22:42:59.774 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 8193, 2024-09-17T22:42:59.774 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T22:42:59.774 INFO:teuthology.orchestra.run.smithi115.stdout: "type": 1, 2024-09-17T22:42:59.774 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 2, 2024-09-17T22:42:59.774 INFO:teuthology.orchestra.run.smithi115.stdout: "min_size": 1, 2024-09-17T22:42:59.774 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_rule": 0, 2024-09-17T22:42:59.774 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:42:59.774 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:42:59.775 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:42:59.775 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:42:59.775 INFO:teuthology.orchestra.run.smithi115.stdout: "is_stretch_pool": false, 2024-09-17T22:42:59.775 INFO:teuthology.orchestra.run.smithi115.stdout: "object_hash": 2, 2024-09-17T22:42:59.775 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:42:59.775 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num": 32, 2024-09-17T22:42:59.775 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num": 32, 2024-09-17T22:42:59.775 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num_target": 32, 2024-09-17T22:42:59.775 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_target": 32, 2024-09-17T22:42:59.775 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_pending": 32, 2024-09-17T22:42:59.775 INFO:teuthology.orchestra.run.smithi115.stdout: "last_pg_merge_meta": { 2024-09-17T22:42:59.775 INFO:teuthology.orchestra.run.smithi115.stdout: "source_pgid": "0.0", 2024-09-17T22:42:59.775 INFO:teuthology.orchestra.run.smithi115.stdout: "ready_epoch": 0, 2024-09-17T22:42:59.775 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_started": 0, 2024-09-17T22:42:59.775 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_clean": 0, 2024-09-17T22:42:59.776 INFO:teuthology.orchestra.run.smithi115.stdout: "source_version": "0'0", 2024-09-17T22:42:59.776 INFO:teuthology.orchestra.run.smithi115.stdout: "target_version": "0'0" 2024-09-17T22:42:59.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.776 INFO:teuthology.orchestra.run.smithi115.stdout: "last_change": "111", 2024-09-17T22:42:59.776 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend": "0", 2024-09-17T22:42:59.776 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:42:59.776 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:42:59.776 INFO:teuthology.orchestra.run.smithi115.stdout: "auid": 0, 2024-09-17T22:42:59.776 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:42:59.776 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_seq": 40, 2024-09-17T22:42:59.776 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_epoch": 111, 2024-09-17T22:42:59.776 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_snaps": [], 2024-09-17T22:42:59.776 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps": "[]", 2024-09-17T22:42:59.777 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_bytes": 0, 2024-09-17T22:42:59.777 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_objects": 0, 2024-09-17T22:42:59.777 INFO:teuthology.orchestra.run.smithi115.stdout: "tiers": [], 2024-09-17T22:42:59.777 INFO:teuthology.orchestra.run.smithi115.stdout: "tier_of": -1, 2024-09-17T22:42:59.777 INFO:teuthology.orchestra.run.smithi115.stdout: "read_tier": -1, 2024-09-17T22:42:59.777 INFO:teuthology.orchestra.run.smithi115.stdout: "write_tier": -1, 2024-09-17T22:42:59.777 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_mode": "none", 2024-09-17T22:42:59.777 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_bytes": 0, 2024-09-17T22:42:59.777 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_objects": 0, 2024-09-17T22:42:59.777 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:42:59.777 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:42:59.777 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:42:59.777 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_flush_age": 0, 2024-09-17T22:42:59.777 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_evict_age": 0, 2024-09-17T22:42:59.778 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profile": "", 2024-09-17T22:42:59.778 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_params": { 2024-09-17T22:42:59.778 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "none" 2024-09-17T22:42:59.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.778 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_period": 0, 2024-09-17T22:42:59.778 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_count": 0, 2024-09-17T22:42:59.778 INFO:teuthology.orchestra.run.smithi115.stdout: "use_gmt_hitset": true, 2024-09-17T22:42:59.778 INFO:teuthology.orchestra.run.smithi115.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:42:59.778 INFO:teuthology.orchestra.run.smithi115.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:42:59.778 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:42:59.778 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:42:59.778 INFO:teuthology.orchestra.run.smithi115.stdout: "grade_table": [], 2024-09-17T22:42:59.778 INFO:teuthology.orchestra.run.smithi115.stdout: "stripe_width": 0, 2024-09-17T22:42:59.779 INFO:teuthology.orchestra.run.smithi115.stdout: "expected_num_objects": 0, 2024-09-17T22:42:59.779 INFO:teuthology.orchestra.run.smithi115.stdout: "fast_read": false, 2024-09-17T22:42:59.779 INFO:teuthology.orchestra.run.smithi115.stdout: "options": {}, 2024-09-17T22:42:59.779 INFO:teuthology.orchestra.run.smithi115.stdout: "application_metadata": { 2024-09-17T22:42:59.779 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd": {} 2024-09-17T22:42:59.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.779 INFO:teuthology.orchestra.run.smithi115.stdout: "read_balance": { 2024-09-17T22:42:59.779 INFO:teuthology.orchestra.run.smithi115.stdout: "score_type": "Fair distribution", 2024-09-17T22:42:59.779 INFO:teuthology.orchestra.run.smithi115.stdout: "score_acting": 1.5299999713897705, 2024-09-17T22:42:59.779 INFO:teuthology.orchestra.run.smithi115.stdout: "score_stable": 1.5299999713897705, 2024-09-17T22:42:59.779 INFO:teuthology.orchestra.run.smithi115.stdout: "optimal_score": 1, 2024-09-17T22:42:59.779 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-17T22:42:59.779 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-17T22:42:59.779 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity_weighted": 0.81284332275390625, 2024-09-17T22:42:59.779 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-17T22:42:59.780 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity_weighted": 0.81000000238418579 2024-09-17T22:42:59.780 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.780 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.780 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:59.780 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:42:59.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.780 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 0, 2024-09-17T22:42:59.780 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "3bd5c11f-e7c2-4968-839a-2799c00d93a6", 2024-09-17T22:42:59.780 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 0, 2024-09-17T22:42:59.780 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:59.780 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:59.780 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 0, 2024-09-17T22:42:59.780 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:59.780 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:59.780 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:59.781 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 23, 2024-09-17T22:42:59.781 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 51, 2024-09-17T22:42:59.781 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:59.781 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:59.781 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.781 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.781 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6824", 2024-09-17T22:42:59.781 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:59.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.781 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.781 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6825", 2024-09-17T22:42:59.781 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:59.781 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.782 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.782 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:59.782 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.782 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.782 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6826", 2024-09-17T22:42:59.782 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:59.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.782 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.782 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6827", 2024-09-17T22:42:59.782 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:59.783 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.783 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.783 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:59.783 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.783 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.783 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6830", 2024-09-17T22:42:59.783 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:59.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.783 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.783 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6831", 2024-09-17T22:42:59.783 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:59.783 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.784 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.784 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:59.784 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.784 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.784 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6828", 2024-09-17T22:42:59.784 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:59.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.784 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.784 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6829", 2024-09-17T22:42:59.784 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:42:59.785 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.785 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.785 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6825/2782271783", 2024-09-17T22:42:59.785 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6827/2782271783", 2024-09-17T22:42:59.785 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6831/2782271783", 2024-09-17T22:42:59.785 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6829/2782271783", 2024-09-17T22:42:59.785 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:59.785 INFO:teuthology.orchestra.run.smithi115.stdout: "exists" 2024-09-17T22:42:59.785 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.785 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 1, 2024-09-17T22:42:59.786 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "02918203-cf2e-49fc-b994-743b5d81b4a1", 2024-09-17T22:42:59.786 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:59.786 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:59.786 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:59.786 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:59.786 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:59.786 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:59.786 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:59.786 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 109, 2024-09-17T22:42:59.786 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:59.786 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:59.786 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:59.786 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.787 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.787 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6816", 2024-09-17T22:42:59.787 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:59.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.787 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.787 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6817", 2024-09-17T22:42:59.787 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:59.787 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.787 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.787 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:59.787 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.788 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.788 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6818", 2024-09-17T22:42:59.788 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:59.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.788 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.788 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6819", 2024-09-17T22:42:59.788 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:59.788 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.788 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.788 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:59.788 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.789 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.789 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6822", 2024-09-17T22:42:59.789 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:59.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.789 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.789 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6823", 2024-09-17T22:42:59.789 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:59.789 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.789 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.789 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:59.789 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.790 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.790 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6820", 2024-09-17T22:42:59.790 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:59.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.790 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.790 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6821", 2024-09-17T22:42:59.790 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:42:59.790 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.790 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.790 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6817/1941780390", 2024-09-17T22:42:59.790 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6819/1941780390", 2024-09-17T22:42:59.791 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6823/1941780390", 2024-09-17T22:42:59.791 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6821/1941780390", 2024-09-17T22:42:59.791 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:59.791 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:59.791 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:59.791 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.791 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 2, 2024-09-17T22:42:59.791 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "22ee34ef-522e-4121-85b7-38bbafeca5de", 2024-09-17T22:42:59.791 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:59.791 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:59.791 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:59.791 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:59.791 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:59.792 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:59.792 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 13, 2024-09-17T22:42:59.792 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 99, 2024-09-17T22:42:59.792 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:59.792 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:59.792 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:59.792 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.792 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.792 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6808", 2024-09-17T22:42:59.792 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:59.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.793 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.793 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6809", 2024-09-17T22:42:59.793 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:59.793 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.793 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.793 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:59.793 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.793 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.793 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6810", 2024-09-17T22:42:59.793 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:59.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.793 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.794 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6811", 2024-09-17T22:42:59.794 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:59.794 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.794 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.794 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:59.794 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.794 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.794 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6814", 2024-09-17T22:42:59.794 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:59.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.795 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.795 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6815", 2024-09-17T22:42:59.795 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:59.795 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.795 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.795 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:59.795 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.795 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.795 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6812", 2024-09-17T22:42:59.795 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:59.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.796 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.796 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6813", 2024-09-17T22:42:59.796 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:42:59.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.796 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.796 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6809/689014698", 2024-09-17T22:42:59.796 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6811/689014698", 2024-09-17T22:42:59.796 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6815/689014698", 2024-09-17T22:42:59.796 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6813/689014698", 2024-09-17T22:42:59.796 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:59.796 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:59.796 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:59.797 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.797 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 3, 2024-09-17T22:42:59.797 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "5d464921-5696-43c5-b89d-430a54ddefb5", 2024-09-17T22:42:59.797 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:59.797 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:59.797 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:59.797 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 0.50274658203125, 2024-09-17T22:42:59.797 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:59.797 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:59.797 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 13, 2024-09-17T22:42:59.797 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 54, 2024-09-17T22:42:59.797 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:59.797 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:59.798 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:59.798 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.798 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.798 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6800", 2024-09-17T22:42:59.798 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:59.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.798 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.798 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6801", 2024-09-17T22:42:59.798 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:59.798 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.798 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.799 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:59.799 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.799 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.799 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6802", 2024-09-17T22:42:59.799 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:59.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.799 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.799 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6803", 2024-09-17T22:42:59.799 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:59.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.799 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.799 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:59.800 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.800 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.800 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6806", 2024-09-17T22:42:59.800 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:59.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.800 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.800 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6807", 2024-09-17T22:42:59.800 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:59.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.800 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.800 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:59.800 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.801 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.801 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6804", 2024-09-17T22:42:59.801 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:59.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.801 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.801 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6805", 2024-09-17T22:42:59.801 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:42:59.801 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.801 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.801 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6801/453192890", 2024-09-17T22:42:59.801 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6803/453192890", 2024-09-17T22:42:59.802 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6807/453192890", 2024-09-17T22:42:59.802 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6805/453192890", 2024-09-17T22:42:59.802 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:59.802 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:59.802 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:59.802 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.802 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 4, 2024-09-17T22:42:59.802 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "4cb6973a-63ae-4f48-af94-37610aee3c06", 2024-09-17T22:42:59.802 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:59.802 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:59.802 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:59.803 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:59.803 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:59.803 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:59.803 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:59.803 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 51, 2024-09-17T22:42:59.803 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:59.803 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:59.803 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:59.803 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.803 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.803 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6800", 2024-09-17T22:42:59.803 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:59.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.804 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.804 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6801", 2024-09-17T22:42:59.804 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:59.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.804 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.804 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:59.804 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.804 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.804 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6802", 2024-09-17T22:42:59.804 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:59.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.805 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.805 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6803", 2024-09-17T22:42:59.805 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:59.805 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.805 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.805 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:59.805 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.805 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.805 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6806", 2024-09-17T22:42:59.805 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:59.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.806 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.806 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6807", 2024-09-17T22:42:59.806 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:59.806 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.806 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.806 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:59.806 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.806 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.807 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6804", 2024-09-17T22:42:59.807 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:59.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.807 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.807 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6805", 2024-09-17T22:42:59.807 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:42:59.807 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.807 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.807 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6801/380388860", 2024-09-17T22:42:59.807 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6803/380388860", 2024-09-17T22:42:59.807 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6807/380388860", 2024-09-17T22:42:59.808 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6805/380388860", 2024-09-17T22:42:59.808 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:59.808 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:59.808 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:59.808 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.808 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 5, 2024-09-17T22:42:59.808 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "ef9fe11b-b719-48f4-9c6c-e1b1b355acdc", 2024-09-17T22:42:59.808 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:59.808 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:59.808 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:59.808 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:59.808 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:59.809 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:59.809 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:59.809 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 23, 2024-09-17T22:42:59.809 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:59.809 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:59.809 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:59.809 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.809 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.809 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6808", 2024-09-17T22:42:59.809 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:59.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.809 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.809 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6809", 2024-09-17T22:42:59.810 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:59.810 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.810 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.810 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:59.810 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.810 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.810 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6810", 2024-09-17T22:42:59.810 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:59.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.811 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.811 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6811", 2024-09-17T22:42:59.811 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:59.811 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.811 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.811 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:59.811 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.811 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.811 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6814", 2024-09-17T22:42:59.811 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:59.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.811 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.812 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6815", 2024-09-17T22:42:59.812 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:59.812 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.812 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.812 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:59.812 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.812 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.812 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6812", 2024-09-17T22:42:59.812 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:59.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.812 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.812 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6813", 2024-09-17T22:42:59.813 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:42:59.813 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.813 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.813 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6809/943488949", 2024-09-17T22:42:59.813 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6811/943488949", 2024-09-17T22:42:59.813 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6815/943488949", 2024-09-17T22:42:59.813 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6813/943488949", 2024-09-17T22:42:59.813 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:59.813 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:59.813 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:59.813 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.814 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 6, 2024-09-17T22:42:59.814 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "e4045aec-22c1-401b-bfbf-64f115bc32af", 2024-09-17T22:42:59.814 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:59.814 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:59.814 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:59.814 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:59.814 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:59.814 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:59.814 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:59.814 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 99, 2024-09-17T22:42:59.814 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:59.814 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:59.814 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:59.814 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.815 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.815 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6816", 2024-09-17T22:42:59.815 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:59.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.815 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.815 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6817", 2024-09-17T22:42:59.815 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:59.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.815 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.815 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:59.815 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.816 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.816 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6818", 2024-09-17T22:42:59.816 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:59.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.816 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.816 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6819", 2024-09-17T22:42:59.816 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:59.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.816 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.816 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:59.816 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.817 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.817 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6822", 2024-09-17T22:42:59.817 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:59.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.817 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.817 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6823", 2024-09-17T22:42:59.817 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:59.817 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.817 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.817 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:59.817 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.818 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.818 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6820", 2024-09-17T22:42:59.818 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:59.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.818 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.818 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6821", 2024-09-17T22:42:59.818 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:42:59.818 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.818 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.818 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6817/1873501077", 2024-09-17T22:42:59.818 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6819/1873501077", 2024-09-17T22:42:59.818 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6823/1873501077", 2024-09-17T22:42:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6821/1873501077", 2024-09-17T22:42:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 7, 2024-09-17T22:42:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "4a05a5d7-8281-4572-a9d0-6095ee5e168b", 2024-09-17T22:42:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:42:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:42:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:42:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:42:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:42:59.819 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:42:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:42:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 108, 2024-09-17T22:42:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:42:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:42:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:42:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6824", 2024-09-17T22:42:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6825", 2024-09-17T22:42:59.820 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:42:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6826", 2024-09-17T22:42:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.822 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.822 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6827", 2024-09-17T22:42:59.822 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:59.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.822 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.822 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:42:59.822 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.822 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.822 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6830", 2024-09-17T22:42:59.822 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6831", 2024-09-17T22:42:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:42:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:42:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.823 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:42:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6828", 2024-09-17T22:42:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:42:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6829", 2024-09-17T22:42:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:42:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6825/3609357666", 2024-09-17T22:42:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6827/3609357666", 2024-09-17T22:42:59.824 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6831/3609357666", 2024-09-17T22:42:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6829/3609357666", 2024-09-17T22:42:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:42:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:42:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:42:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_xinfo": [ 2024-09-17T22:42:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 0, 2024-09-17T22:42:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:59.825 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:59.351758+0000", 2024-09-17T22:42:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 51 2024-09-17T22:42:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 1, 2024-09-17T22:42:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:59.826 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:59.058915+0000", 2024-09-17T22:42:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 2, 2024-09-17T22:42:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.922383+0000", 2024-09-17T22:42:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.827 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 3, 2024-09-17T22:42:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.802379+0000", 2024-09-17T22:42:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 4, 2024-09-17T22:42:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.881681+0000", 2024-09-17T22:42:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 5, 2024-09-17T22:42:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.616649+0000", 2024-09-17T22:42:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:59.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 6, 2024-09-17T22:42:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.589555+0000", 2024-09-17T22:42:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 7, 2024-09-17T22:42:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:42:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:42:59.830 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:42:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:42:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:42:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:59.047783+0000", 2024-09-17T22:42:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:42:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_upmap": [ 2024-09-17T22:42:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.1c", 2024-09-17T22:42:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:42:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-09-17T22:42:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: 3 2024-09-17T22:42:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.1d", 2024-09-17T22:42:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:42:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-09-17T22:42:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: 6 2024-09-17T22:42:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_upmap_items": [ 2024-09-17T22:42:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.12", 2024-09-17T22:42:59.832 INFO:teuthology.orchestra.run.smithi115.stdout: "mappings": [ 2024-09-17T22:42:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: "from": 3, 2024-09-17T22:42:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: "to": 0 2024-09-17T22:42:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.16", 2024-09-17T22:42:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: "mappings": [ 2024-09-17T22:42:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: "from": 2, 2024-09-17T22:42:59.833 INFO:teuthology.orchestra.run.smithi115.stdout: "to": 0 2024-09-17T22:42:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.1d", 2024-09-17T22:42:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: "mappings": [ 2024-09-17T22:42:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: "from": 3, 2024-09-17T22:42:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: "to": 7 2024-09-17T22:42:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_upmap_primaries": [], 2024-09-17T22:42:59.834 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_temp": [ 2024-09-17T22:42:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.12", 2024-09-17T22:42:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:42:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-09-17T22:42:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: 3 2024-09-17T22:42:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.16", 2024-09-17T22:42:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:42:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-09-17T22:42:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: 2 2024-09-17T22:42:59.835 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_temp": [], 2024-09-17T22:42:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "blocklist": {}, 2024-09-17T22:42:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "range_blocklist": {}, 2024-09-17T22:42:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profiles": { 2024-09-17T22:42:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "default": { 2024-09-17T22:42:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "crush-failure-domain": "osd", 2024-09-17T22:42:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "k": "2", 2024-09-17T22:42:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "m": "1", 2024-09-17T22:42:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "plugin": "jerasure", 2024-09-17T22:42:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: "technique": "reed_sol_van" 2024-09-17T22:42:59.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:42:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps_queue": [ 2024-09-17T22:42:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 3, 2024-09-17T22:42:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: "snaps": [ 2024-09-17T22:42:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: "begin": 23, 2024-09-17T22:42:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: "length": 18 2024-09-17T22:42:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: "new_removed_snaps": [ 2024-09-17T22:42:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 3, 2024-09-17T22:42:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: "snaps": [ 2024-09-17T22:42:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:42:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: "begin": 23, 2024-09-17T22:42:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: "length": 1 2024-09-17T22:42:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:42:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:42:59.838 INFO:teuthology.orchestra.run.smithi115.stdout: "new_purged_snaps": [], 2024-09-17T22:42:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_node_flags": {}, 2024-09-17T22:42:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "device_class_flags": {}, 2024-09-17T22:42:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": { 2024-09-17T22:42:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode_enabled": false, 2024-09-17T22:42:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_bucket_count": 0, 2024-09-17T22:42:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "degraded_stretch_mode": 0, 2024-09-17T22:42:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "recovering_stretch_mode": 0, 2024-09-17T22:42:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode_bucket": 0 2024-09-17T22:42:59.839 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:42:59.839 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:42:59.841 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:59.841 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:59.842 INFO:tasks.thrashosds.thrasher:j is {'epoch': 111, 'fsid': '551c5ed8-e790-485a-bc95-a9578a29275c', 'created': '2024-09-17T22:40:50.189230+0000', 'modified': '2024-09-17T22:42:59.003297+0000', 'last_up_change': '2024-09-17T22:41:53.099515+0000', 'last_in_change': '2024-09-17T22:40:52.260526+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T22:41:02.338356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8128433227539062, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.8100000023841858}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T22:41:08.097861+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.619999885559082, 'score_stable': 2.619999885559082, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8128433227539062, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.8100000023841858}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T22:41:17.501542+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 40, 'snap_epoch': 111, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5299999713897705, 'score_stable': 1.5299999713897705, 'optimal_score': 1, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 0.8128433227539062, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.8100000023841858}}], 'osds': [{'osd': 0, 'uuid': '3bd5c11f-e7c2-4968-839a-2799c00d93a6', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 51, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6824', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6825', 'nonce': 2782271783}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 2782271783}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 2782271783}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 2782271783}]}, 'public_addr': '172.21.15.115:6825/2782271783', 'cluster_addr': '172.21.15.115:6827/2782271783', 'heartbeat_back_addr': '172.21.15.115:6831/2782271783', 'heartbeat_front_addr': '172.21.15.115:6829/2782271783', 'state': ['exists']}, {'osd': 1, 'uuid': '02918203-cf2e-49fc-b994-743b5d81b4a1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 109, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 1941780390}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 1941780390}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 1941780390}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 1941780390}]}, 'public_addr': '172.21.15.115:6817/1941780390', 'cluster_addr': '172.21.15.115:6819/1941780390', 'heartbeat_back_addr': '172.21.15.115:6823/1941780390', 'heartbeat_front_addr': '172.21.15.115:6821/1941780390', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '22ee34ef-522e-4121-85b7-38bbafeca5de', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 689014698}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 689014698}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 689014698}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 689014698}]}, 'public_addr': '172.21.15.115:6809/689014698', 'cluster_addr': '172.21.15.115:6811/689014698', 'heartbeat_back_addr': '172.21.15.115:6815/689014698', 'heartbeat_front_addr': '172.21.15.115:6813/689014698', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5d464921-5696-43c5-b89d-430a54ddefb5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.50274658203125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 453192890}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 453192890}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 453192890}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 453192890}]}, 'public_addr': '172.21.15.115:6801/453192890', 'cluster_addr': '172.21.15.115:6803/453192890', 'heartbeat_back_addr': '172.21.15.115:6807/453192890', 'heartbeat_front_addr': '172.21.15.115:6805/453192890', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '4cb6973a-63ae-4f48-af94-37610aee3c06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6800', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6801', 'nonce': 380388860}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6802', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6803', 'nonce': 380388860}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6806', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6807', 'nonce': 380388860}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6804', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6805', 'nonce': 380388860}]}, 'public_addr': '172.21.15.129:6801/380388860', 'cluster_addr': '172.21.15.129:6803/380388860', 'heartbeat_back_addr': '172.21.15.129:6807/380388860', 'heartbeat_front_addr': '172.21.15.129:6805/380388860', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ef9fe11b-b719-48f4-9c6c-e1b1b355acdc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6808', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6809', 'nonce': 943488949}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6810', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6811', 'nonce': 943488949}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6814', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6815', 'nonce': 943488949}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6812', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6813', 'nonce': 943488949}]}, 'public_addr': '172.21.15.129:6809/943488949', 'cluster_addr': '172.21.15.129:6811/943488949', 'heartbeat_back_addr': '172.21.15.129:6815/943488949', 'heartbeat_front_addr': '172.21.15.129:6813/943488949', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e4045aec-22c1-401b-bfbf-64f115bc32af', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6816', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6817', 'nonce': 1873501077}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6818', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6819', 'nonce': 1873501077}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6822', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6823', 'nonce': 1873501077}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6820', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6821', 'nonce': 1873501077}]}, 'public_addr': '172.21.15.129:6817/1873501077', 'cluster_addr': '172.21.15.129:6819/1873501077', 'heartbeat_back_addr': '172.21.15.129:6823/1873501077', 'heartbeat_front_addr': '172.21.15.129:6821/1873501077', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '4a05a5d7-8281-4572-a9d0-6095ee5e168b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 108, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6824', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6825', 'nonce': 3609357666}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6826', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6827', 'nonce': 3609357666}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6830', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6831', 'nonce': 3609357666}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6828', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6829', 'nonce': 3609357666}]}, 'public_addr': '172.21.15.129:6825/3609357666', 'cluster_addr': '172.21.15.129:6827/3609357666', 'heartbeat_back_addr': '172.21.15.129:6831/3609357666', 'heartbeat_front_addr': '172.21.15.129:6829/3609357666', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:59.351758+0000', 'dead_epoch': 51}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:59.058915+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.922383+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.802379+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.881681+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.616649+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.589555+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:59.047783+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1c', 'osds': [1, 3]}, {'pgid': '3.1d', 'osds': [1, 6]}], 'pg_upmap_items': [{'pgid': '3.12', 'mappings': [{'from': 3, 'to': 0}]}, {'pgid': '3.16', 'mappings': [{'from': 2, 'to': 0}]}, {'pgid': '3.1d', 'mappings': [{'from': 3, 'to': 7}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.12', 'osds': [6, 3]}, {'pgid': '3.16', 'osds': [3, 2]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 23, 'length': 18}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 23, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T22:42:59.842 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.1d 2024-09-17T22:42:59.843 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap-items 3.1d 2024-09-17T22:42:59.843 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:42:59.838+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:42:59.843 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:42:59.942 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:42:59.939+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:42:59.986 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:42:59.987 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:42:59.995 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:43:00.020 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:43:00.063 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:00.061+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:00.143 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:00.140+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:00.145 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:00.145 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:00.155 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:43:00.244 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:00.241+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:00.298 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:00.298 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:00.307 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:43:00.345 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:00.342+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:00.410 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:00.411 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:00.420 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:43:00.463 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:00.461+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:00.523 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:00.523 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:00.532 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:43:00.575 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:00.573+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:00.635 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:00.636 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:00.644 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:43:00.647 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:43:00.645+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:43:00.748 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:00.746+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:00.787 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:00.787 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:00.795 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:43:00.849 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:00.847+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:00.938 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:00.938 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:00.947 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:43:00.950 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:00.948+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:01.021 INFO:teuthology.orchestra.run.smithi115.stderr:clear 3.1d pg_upmap_items mapping 2024-09-17T22:43:01.025 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is unset 2024-09-17T22:43:01.050 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:01.047+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:01.063 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:01.064 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:01.074 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:43:01.151 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:43:01.149+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:43:01.216 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:01.216 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:01.226 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:43:01.270 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:01.267+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:01.327 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:01.328 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:01.336 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:43:01.353 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:01.351+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:01.454 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:01.452+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:01.481 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:01.481 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:01.489 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:43:01.544 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:43:01.575 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:01.572+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:01.636 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:01.636 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:01.646 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:43:01.680 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:43:01.680 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:01.677+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:01.698 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:43:01.698 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:43:01.715 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:43:01.715 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:43:01.732 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:43:01.732 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:43:01.749 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:43:01.749 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:43:01.754 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:01.755 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:43:01.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:43:01.755 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:43:01.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.755 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4229.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-17T22:43:01.755 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:43:01.755 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 112.357209121, 2024-09-17T22:43:01.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25708877200000002, 2024-09-17T22:43:01.755 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.755 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.755 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-17T22:43:01.755 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.755 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4229", 2024-09-17T22:43:01.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/3080427038", 2024-09-17T22:43:01.756 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2 2024-09-17T22:43:01.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.756 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:43:01.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:43:01.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394611+0000", 2024-09-17T22:43:01.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3149999999999999e-06 2024-09-17T22:43:01.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394619+0000", 2024-09-17T22:43:01.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240000000000003e-06 2024-09-17T22:43:01.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394632+0000", 2024-09-17T22:43:01.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2921e-05 2024-09-17T22:43:01.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394639+0000", 2024-09-17T22:43:01.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-09-17T22:43:01.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394703+0000", 2024-09-17T22:43:01.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4407999999999997e-05 2024-09-17T22:43:01.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:43:01.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394711+0000", 2024-09-17T22:43:01.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5939999999999996e-06 2024-09-17T22:43:01.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651545+0000", 2024-09-17T22:43:01.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25683409200000001 2024-09-17T22:43:01.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651698+0000", 2024-09-17T22:43:01.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000153678 2024-09-17T22:43:01.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.760 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.760 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38736 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:01.760 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:43:01.760 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.974613116, 2024-09-17T22:43:01.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051395418999999998, 2024-09-17T22:43:01.760 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.760 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.760 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:01.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:01.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:01.761 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38736 2024-09-17T22:43:01.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.761 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:43:01.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:43:01.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:43:01.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6899999999999998e-07 2024-09-17T22:43:01.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777212+0000", 2024-09-17T22:43:01.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1240000000000002e-06 2024-09-17T22:43:01.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777219+0000", 2024-09-17T22:43:01.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5409999999999997e-06 2024-09-17T22:43:01.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777224+0000", 2024-09-17T22:43:01.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:43:01.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.826131+0000", 2024-09-17T22:43:01.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048907369999999999 2024-09-17T22:43:01.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:01.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828457+0000", 2024-09-17T22:43:01.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023256209999999999 2024-09-17T22:43:01.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828601+0000", 2024-09-17T22:43:01.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014438799999999999 2024-09-17T22:43:01.764 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.764 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.765 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.765 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38752 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 673792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:01.765 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:43:01.765 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.974589045000002, 2024-09-17T22:43:01.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051582301999999997, 2024-09-17T22:43:01.765 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.765 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.765 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:01.765 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.765 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:01.765 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:01.766 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38752 2024-09-17T22:43:01.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.766 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:43:01.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:43:01.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:43:01.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3000000000000001e-07 2024-09-17T22:43:01.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777232+0000", 2024-09-17T22:43:01.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9709999999999998e-06 2024-09-17T22:43:01.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777238+0000", 2024-09-17T22:43:01.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7209999999999999e-06 2024-09-17T22:43:01.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777243+0000", 2024-09-17T22:43:01.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5210000000000004e-06 2024-09-17T22:43:01.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828633+0000", 2024-09-17T22:43:01.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051390915000000002 2024-09-17T22:43:01.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:01.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828711+0000", 2024-09-17T22:43:01.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7474999999999999e-05 2024-09-17T22:43:01.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828812+0000", 2024-09-17T22:43:01.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010116900000000001 2024-09-17T22:43:01.769 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.769 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.769 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.769 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38753 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:01.769 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:43:01.769 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.974569771999995, 2024-09-17T22:43:01.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051770444999999998, 2024-09-17T22:43:01.770 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.770 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.770 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:01.770 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.770 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:01.770 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:01.770 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38753 2024-09-17T22:43:01.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.770 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:43:01.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:43:01.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:43:01.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:43:01.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777251+0000", 2024-09-17T22:43:01.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8059999999999999e-06 2024-09-17T22:43:01.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777257+0000", 2024-09-17T22:43:01.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2480000000000001e-06 2024-09-17T22:43:01.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777262+0000", 2024-09-17T22:43:01.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000004e-06 2024-09-17T22:43:01.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828842+0000", 2024-09-17T22:43:01.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051580203999999998 2024-09-17T22:43:01.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:01.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828917+0000", 2024-09-17T22:43:01.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4980000000000004e-05 2024-09-17T22:43:01.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829019+0000", 2024-09-17T22:43:01.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102248 2024-09-17T22:43:01.774 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.774 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.774 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.774 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38766 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:01.774 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:43:01.774 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.974550786999998, 2024-09-17T22:43:01.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051958839999999999, 2024-09-17T22:43:01.774 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.774 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.774 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:01.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:01.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:01.775 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38766 2024-09-17T22:43:01.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.775 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:43:01.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:43:01.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777269+0000", 2024-09-17T22:43:01.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:43:01.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777271+0000", 2024-09-17T22:43:01.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-17T22:43:01.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777276+0000", 2024-09-17T22:43:01.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:43:01.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777281+0000", 2024-09-17T22:43:01.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:43:01.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829055+0000", 2024-09-17T22:43:01.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051773570999999997 2024-09-17T22:43:01.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:01.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829128+0000", 2024-09-17T22:43:01.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3131000000000002e-05 2024-09-17T22:43:01.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829227+0000", 2024-09-17T22:43:01.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8800000000000003e-05 2024-09-17T22:43:01.778 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.778 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.778 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.778 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38767 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:01.778 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:43:01.779 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.974530803999997, 2024-09-17T22:43:01.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137018, 2024-09-17T22:43:01.779 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.779 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.779 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:01.779 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.779 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:01.779 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:01.779 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38767 2024-09-17T22:43:01.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.779 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:43:01.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:43:01.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:43:01.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:43:01.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777290+0000", 2024-09-17T22:43:01.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.88e-06 2024-09-17T22:43:01.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777296+0000", 2024-09-17T22:43:01.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-09-17T22:43:01.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777300+0000", 2024-09-17T22:43:01.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4510000000000002e-06 2024-09-17T22:43:01.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829257+0000", 2024-09-17T22:43:01.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051956975000000002 2024-09-17T22:43:01.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:01.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829328+0000", 2024-09-17T22:43:01.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1026999999999996e-05 2024-09-17T22:43:01.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829425+0000", 2024-09-17T22:43:01.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.679e-05 2024-09-17T22:43:01.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.782 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.783 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.783 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38784 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 675840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:01.783 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:43:01.783 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.974512349999998, 2024-09-17T22:43:01.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052344522999999997, 2024-09-17T22:43:01.783 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.783 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.783 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:01.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:01.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:01.783 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38784 2024-09-17T22:43:01.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.784 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:43:01.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:43:01.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777307+0000", 2024-09-17T22:43:01.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:43:01.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777309+0000", 2024-09-17T22:43:01.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.063e-06 2024-09-17T22:43:01.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777314+0000", 2024-09-17T22:43:01.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-17T22:43:01.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777319+0000", 2024-09-17T22:43:01.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999996e-06 2024-09-17T22:43:01.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829456+0000", 2024-09-17T22:43:01.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137151 2024-09-17T22:43:01.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:01.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829527+0000", 2024-09-17T22:43:01.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1588000000000006e-05 2024-09-17T22:43:01.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829651+0000", 2024-09-17T22:43:01.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012365 2024-09-17T22:43:01.787 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.787 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.787 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.787 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38785 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 749568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:01.787 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:43:01.787 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.974494234999995, 2024-09-17T22:43:01.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052546572999999999, 2024-09-17T22:43:01.787 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.787 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.787 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:01.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:01.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:01.788 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38785 2024-09-17T22:43:01.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.788 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:43:01.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:43:01.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:43:01.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:43:01.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777327+0000", 2024-09-17T22:43:01.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:43:01.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777333+0000", 2024-09-17T22:43:01.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7309999999999998e-06 2024-09-17T22:43:01.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777337+0000", 2024-09-17T22:43:01.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:43:01.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829686+0000", 2024-09-17T22:43:01.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052349159999999999 2024-09-17T22:43:01.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:01.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829766+0000", 2024-09-17T22:43:01.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9326999999999995e-05 2024-09-17T22:43:01.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829871+0000", 2024-09-17T22:43:01.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000105466 2024-09-17T22:43:01.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.791 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.791 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38798 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:01.792 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:43:01.792 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.973860626999993, 2024-09-17T22:43:01.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052117564999999998, 2024-09-17T22:43:01.792 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.792 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.792 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:01.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:01.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:01.792 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38798 2024-09-17T22:43:01.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.792 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:43:01.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:43:01.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777959+0000", 2024-09-17T22:43:01.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9599999999999999e-07 2024-09-17T22:43:01.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777963+0000", 2024-09-17T22:43:01.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2660000000000003e-06 2024-09-17T22:43:01.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777971+0000", 2024-09-17T22:43:01.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3489999999999992e-06 2024-09-17T22:43:01.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777978+0000", 2024-09-17T22:43:01.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:43:01.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829906+0000", 2024-09-17T22:43:01.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051927635999999999 2024-09-17T22:43:01.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:01.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829986+0000", 2024-09-17T22:43:01.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0444000000000004e-05 2024-09-17T22:43:01.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830076+0000", 2024-09-17T22:43:01.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9473000000000002e-05 2024-09-17T22:43:01.795 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.795 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.795 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.796 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38799 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 882688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:01.796 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:43:01.796 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.973829187999996, 2024-09-17T22:43:01.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052359606000000003, 2024-09-17T22:43:01.796 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.796 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.796 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:01.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:01.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:01.796 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38799 2024-09-17T22:43:01.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.796 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:43:01.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:43:01.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:43:01.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8299999999999997e-07 2024-09-17T22:43:01.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777992+0000", 2024-09-17T22:43:01.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.176e-06 2024-09-17T22:43:01.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777998+0000", 2024-09-17T22:43:01.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.173e-06 2024-09-17T22:43:01.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778003+0000", 2024-09-17T22:43:01.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6349999999999997e-06 2024-09-17T22:43:01.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830112+0000", 2024-09-17T22:43:01.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052109366999999997 2024-09-17T22:43:01.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:01.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830192+0000", 2024-09-17T22:43:01.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9814000000000005e-05 2024-09-17T22:43:01.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830349+0000", 2024-09-17T22:43:01.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015695800000000001 2024-09-17T22:43:01.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.800 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.800 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38814 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:01.800 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:43:01.800 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.973545783999995, 2024-09-17T22:43:01.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052274339000000003, 2024-09-17T22:43:01.800 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.800 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.800 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:01.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:01.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:01.801 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38814 2024-09-17T22:43:01.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.801 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:43:01.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:43:01.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:43:01.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:43:01.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778276+0000", 2024-09-17T22:43:01.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2029999999999999e-06 2024-09-17T22:43:01.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778282+0000", 2024-09-17T22:43:01.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0059999999999998e-06 2024-09-17T22:43:01.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778287+0000", 2024-09-17T22:43:01.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0039999999999999e-06 2024-09-17T22:43:01.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830382+0000", 2024-09-17T22:43:01.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052095379999999997 2024-09-17T22:43:01.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:01.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830455+0000", 2024-09-17T22:43:01.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3127000000000006e-05 2024-09-17T22:43:01.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830547+0000", 2024-09-17T22:43:01.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2207999999999995e-05 2024-09-17T22:43:01.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.804 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.804 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38828 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 163840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:01.804 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:43:01.804 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.973363375999995, 2024-09-17T22:43:01.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052466039999999999, 2024-09-17T22:43:01.804 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.805 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.805 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:01.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:01.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:01.805 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38828 2024-09-17T22:43:01.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.805 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:43:01.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:43:01.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778456+0000", 2024-09-17T22:43:01.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7599999999999996e-07 2024-09-17T22:43:01.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778460+0000", 2024-09-17T22:43:01.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8229999999999997e-06 2024-09-17T22:43:01.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778470+0000", 2024-09-17T22:43:01.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8199999999999992e-06 2024-09-17T22:43:01.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778476+0000", 2024-09-17T22:43:01.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2589999999999997e-06 2024-09-17T22:43:01.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830578+0000", 2024-09-17T22:43:01.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052102273999999997 2024-09-17T22:43:01.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:01.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830667+0000", 2024-09-17T22:43:01.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8701000000000003e-05 2024-09-17T22:43:01.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830921+0000", 2024-09-17T22:43:01.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025438700000000002 2024-09-17T22:43:01.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.808 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.809 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38829 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:01.809 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:43:01.809 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.973331978999994, 2024-09-17T22:43:01.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052648882000000001, 2024-09-17T22:43:01.809 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.809 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.809 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:01.809 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.809 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:01.809 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:01.809 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38829 2024-09-17T22:43:01.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.809 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:43:01.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:43:01.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:43:01.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7e-07 2024-09-17T22:43:01.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778490+0000", 2024-09-17T22:43:01.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:43:01.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778495+0000", 2024-09-17T22:43:01.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9329999999999998e-06 2024-09-17T22:43:01.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778500+0000", 2024-09-17T22:43:01.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9509999999999997e-06 2024-09-17T22:43:01.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830958+0000", 2024-09-17T22:43:01.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05245785 2024-09-17T22:43:01.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:01.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831051+0000", 2024-09-17T22:43:01.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2989e-05 2024-09-17T22:43:01.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831136+0000", 2024-09-17T22:43:01.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4448000000000002e-05 2024-09-17T22:43:01.813 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.813 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.813 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.813 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38891 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:01.813 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:43:01.813 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.970019921000002, 2024-09-17T22:43:01.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050847934999999997, 2024-09-17T22:43:01.813 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.813 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.813 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:01.813 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.814 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:01.814 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:01.814 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38891 2024-09-17T22:43:01.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.814 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:43:01.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:43:01.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:43:01.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-09-17T22:43:01.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781802+0000", 2024-09-17T22:43:01.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:43:01.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781810+0000", 2024-09-17T22:43:01.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8019999999999992e-06 2024-09-17T22:43:01.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781816+0000", 2024-09-17T22:43:01.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.285e-06 2024-09-17T22:43:01.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832485+0000", 2024-09-17T22:43:01.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050668462999999997 2024-09-17T22:43:01.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:01.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832565+0000", 2024-09-17T22:43:01.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9692999999999997e-05 2024-09-17T22:43:01.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832647+0000", 2024-09-17T22:43:01.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2224000000000004e-05 2024-09-17T22:43:01.817 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.817 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.817 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.817 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38892 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:01.817 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:43:01.817 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.969991672000006, 2024-09-17T22:43:01.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050946120999999997, 2024-09-17T22:43:01.817 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.818 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.818 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:01.818 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.818 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:01.818 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:01.818 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38892 2024-09-17T22:43:01.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.818 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:43:01.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:43:01.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781828+0000", 2024-09-17T22:43:01.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5400000000000002e-07 2024-09-17T22:43:01.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781830+0000", 2024-09-17T22:43:01.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1959999999999998e-06 2024-09-17T22:43:01.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781837+0000", 2024-09-17T22:43:01.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8310000000000003e-06 2024-09-17T22:43:01.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781843+0000", 2024-09-17T22:43:01.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1e-06 2024-09-17T22:43:01.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832674+0000", 2024-09-17T22:43:01.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050831182000000003 2024-09-17T22:43:01.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:01.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832725+0000", 2024-09-17T22:43:01.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1588000000000001e-05 2024-09-17T22:43:01.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832773+0000", 2024-09-17T22:43:01.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7769999999999998e-05 2024-09-17T22:43:01.821 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.821 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.821 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.821 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38909 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 36864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:01.821 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:43:01.822 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.969968902000005, 2024-09-17T22:43:01.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051046518999999999, 2024-09-17T22:43:01.822 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.822 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.822 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:01.822 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.822 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:01.822 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:01.822 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38909 2024-09-17T22:43:01.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.822 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:43:01.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:43:01.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:43:01.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:43:01.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781852+0000", 2024-09-17T22:43:01.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0590000000000001e-06 2024-09-17T22:43:01.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781859+0000", 2024-09-17T22:43:01.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3380000000000001e-06 2024-09-17T22:43:01.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781864+0000", 2024-09-17T22:43:01.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:43:01.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832804+0000", 2024-09-17T22:43:01.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050939614000000001 2024-09-17T22:43:01.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:01.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832845+0000", 2024-09-17T22:43:01.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1014999999999999e-05 2024-09-17T22:43:01.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832896+0000", 2024-09-17T22:43:01.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1836999999999999e-05 2024-09-17T22:43:01.825 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.825 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.826 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.826 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38910 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:01.826 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:43:01.826 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.969948591999994, 2024-09-17T22:43:01.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051216818999999997, 2024-09-17T22:43:01.826 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.826 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.826 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:01.826 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.826 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:01.826 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:01.826 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38910 2024-09-17T22:43:01.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.826 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:43:01.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:43:01.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781871+0000", 2024-09-17T22:43:01.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:43:01.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781873+0000", 2024-09-17T22:43:01.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:43:01.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781879+0000", 2024-09-17T22:43:01.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0700000000000003e-06 2024-09-17T22:43:01.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781884+0000", 2024-09-17T22:43:01.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-09-17T22:43:01.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832932+0000", 2024-09-17T22:43:01.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051048219999999998 2024-09-17T22:43:01.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:01.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833001+0000", 2024-09-17T22:43:01.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8504999999999995e-05 2024-09-17T22:43:01.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833087+0000", 2024-09-17T22:43:01.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6464000000000004e-05 2024-09-17T22:43:01.830 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.830 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.830 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.830 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38923 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:01.830 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:43:01.830 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.969928272000004, 2024-09-17T22:43:01.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051379573999999997, 2024-09-17T22:43:01.830 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.830 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.831 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:01.831 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.831 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:01.831 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:01.831 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38923 2024-09-17T22:43:01.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.831 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:43:01.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:43:01.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:43:01.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:43:01.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781893+0000", 2024-09-17T22:43:01.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-09-17T22:43:01.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781900+0000", 2024-09-17T22:43:01.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3860000000000004e-06 2024-09-17T22:43:01.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781904+0000", 2024-09-17T22:43:01.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9300000000000002e-06 2024-09-17T22:43:01.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833115+0000", 2024-09-17T22:43:01.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051210845999999997 2024-09-17T22:43:01.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:01.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833188+0000", 2024-09-17T22:43:01.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2276999999999999e-05 2024-09-17T22:43:01.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833270+0000", 2024-09-17T22:43:01.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2501000000000002e-05 2024-09-17T22:43:01.834 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.834 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.834 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.834 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38941 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:01.834 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:43:01.834 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 91.969135737000002, 2024-09-17T22:43:01.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050969459000000002, 2024-09-17T22:43:01.835 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.835 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.835 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:01.835 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.835 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:01.835 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:01.835 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38941 2024-09-17T22:43:01.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.835 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:43:01.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:43:01.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782684+0000", 2024-09-17T22:43:01.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:43:01.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782686+0000", 2024-09-17T22:43:01.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.396e-06 2024-09-17T22:43:01.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782693+0000", 2024-09-17T22:43:01.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1999999999999997e-06 2024-09-17T22:43:01.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782699+0000", 2024-09-17T22:43:01.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7910000000000001e-06 2024-09-17T22:43:01.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833471+0000", 2024-09-17T22:43:01.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050772333000000003 2024-09-17T22:43:01.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:01.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833536+0000", 2024-09-17T22:43:01.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4380000000000004e-05 2024-09-17T22:43:01.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833653+0000", 2024-09-17T22:43:01.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011691799999999999 2024-09-17T22:43:01.838 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.838 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.838 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.838 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:151189 3.14 3:2b696b13:::rbd_data.115f14f2d1d5.0000000000000009:e [read 3127296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:01.839 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:43:01.839 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 72.038914992000002, 2024-09-17T22:43:01.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.054254700000000003, 2024-09-17T22:43:01.839 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:01.839 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:01.839 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:01.839 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:01.839 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:01.839 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:01.839 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 151189 2024-09-17T22:43:01.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.839 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:01.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:01.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:43:01.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:01.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:43:01.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:01.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:01.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713051+0000", 2024-09-17T22:43:01.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014722100000000001 2024-09-17T22:43:01.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:01.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713055+0000", 2024-09-17T22:43:01.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.16e-06 2024-09-17T22:43:01.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:01.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713065+0000", 2024-09-17T22:43:01.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8390000000000002e-06 2024-09-17T22:43:01.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:01.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713073+0000", 2024-09-17T22:43:01.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0390000000000005e-06 2024-09-17T22:43:01.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:01.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762644+0000", 2024-09-17T22:43:01.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049571160000000003 2024-09-17T22:43:01.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:01.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762739+0000", 2024-09-17T22:43:01.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4565e-05 2024-09-17T22:43:01.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:01.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:01.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:01.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.767159+0000", 2024-09-17T22:43:01.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044197159999999997 2024-09-17T22:43:01.842 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.843 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.843 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.843 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:01.843 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:01.843 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:01.844 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:43:01.845 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:43:01.845 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:43:01.845 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:43:01.846 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:43:01.846 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:43:01.846 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:43:01.847 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:43:01.847 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:43:01.847 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:01.841+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:01.945 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:01.942+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:01.954 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:01.954 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:01.954 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:43:01.954 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:01.962 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:43:02.046 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:02.044+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:02.121 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:02.121 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:02.122 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:43:02.122 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:43:02.122 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:02.131 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:43:02.173 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:02.170+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:02.248 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:02.245+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:02.276 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:02.277 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:43:02.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:43:02.277 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:43:02.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.277 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158673 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.277 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:43:02.277 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.032074682000001, 2024-09-17T22:43:02.277 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039400614, 2024-09-17T22:43:02.277 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.277 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.277 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.278 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.278 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.278 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.278 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158673 2024-09-17T22:43:02.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.278 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:43:02.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:43:02.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:43:02.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.04e-07 2024-09-17T22:43:02.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241563+0000", 2024-09-17T22:43:02.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5799999999999998e-07 2024-09-17T22:43:02.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241566+0000", 2024-09-17T22:43:02.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8279999999999999e-06 2024-09-17T22:43:02.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241568+0000", 2024-09-17T22:43:02.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2709999999999999e-06 2024-09-17T22:43:02.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280954+0000", 2024-09-17T22:43:02.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039386051999999998 2024-09-17T22:43:02.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.281 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.281 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280962+0000", 2024-09-17T22:43:02.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3010000000000007e-06 2024-09-17T22:43:02.281 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.281 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.281 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.282 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158675 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.282 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:43:02.282 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.032065391000003, 2024-09-17T22:43:02.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039408033000000002, 2024-09-17T22:43:02.282 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.282 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.282 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.282 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.282 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.282 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.282 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158675 2024-09-17T22:43:02.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.282 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:43:02.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:43:02.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:43:02.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.36e-07 2024-09-17T22:43:02.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241572+0000", 2024-09-17T22:43:02.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6599999999999994e-07 2024-09-17T22:43:02.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241575+0000", 2024-09-17T22:43:02.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8389999999999998e-06 2024-09-17T22:43:02.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241577+0000", 2024-09-17T22:43:02.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4119999999999999e-06 2024-09-17T22:43:02.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280975+0000", 2024-09-17T22:43:02.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039397205999999997 2024-09-17T22:43:02.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280979+0000", 2024-09-17T22:43:02.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.374e-06 2024-09-17T22:43:02.285 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.286 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.286 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.286 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158677 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.286 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:43:02.286 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.032042947999997, 2024-09-17T22:43:02.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039409307999999997, 2024-09-17T22:43:02.286 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.286 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.286 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.286 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.286 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.286 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.287 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158677 2024-09-17T22:43:02.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.287 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:43:02.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:43:02.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241594+0000", 2024-09-17T22:43:02.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:43:02.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241595+0000", 2024-09-17T22:43:02.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2729999999999999e-06 2024-09-17T22:43:02.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241602+0000", 2024-09-17T22:43:02.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:43:02.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241606+0000", 2024-09-17T22:43:02.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3019999999999999e-06 2024-09-17T22:43:02.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280994+0000", 2024-09-17T22:43:02.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039388484000000001 2024-09-17T22:43:02.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281003+0000", 2024-09-17T22:43:02.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5459999999999992e-06 2024-09-17T22:43:02.290 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.290 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.290 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.290 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158679 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.290 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:43:02.290 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.03202727, 2024-09-17T22:43:02.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039413403999999999, 2024-09-17T22:43:02.290 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.290 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.290 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.290 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.291 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.291 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.291 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158679 2024-09-17T22:43:02.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.291 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.291 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.291 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:43:02.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.291 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.291 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:43:02.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:43:02.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4999999999999999e-07 2024-09-17T22:43:02.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241613+0000", 2024-09-17T22:43:02.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.207e-06 2024-09-17T22:43:02.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241616+0000", 2024-09-17T22:43:02.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.275e-06 2024-09-17T22:43:02.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241618+0000", 2024-09-17T22:43:02.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:43:02.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281017+0000", 2024-09-17T22:43:02.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039398385000000001 2024-09-17T22:43:02.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281022+0000", 2024-09-17T22:43:02.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.896e-06 2024-09-17T22:43:02.294 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.294 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.294 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.294 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158681 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.294 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:43:02.294 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.032015091999995, 2024-09-17T22:43:02.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039425292000000001, 2024-09-17T22:43:02.294 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.295 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.295 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.295 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.295 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.295 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.295 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158681 2024-09-17T22:43:02.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.295 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.295 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.295 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:43:02.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.295 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:43:02.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:43:02.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:43:02.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241622+0000", 2024-09-17T22:43:02.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:43:02.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241625+0000", 2024-09-17T22:43:02.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9629999999999998e-06 2024-09-17T22:43:02.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241628+0000", 2024-09-17T22:43:02.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4339999999999999e-06 2024-09-17T22:43:02.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281039+0000", 2024-09-17T22:43:02.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039410672000000001 2024-09-17T22:43:02.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281047+0000", 2024-09-17T22:43:02.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9969999999999998e-06 2024-09-17T22:43:02.298 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.298 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.298 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.298 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158682 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.298 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:43:02.298 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.032005807999994, 2024-09-17T22:43:02.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039436455000000002, 2024-09-17T22:43:02.299 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.299 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.299 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.299 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.299 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.299 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.299 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158682 2024-09-17T22:43:02.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.299 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:43:02.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:43:02.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:43:02.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:43:02.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241632+0000", 2024-09-17T22:43:02.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:43:02.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241635+0000", 2024-09-17T22:43:02.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8940000000000002e-06 2024-09-17T22:43:02.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241637+0000", 2024-09-17T22:43:02.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3659999999999999e-06 2024-09-17T22:43:02.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281061+0000", 2024-09-17T22:43:02.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039423664999999997 2024-09-17T22:43:02.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281067+0000", 2024-09-17T22:43:02.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.285e-06 2024-09-17T22:43:02.302 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.302 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.302 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.303 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158684 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.303 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:43:02.303 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.031996461999995, 2024-09-17T22:43:02.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462980000000002, 2024-09-17T22:43:02.303 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.303 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.303 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.303 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.303 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.303 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.303 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158684 2024-09-17T22:43:02.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.303 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:43:02.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:43:02.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.304 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.304 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:43:02.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:43:02.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241641+0000", 2024-09-17T22:43:02.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:43:02.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241644+0000", 2024-09-17T22:43:02.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8870000000000001e-06 2024-09-17T22:43:02.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241646+0000", 2024-09-17T22:43:02.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3530000000000002e-06 2024-09-17T22:43:02.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281093+0000", 2024-09-17T22:43:02.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039446947000000003 2024-09-17T22:43:02.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281103+0000", 2024-09-17T22:43:02.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6299999999999993e-06 2024-09-17T22:43:02.306 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.306 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.307 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.307 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158686 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.307 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:43:02.307 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.031987291999997, 2024-09-17T22:43:02.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039474214000000001, 2024-09-17T22:43:02.307 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.307 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.307 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.307 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.307 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.307 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.307 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158686 2024-09-17T22:43:02.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.308 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:43:02.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:43:02.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:43:02.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:43:02.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241650+0000", 2024-09-17T22:43:02.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2999999999999999e-07 2024-09-17T22:43:02.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241653+0000", 2024-09-17T22:43:02.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9689999999999999e-06 2024-09-17T22:43:02.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241655+0000", 2024-09-17T22:43:02.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3460000000000001e-06 2024-09-17T22:43:02.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281118+0000", 2024-09-17T22:43:02.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462876000000001 2024-09-17T22:43:02.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281123+0000", 2024-09-17T22:43:02.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8840000000000002e-06 2024-09-17T22:43:02.311 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.311 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.311 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.311 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158688 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.311 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:43:02.311 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.031978061000004, 2024-09-17T22:43:02.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039490458999999999, 2024-09-17T22:43:02.311 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.311 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.311 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.312 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.312 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.312 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.312 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158688 2024-09-17T22:43:02.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.312 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:43:02.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:43:02.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:43:02.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:43:02.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.313 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.313 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241659+0000", 2024-09-17T22:43:02.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3699999999999999e-07 2024-09-17T22:43:02.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241662+0000", 2024-09-17T22:43:02.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8499999999999998e-06 2024-09-17T22:43:02.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241665+0000", 2024-09-17T22:43:02.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.356e-06 2024-09-17T22:43:02.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281141+0000", 2024-09-17T22:43:02.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039476031000000002 2024-09-17T22:43:02.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281149+0000", 2024-09-17T22:43:02.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0709999999999995e-06 2024-09-17T22:43:02.315 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.315 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.315 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.315 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158690 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.315 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:43:02.315 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.031969001999997, 2024-09-17T22:43:02.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039502378999999997, 2024-09-17T22:43:02.316 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.316 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.316 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.316 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.316 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.316 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.316 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158690 2024-09-17T22:43:02.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.316 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:43:02.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:43:02.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241668+0000", 2024-09-17T22:43:02.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:43:02.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241669+0000", 2024-09-17T22:43:02.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1009999999999999e-06 2024-09-17T22:43:02.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241672+0000", 2024-09-17T22:43:02.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9059999999999998e-06 2024-09-17T22:43:02.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.318 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.318 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241674+0000", 2024-09-17T22:43:02.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4439999999999998e-06 2024-09-17T22:43:02.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281162+0000", 2024-09-17T22:43:02.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039488189 2024-09-17T22:43:02.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281170+0000", 2024-09-17T22:43:02.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5310000000000004e-06 2024-09-17T22:43:02.319 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.319 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.319 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.320 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158692 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.320 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:43:02.320 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.031959568999994, 2024-09-17T22:43:02.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039531064999999997, 2024-09-17T22:43:02.320 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.320 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.320 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.320 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.320 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.320 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.320 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158692 2024-09-17T22:43:02.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.320 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:43:02.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:43:02.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:43:02.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:43:02.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241678+0000", 2024-09-17T22:43:02.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0800000000000003e-07 2024-09-17T22:43:02.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.322 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.322 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241681+0000", 2024-09-17T22:43:02.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8729999999999999e-06 2024-09-17T22:43:02.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241683+0000", 2024-09-17T22:43:02.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.447e-06 2024-09-17T22:43:02.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281199+0000", 2024-09-17T22:43:02.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039515388999999998 2024-09-17T22:43:02.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281208+0000", 2024-09-17T22:43:02.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2399999999999996e-06 2024-09-17T22:43:02.323 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.324 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.324 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.324 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158694 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.324 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:43:02.324 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.031950398000006, 2024-09-17T22:43:02.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039543986000000003, 2024-09-17T22:43:02.324 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.324 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.324 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.324 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.324 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.325 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.325 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158694 2024-09-17T22:43:02.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.325 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:43:02.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:43:02.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:43:02.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:43:02.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241687+0000", 2024-09-17T22:43:02.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:43:02.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.326 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241691+0000", 2024-09-17T22:43:02.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.811e-06 2024-09-17T22:43:02.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241693+0000", 2024-09-17T22:43:02.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.385e-06 2024-09-17T22:43:02.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281223+0000", 2024-09-17T22:43:02.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039529332 2024-09-17T22:43:02.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281230+0000", 2024-09-17T22:43:02.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2139999999999999e-06 2024-09-17T22:43:02.328 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.328 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.328 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.328 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158696 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.328 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:43:02.328 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.03194001, 2024-09-17T22:43:02.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039570458000000003, 2024-09-17T22:43:02.328 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.328 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.328 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.329 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.329 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.329 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.329 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158696 2024-09-17T22:43:02.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.329 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:43:02.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:43:02.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:43:02.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:43:02.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:43:02.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.33e-07 2024-09-17T22:43:02.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241700+0000", 2024-09-17T22:43:02.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8949999999999998e-06 2024-09-17T22:43:02.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241703+0000", 2024-09-17T22:43:02.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3159999999999999e-06 2024-09-17T22:43:02.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.331 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.331 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281257+0000", 2024-09-17T22:43:02.331 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039553862000000002 2024-09-17T22:43:02.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.332 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.332 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281267+0000", 2024-09-17T22:43:02.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.024e-05 2024-09-17T22:43:02.332 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.332 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.332 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.332 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.332 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158698 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.333 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:43:02.333 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.031929931999997, 2024-09-17T22:43:02.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039576493999999997, 2024-09-17T22:43:02.333 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.333 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.333 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.333 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.333 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.333 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.333 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158698 2024-09-17T22:43:02.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.333 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:43:02.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:43:02.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241707+0000", 2024-09-17T22:43:02.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:43:02.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241708+0000", 2024-09-17T22:43:02.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5000000000000001e-07 2024-09-17T22:43:02.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241711+0000", 2024-09-17T22:43:02.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9670000000000001e-06 2024-09-17T22:43:02.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241713+0000", 2024-09-17T22:43:02.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2689999999999998e-06 2024-09-17T22:43:02.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281278+0000", 2024-09-17T22:43:02.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039565206999999998 2024-09-17T22:43:02.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281283+0000", 2024-09-17T22:43:02.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:43:02.336 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.336 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.337 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.337 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158700 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.337 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:43:02.337 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.031920721000006, 2024-09-17T22:43:02.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039604292999999999, 2024-09-17T22:43:02.337 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.337 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.337 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.337 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.337 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.337 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.338 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158700 2024-09-17T22:43:02.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.338 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:43:02.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:43:02.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:43:02.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:43:02.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241717+0000", 2024-09-17T22:43:02.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.78e-07 2024-09-17T22:43:02.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241720+0000", 2024-09-17T22:43:02.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8119999999999999e-06 2024-09-17T22:43:02.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241722+0000", 2024-09-17T22:43:02.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-17T22:43:02.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281311+0000", 2024-09-17T22:43:02.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039588894999999999 2024-09-17T22:43:02.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281320+0000", 2024-09-17T22:43:02.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9989999999999997e-06 2024-09-17T22:43:02.341 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.341 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.341 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.341 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158702 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.341 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:43:02.341 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.031909399, 2024-09-17T22:43:02.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039616142, 2024-09-17T22:43:02.341 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.341 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.341 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.341 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.342 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.342 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.342 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158702 2024-09-17T22:43:02.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.342 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:43:02.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:43:02.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:43:02.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4400000000000001e-07 2024-09-17T22:43:02.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241728+0000", 2024-09-17T22:43:02.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0839999999999999e-06 2024-09-17T22:43:02.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241731+0000", 2024-09-17T22:43:02.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0690000000000002e-06 2024-09-17T22:43:02.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241734+0000", 2024-09-17T22:43:02.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4159999999999998e-06 2024-09-17T22:43:02.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281334+0000", 2024-09-17T22:43:02.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039600110000000001 2024-09-17T22:43:02.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281343+0000", 2024-09-17T22:43:02.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2189999999999993e-06 2024-09-17T22:43:02.345 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.345 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.345 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.345 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158704 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.345 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:43:02.345 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.031899534999994, 2024-09-17T22:43:02.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039641485999999997, 2024-09-17T22:43:02.346 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.346 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.346 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.346 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158704 2024-09-17T22:43:02.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.346 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:43:02.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:43:02.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:43:02.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:43:02.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241738+0000", 2024-09-17T22:43:02.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0589999999999999e-06 2024-09-17T22:43:02.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241741+0000", 2024-09-17T22:43:02.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.943e-06 2024-09-17T22:43:02.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241744+0000", 2024-09-17T22:43:02.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4819999999999999e-06 2024-09-17T22:43:02.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281370+0000", 2024-09-17T22:43:02.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039625661 2024-09-17T22:43:02.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281378+0000", 2024-09-17T22:43:02.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1310000000000006e-06 2024-09-17T22:43:02.349 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.349 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.350 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.350 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158705 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.350 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:43:02.350 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.031888984999995, 2024-09-17T22:43:02.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039652637999999997, 2024-09-17T22:43:02.350 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.350 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.350 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.350 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158705 2024-09-17T22:43:02.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.351 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:43:02.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:43:02.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:43:02.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-07 2024-09-17T22:43:02.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:43:02.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0599999999999999e-07 2024-09-17T22:43:02.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241751+0000", 2024-09-17T22:43:02.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.926e-06 2024-09-17T22:43:02.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241754+0000", 2024-09-17T22:43:02.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4360000000000001e-06 2024-09-17T22:43:02.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281392+0000", 2024-09-17T22:43:02.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039638566 2024-09-17T22:43:02.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281400+0000", 2024-09-17T22:43:02.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.571e-06 2024-09-17T22:43:02.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.354 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.354 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158707 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.354 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:43:02.354 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.031867883999993, 2024-09-17T22:43:02.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039654809999999999, 2024-09-17T22:43:02.354 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.354 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.354 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.354 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.355 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158707 2024-09-17T22:43:02.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.355 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:43:02.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:43:02.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241769+0000", 2024-09-17T22:43:02.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4299999999999999e-07 2024-09-17T22:43:02.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241770+0000", 2024-09-17T22:43:02.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3769999999999999e-06 2024-09-17T22:43:02.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241773+0000", 2024-09-17T22:43:02.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2109999999999998e-06 2024-09-17T22:43:02.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241776+0000", 2024-09-17T22:43:02.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5069999999999999e-06 2024-09-17T22:43:02.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281415+0000", 2024-09-17T22:43:02.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039639499000000002 2024-09-17T22:43:02.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281423+0000", 2024-09-17T22:43:02.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8730000000000007e-06 2024-09-17T22:43:02.358 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.358 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.358 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.358 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158709 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.358 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:43:02.358 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.031857274999993, 2024-09-17T22:43:02.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039661937000000001, 2024-09-17T22:43:02.358 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.359 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.359 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:02.359 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.359 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.359 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.359 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158709 2024-09-17T22:43:02.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.359 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.359 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.359 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:43:02.359 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.359 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:43:02.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242843+0000", 2024-09-17T22:43:02.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010634780000000001 2024-09-17T22:43:02.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.360 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.360 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.360 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242845+0000", 2024-09-17T22:43:02.360 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7209999999999999e-06 2024-09-17T22:43:02.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242851+0000", 2024-09-17T22:43:02.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6949999999999997e-06 2024-09-17T22:43:02.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242855+0000", 2024-09-17T22:43:02.361 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7960000000000002e-06 2024-09-17T22:43:02.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.361 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.361 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.361 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281434+0000", 2024-09-17T22:43:02.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038579519 2024-09-17T22:43:02.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.362 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.362 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.362 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281441+0000", 2024-09-17T22:43:02.362 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7279999999999998e-06 2024-09-17T22:43:02.362 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.362 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.362 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.362 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.362 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.362 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:02.364 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:43:02.364 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:02.361+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:02.465 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:02.462+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:02.468 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:02.468 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:02.468 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:43:02.468 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:02.477 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:43:02.566 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:02.563+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:02.622 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:02.622 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:02.622 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:43:02.622 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:43:02.622 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:02.631 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:43:02.671 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:02.669+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:02.768 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:02.765+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:02.777 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:43:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:43:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:43:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157229 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4073472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:43:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.627571363000001, 2024-09-17T22:43:02.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041957959000000003, 2024-09-17T22:43:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157229 2024-09-17T22:43:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:43:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:43:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146207+0000", 2024-09-17T22:43:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0299999999999999e-07 2024-09-17T22:43:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.779 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146215+0000", 2024-09-17T22:43:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3059999999999998e-06 2024-09-17T22:43:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.780 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146222+0000", 2024-09-17T22:43:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1180000000000003e-06 2024-09-17T22:43:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146227+0000", 2024-09-17T22:43:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1749999999999998e-06 2024-09-17T22:43:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.187985+0000", 2024-09-17T22:43:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041758391999999998 2024-09-17T22:43:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.781 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188065+0000", 2024-09-17T22:43:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0290000000000005e-05 2024-09-17T22:43:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188164+0000", 2024-09-17T22:43:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9074999999999996e-05 2024-09-17T22:43:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.782 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157231 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4079616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:43:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.627433738999997, 2024-09-17T22:43:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042034581000000001, 2024-09-17T22:43:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157231 2024-09-17T22:43:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.783 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:43:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:43:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146345+0000", 2024-09-17T22:43:02.784 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:43:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146347+0000", 2024-09-17T22:43:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.357e-06 2024-09-17T22:43:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146352+0000", 2024-09-17T22:43:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T22:43:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.785 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146361+0000", 2024-09-17T22:43:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0219999999999993e-06 2024-09-17T22:43:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188199+0000", 2024-09-17T22:43:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041838130000000001 2024-09-17T22:43:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188285+0000", 2024-09-17T22:43:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5160000000000005e-05 2024-09-17T22:43:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.786 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188379+0000", 2024-09-17T22:43:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3949000000000005e-05 2024-09-17T22:43:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157233 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4085760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:43:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.627217590000001, 2024-09-17T22:43:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042008960999999997, 2024-09-17T22:43:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.787 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157233 2024-09-17T22:43:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:43:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.788 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:43:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146561+0000", 2024-09-17T22:43:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6300000000000005e-07 2024-09-17T22:43:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146564+0000", 2024-09-17T22:43:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-17T22:43:02.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146575+0000", 2024-09-17T22:43:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0891e-05 2024-09-17T22:43:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146579+0000", 2024-09-17T22:43:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.284e-06 2024-09-17T22:43:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.790 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188407+0000", 2024-09-17T22:43:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041828390999999999 2024-09-17T22:43:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188476+0000", 2024-09-17T22:43:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9129999999999997e-05 2024-09-17T22:43:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.791 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188569+0000", 2024-09-17T22:43:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2669000000000003e-05 2024-09-17T22:43:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.792 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157235 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4091904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:43:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.627023593000004, 2024-09-17T22:43:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042047517, 2024-09-17T22:43:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.793 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157235 2024-09-17T22:43:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:43:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:43:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146755+0000", 2024-09-17T22:43:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0900000000000001e-07 2024-09-17T22:43:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146758+0000", 2024-09-17T22:43:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.399e-06 2024-09-17T22:43:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146768+0000", 2024-09-17T22:43:02.795 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4730000000000006e-06 2024-09-17T22:43:02.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146774+0000", 2024-09-17T22:43:02.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6939999999999998e-06 2024-09-17T22:43:02.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.796 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188616+0000", 2024-09-17T22:43:02.796 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041841481999999999 2024-09-17T22:43:02.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.796 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.796 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188700+0000", 2024-09-17T22:43:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4227999999999994e-05 2024-09-17T22:43:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188802+0000", 2024-09-17T22:43:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101532 2024-09-17T22:43:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157237 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4098048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.797 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:43:02.798 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.626993322000004, 2024-09-17T22:43:02.798 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165452999999999, 2024-09-17T22:43:02.798 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.798 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.798 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.798 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.798 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.798 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.798 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157237 2024-09-17T22:43:02.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.798 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.798 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:43:02.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.799 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:43:02.799 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.799 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.799 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.799 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146785+0000", 2024-09-17T22:43:02.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6800000000000001e-07 2024-09-17T22:43:02.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146787+0000", 2024-09-17T22:43:02.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.046e-06 2024-09-17T22:43:02.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.800 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.800 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.800 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146794+0000", 2024-09-17T22:43:02.800 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9249999999999998e-06 2024-09-17T22:43:02.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146799+0000", 2024-09-17T22:43:02.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9289999999999997e-06 2024-09-17T22:43:02.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.801 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.801 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.801 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188844+0000", 2024-09-17T22:43:02.801 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042045386999999997 2024-09-17T22:43:02.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188888+0000", 2024-09-17T22:43:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3760000000000001e-05 2024-09-17T22:43:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188950+0000", 2024-09-17T22:43:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1938000000000002e-05 2024-09-17T22:43:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.802 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157239 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4104192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.803 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:43:02.803 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.626964935999993, 2024-09-17T22:43:02.803 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042347193999999998, 2024-09-17T22:43:02.803 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.803 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.803 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.803 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.803 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.803 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.803 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157239 2024-09-17T22:43:02.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:43:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:43:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.804 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:43:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:43:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146816+0000", 2024-09-17T22:43:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4700000000000001e-06 2024-09-17T22:43:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146822+0000", 2024-09-17T22:43:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.579e-06 2024-09-17T22:43:02.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146826+0000", 2024-09-17T22:43:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3780000000000001e-06 2024-09-17T22:43:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188991+0000", 2024-09-17T22:43:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165553000000001 2024-09-17T22:43:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189068+0000", 2024-09-17T22:43:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6548e-05 2024-09-17T22:43:02.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189160+0000", 2024-09-17T22:43:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2163000000000005e-05 2024-09-17T22:43:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157242 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4110336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:43:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.62692715, 2024-09-17T22:43:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042499783999999999, 2024-09-17T22:43:02.807 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157242 2024-09-17T22:43:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:43:02.808 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:43:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:43:02.809 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:43:02.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146852+0000", 2024-09-17T22:43:02.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.37e-06 2024-09-17T22:43:02.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.810 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.810 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.810 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146856+0000", 2024-09-17T22:43:02.810 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1690000000000002e-06 2024-09-17T22:43:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146863+0000", 2024-09-17T22:43:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9920000000000001e-06 2024-09-17T22:43:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189193+0000", 2024-09-17T22:43:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042329355999999999 2024-09-17T22:43:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189246+0000", 2024-09-17T22:43:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3643000000000001e-05 2024-09-17T22:43:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189350+0000", 2024-09-17T22:43:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010394 2024-09-17T22:43:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.812 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157244 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4116480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:43:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.626907939000006, 2024-09-17T22:43:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042675684999999998, 2024-09-17T22:43:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.813 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157244 2024-09-17T22:43:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:43:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:43:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:43:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1e-07 2024-09-17T22:43:02.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146871+0000", 2024-09-17T22:43:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.189e-06 2024-09-17T22:43:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146877+0000", 2024-09-17T22:43:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8100000000000003e-06 2024-09-17T22:43:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.816 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146880+0000", 2024-09-17T22:43:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.83e-06 2024-09-17T22:43:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189388+0000", 2024-09-17T22:43:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042507637000000001 2024-09-17T22:43:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189443+0000", 2024-09-17T22:43:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5742999999999998e-05 2024-09-17T22:43:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189545+0000", 2024-09-17T22:43:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102166 2024-09-17T22:43:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157246 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4122624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:43:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.626878887000004, 2024-09-17T22:43:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042847083000000001, 2024-09-17T22:43:02.818 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157246 2024-09-17T22:43:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:43:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:43:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:43:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0100000000000001e-07 2024-09-17T22:43:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146901+0000", 2024-09-17T22:43:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.531e-06 2024-09-17T22:43:02.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146905+0000", 2024-09-17T22:43:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0759999999999996e-06 2024-09-17T22:43:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146908+0000", 2024-09-17T22:43:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.083e-06 2024-09-17T22:43:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189585+0000", 2024-09-17T22:43:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042677009000000002 2024-09-17T22:43:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189656+0000", 2024-09-17T22:43:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0712999999999999e-05 2024-09-17T22:43:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189746+0000", 2024-09-17T22:43:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0370000000000004e-05 2024-09-17T22:43:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157247 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4128768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:43:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.626807051, 2024-09-17T22:43:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043016916000000002, 2024-09-17T22:43:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.823 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157247 2024-09-17T22:43:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:43:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:43:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147071+0000", 2024-09-17T22:43:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100769 2024-09-17T22:43:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147081+0000", 2024-09-17T22:43:02.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9229999999999997e-06 2024-09-17T22:43:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147090+0000", 2024-09-17T22:43:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.935e-06 2024-09-17T22:43:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147097+0000", 2024-09-17T22:43:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T22:43:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189787+0000", 2024-09-17T22:43:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042690525 2024-09-17T22:43:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189884+0000", 2024-09-17T22:43:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6858e-05 2024-09-17T22:43:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189988+0000", 2024-09-17T22:43:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103336 2024-09-17T22:43:02.827 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157250 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4134912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:43:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.626663590000007, 2024-09-17T22:43:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043099446, 2024-09-17T22:43:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.828 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.829 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.829 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.829 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157250 2024-09-17T22:43:02.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.829 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.829 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:43:02.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.829 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:43:02.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147235+0000", 2024-09-17T22:43:02.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120975 2024-09-17T22:43:02.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147238+0000", 2024-09-17T22:43:02.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5289999999999998e-06 2024-09-17T22:43:02.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147244+0000", 2024-09-17T22:43:02.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6050000000000003e-06 2024-09-17T22:43:02.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147249+0000", 2024-09-17T22:43:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4939999999999997e-06 2024-09-17T22:43:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190020+0000", 2024-09-17T22:43:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042771300999999998 2024-09-17T22:43:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190102+0000", 2024-09-17T22:43:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1600000000000005e-05 2024-09-17T22:43:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190214+0000", 2024-09-17T22:43:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011194200000000001 2024-09-17T22:43:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157252 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4141056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.833 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:43:02.834 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.626520920999994, 2024-09-17T22:43:02.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043165508999999998, 2024-09-17T22:43:02.834 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.834 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.834 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.834 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157252 2024-09-17T22:43:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:43:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:43:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147402+0000", 2024-09-17T22:43:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014475600000000001 2024-09-17T22:43:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147411+0000", 2024-09-17T22:43:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4350000000000003e-06 2024-09-17T22:43:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147420+0000", 2024-09-17T22:43:02.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4790000000000002e-06 2024-09-17T22:43:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147427+0000", 2024-09-17T22:43:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5400000000000001e-06 2024-09-17T22:43:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190254+0000", 2024-09-17T22:43:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042826731 2024-09-17T22:43:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190326+0000", 2024-09-17T22:43:02.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1798999999999995e-05 2024-09-17T22:43:02.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.838 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.838 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190422+0000", 2024-09-17T22:43:02.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6768999999999995e-05 2024-09-17T22:43:02.838 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.838 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.839 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.839 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157253 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4147200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.839 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:43:02.839 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.626334555, 2024-09-17T22:43:02.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043157752000000001, 2024-09-17T22:43:02.839 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.840 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.840 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.840 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.840 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157253 2024-09-17T22:43:02.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.840 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:43:02.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:43:02.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147590+0000", 2024-09-17T22:43:02.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014709 2024-09-17T22:43:02.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147593+0000", 2024-09-17T22:43:02.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5840000000000002e-06 2024-09-17T22:43:02.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147598+0000", 2024-09-17T22:43:02.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2469999999999999e-06 2024-09-17T22:43:02.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147602+0000", 2024-09-17T22:43:02.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0049999999999998e-06 2024-09-17T22:43:02.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190475+0000", 2024-09-17T22:43:02.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872767999999999 2024-09-17T22:43:02.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190528+0000", 2024-09-17T22:43:02.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2889999999999997e-05 2024-09-17T22:43:02.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190601+0000", 2024-09-17T22:43:02.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3168000000000001e-05 2024-09-17T22:43:02.845 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.845 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.845 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.845 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157255 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4153344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.846 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:43:02.846 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.626168574000005, 2024-09-17T22:43:02.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043132251000000003, 2024-09-17T22:43:02.846 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.846 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.846 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.846 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.846 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.846 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.846 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157255 2024-09-17T22:43:02.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.846 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:43:02.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:43:02.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.847 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.847 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147820+0000", 2024-09-17T22:43:02.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021123600000000001 2024-09-17T22:43:02.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147824+0000", 2024-09-17T22:43:02.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6780000000000002e-06 2024-09-17T22:43:02.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147833+0000", 2024-09-17T22:43:02.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4250000000000004e-06 2024-09-17T22:43:02.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147842+0000", 2024-09-17T22:43:02.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0679999999999997e-06 2024-09-17T22:43:02.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190632+0000", 2024-09-17T22:43:02.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042790863999999998 2024-09-17T22:43:02.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190674+0000", 2024-09-17T22:43:02.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1804000000000002e-05 2024-09-17T22:43:02.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190741+0000", 2024-09-17T22:43:02.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7175999999999995e-05 2024-09-17T22:43:02.850 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.850 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.850 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.851 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157257 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4159488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.851 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:43:02.851 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.625928083999995, 2024-09-17T22:43:02.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.04307035, 2024-09-17T22:43:02.851 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.851 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.851 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.851 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.851 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.851 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.851 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157257 2024-09-17T22:43:02.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.852 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:43:02.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:43:02.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148844+0000", 2024-09-17T22:43:02.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00099455599999999996 2024-09-17T22:43:02.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148848+0000", 2024-09-17T22:43:02.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0180000000000003e-06 2024-09-17T22:43:02.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148858+0000", 2024-09-17T22:43:02.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.825e-06 2024-09-17T22:43:02.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148865+0000", 2024-09-17T22:43:02.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8290000000000002e-06 2024-09-17T22:43:02.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190789+0000", 2024-09-17T22:43:02.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041924630999999997 2024-09-17T22:43:02.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190860+0000", 2024-09-17T22:43:02.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0557999999999998e-05 2024-09-17T22:43:02.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190920+0000", 2024-09-17T22:43:02.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9932999999999997e-05 2024-09-17T22:43:02.856 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.856 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.856 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.856 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157259 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4165632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.856 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:43:02.856 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.624902065000001, 2024-09-17T22:43:02.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042261411999999998, 2024-09-17T22:43:02.856 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.856 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.856 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.857 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.857 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.857 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.857 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157259 2024-09-17T22:43:02.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.857 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:43:02.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:43:02.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:43:02.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:43:02.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148878+0000", 2024-09-17T22:43:02.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:43:02.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148885+0000", 2024-09-17T22:43:02.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7290000000000003e-06 2024-09-17T22:43:02.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148893+0000", 2024-09-17T22:43:02.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8110000000000003e-06 2024-09-17T22:43:02.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.860 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.860 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190967+0000", 2024-09-17T22:43:02.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042073948999999999 2024-09-17T22:43:02.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191069+0000", 2024-09-17T22:43:02.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102075 2024-09-17T22:43:02.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191137+0000", 2024-09-17T22:43:02.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8281000000000006e-05 2024-09-17T22:43:02.861 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.861 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.862 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.862 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157261 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4171776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.862 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:43:02.862 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.624852458000007, 2024-09-17T22:43:02.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042437255, 2024-09-17T22:43:02.862 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.863 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.863 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.863 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.863 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157261 2024-09-17T22:43:02.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.864 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:43:02.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:43:02.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148926+0000", 2024-09-17T22:43:02.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T22:43:02.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.865 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.865 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148928+0000", 2024-09-17T22:43:02.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4329999999999998e-06 2024-09-17T22:43:02.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148936+0000", 2024-09-17T22:43:02.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6969999999999993e-06 2024-09-17T22:43:02.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148942+0000", 2024-09-17T22:43:02.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9479999999999996e-06 2024-09-17T22:43:02.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191185+0000", 2024-09-17T22:43:02.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042243147000000002 2024-09-17T22:43:02.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191266+0000", 2024-09-17T22:43:02.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1186000000000005e-05 2024-09-17T22:43:02.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191362+0000", 2024-09-17T22:43:02.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6384000000000004e-05 2024-09-17T22:43:02.868 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.868 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.868 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.868 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157263 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4177920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.868 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:43:02.868 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.624828058999995, 2024-09-17T22:43:02.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042636395000000001, 2024-09-17T22:43:02.868 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.869 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.869 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.869 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.869 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.869 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.869 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157263 2024-09-17T22:43:02.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.869 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.869 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:43:02.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:43:02.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:43:02.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.37e-07 2024-09-17T22:43:02.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148952+0000", 2024-09-17T22:43:02.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.88e-06 2024-09-17T22:43:02.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148959+0000", 2024-09-17T22:43:02.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5649999999999998e-06 2024-09-17T22:43:02.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148963+0000", 2024-09-17T22:43:02.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8570000000000003e-06 2024-09-17T22:43:02.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191408+0000", 2024-09-17T22:43:02.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042444707999999998 2024-09-17T22:43:02.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191489+0000", 2024-09-17T22:43:02.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0612999999999995e-05 2024-09-17T22:43:02.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191586+0000", 2024-09-17T22:43:02.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7235e-05 2024-09-17T22:43:02.873 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.873 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.873 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.874 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157265 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4184064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.874 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:43:02.874 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.624806509999999, 2024-09-17T22:43:02.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042791830000000003, 2024-09-17T22:43:02.874 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.874 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.874 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.874 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.874 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.874 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.874 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157265 2024-09-17T22:43:02.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.875 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:43:02.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:43:02.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148972+0000", 2024-09-17T22:43:02.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:43:02.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148973+0000", 2024-09-17T22:43:02.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.894e-06 2024-09-17T22:43:02.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148985+0000", 2024-09-17T22:43:02.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1995e-05 2024-09-17T22:43:02.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148990+0000", 2024-09-17T22:43:02.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8269999999999997e-06 2024-09-17T22:43:02.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191630+0000", 2024-09-17T22:43:02.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042639639 2024-09-17T22:43:02.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.879 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.879 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191686+0000", 2024-09-17T22:43:02.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6066000000000002e-05 2024-09-17T22:43:02.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191763+0000", 2024-09-17T22:43:02.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7035999999999999e-05 2024-09-17T22:43:02.880 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.880 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.880 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.880 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157267 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4190208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:02.880 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:43:02.881 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 72.624780697000006, 2024-09-17T22:43:02.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043028813999999999, 2024-09-17T22:43:02.881 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:02.881 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:02.881 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:02.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:02.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:02.881 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:02.881 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157267 2024-09-17T22:43:02.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.881 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:02.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:02.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:43:02.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:02.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:43:02.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:02.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:02.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:43:02.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:43:02.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:02.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148999+0000", 2024-09-17T22:43:02.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.432e-06 2024-09-17T22:43:02.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:02.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149002+0000", 2024-09-17T22:43:02.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7850000000000002e-06 2024-09-17T22:43:02.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:02.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149005+0000", 2024-09-17T22:43:02.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:43:02.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:02.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191878+0000", 2024-09-17T22:43:02.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872312000000003 2024-09-17T22:43:02.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:02.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191961+0000", 2024-09-17T22:43:02.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3418000000000007e-05 2024-09-17T22:43:02.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:02.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:02.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:02.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.192026+0000", 2024-09-17T22:43:02.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4689000000000004e-05 2024-09-17T22:43:02.885 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.886 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.886 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.886 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:02.886 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:02.886 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:02.887 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:43:02.888 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:02.885+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:02.989 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:02.986+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:02.992 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:02.992 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:02.992 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:43:02.992 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:03.001 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:43:03.044 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:43:03.089 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:03.087+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:03.151 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:03.151 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:03.151 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:43:03.151 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:43:03.151 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:03.161 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:43:03.204 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:03.202+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:03.266 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:03.266 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:43:03.266 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:43:03.267 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:43:03.267 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.267 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188693 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5881856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:03.267 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:43:03.267 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.436244330999997, 2024-09-17T22:43:03.267 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028193223, 2024-09-17T22:43:03.267 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.267 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.267 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.267 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.267 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.267 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.267 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188693 2024-09-17T22:43:03.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.268 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:43:03.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.268 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.268 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:43:03.268 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.268 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:43:03.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:43:03.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827090+0000", 2024-09-17T22:43:03.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2170000000000001e-06 2024-09-17T22:43:03.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.269 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.269 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.269 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827098+0000", 2024-09-17T22:43:03.269 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1510000000000004e-06 2024-09-17T22:43:03.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827103+0000", 2024-09-17T22:43:03.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0839999999999999e-06 2024-09-17T22:43:03.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855158+0000", 2024-09-17T22:43:03.270 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028055217 2024-09-17T22:43:03.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.270 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.270 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.270 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855201+0000", 2024-09-17T22:43:03.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3272000000000002e-05 2024-09-17T22:43:03.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.271 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.271 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855280+0000", 2024-09-17T22:43:03.271 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8845999999999997e-05 2024-09-17T22:43:03.271 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.271 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.271 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.271 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.271 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.271 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188694 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5971968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:03.271 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:43:03.272 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.436220444, 2024-09-17T22:43:03.272 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028312278, 2024-09-17T22:43:03.272 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.272 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.272 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.272 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.272 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.272 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.272 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188694 2024-09-17T22:43:03.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.272 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.272 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:43:03.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:43:03.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.273 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.273 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.273 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:43:03.273 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:43:03.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827114+0000", 2024-09-17T22:43:03.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3029999999999998e-06 2024-09-17T22:43:03.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827121+0000", 2024-09-17T22:43:03.274 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3699999999999997e-06 2024-09-17T22:43:03.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.274 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.274 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.274 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827126+0000", 2024-09-17T22:43:03.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7350000000000004e-06 2024-09-17T22:43:03.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855307+0000", 2024-09-17T22:43:03.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028180851999999999 2024-09-17T22:43:03.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.275 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.275 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.275 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855346+0000", 2024-09-17T22:43:03.275 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9274999999999999e-05 2024-09-17T22:43:03.275 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.276 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.276 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855423+0000", 2024-09-17T22:43:03.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7294999999999998e-05 2024-09-17T22:43:03.276 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.276 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.276 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.276 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.276 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188695 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:03.276 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:43:03.276 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.436199666, 2024-09-17T22:43:03.276 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028435876999999998, 2024-09-17T22:43:03.277 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.277 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.277 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.277 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.277 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.277 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.277 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188695 2024-09-17T22:43:03.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.277 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:43:03.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:43:03.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.278 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:43:03.278 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:43:03.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.278 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.278 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827138+0000", 2024-09-17T22:43:03.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2580000000000001e-06 2024-09-17T22:43:03.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827145+0000", 2024-09-17T22:43:03.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0129999999999996e-06 2024-09-17T22:43:03.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.279 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.279 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.279 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827150+0000", 2024-09-17T22:43:03.279 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3749999999999996e-06 2024-09-17T22:43:03.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855447+0000", 2024-09-17T22:43:03.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028297224999999999 2024-09-17T22:43:03.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855487+0000", 2024-09-17T22:43:03.280 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0454000000000003e-05 2024-09-17T22:43:03.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.280 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.280 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.280 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855567+0000", 2024-09-17T22:43:03.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0149999999999997e-05 2024-09-17T22:43:03.281 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.281 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.281 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.281 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.281 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188696 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:03.281 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:43:03.281 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.436175882000001, 2024-09-17T22:43:03.281 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028543570000000001, 2024-09-17T22:43:03.281 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.281 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188696 2024-09-17T22:43:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:43:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.282 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:43:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827156+0000", 2024-09-17T22:43:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T22:43:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827158+0000", 2024-09-17T22:43:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.875e-06 2024-09-17T22:43:03.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827164+0000", 2024-09-17T22:43:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7950000000000002e-06 2024-09-17T22:43:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827169+0000", 2024-09-17T22:43:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2880000000000003e-06 2024-09-17T22:43:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.284 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855582+0000", 2024-09-17T22:43:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028412929 2024-09-17T22:43:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855624+0000", 2024-09-17T22:43:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2484000000000002e-05 2024-09-17T22:43:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855699+0000", 2024-09-17T22:43:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4869000000000004e-05 2024-09-17T22:43:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.285 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188697 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6242304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:43:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.436144353000003, 2024-09-17T22:43:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028658598, 2024-09-17T22:43:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.286 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188697 2024-09-17T22:43:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:43:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:43:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:43:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8800000000000003e-07 2024-09-17T22:43:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827190+0000", 2024-09-17T22:43:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.351e-06 2024-09-17T22:43:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827198+0000", 2024-09-17T22:43:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.909e-06 2024-09-17T22:43:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.288 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827203+0000", 2024-09-17T22:43:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4020000000000001e-06 2024-09-17T22:43:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855720+0000", 2024-09-17T22:43:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028516909999999999 2024-09-17T22:43:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855762+0000", 2024-09-17T22:43:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2184999999999997e-05 2024-09-17T22:43:03.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855845+0000", 2024-09-17T22:43:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3353e-05 2024-09-17T22:43:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202883 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2342912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:43:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.538507971999998, 2024-09-17T22:43:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028098636, 2024-09-17T22:43:03.290 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202883 2024-09-17T22:43:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:43:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.291 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:43:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724824+0000", 2024-09-17T22:43:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:43:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724826+0000", 2024-09-17T22:43:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.086e-06 2024-09-17T22:43:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.292 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724832+0000", 2024-09-17T22:43:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9100000000000002e-06 2024-09-17T22:43:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724837+0000", 2024-09-17T22:43:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9439999999999996e-06 2024-09-17T22:43:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752790+0000", 2024-09-17T22:43:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027953657999999999 2024-09-17T22:43:03.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752848+0000", 2024-09-17T22:43:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7677999999999999e-05 2024-09-17T22:43:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752922+0000", 2024-09-17T22:43:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3957999999999999e-05 2024-09-17T22:43:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.294 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202884 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2473984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:43:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.538479746999997, 2024-09-17T22:43:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028206208999999999, 2024-09-17T22:43:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202884 2024-09-17T22:43:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.295 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:43:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:43:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.296 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724852+0000", 2024-09-17T22:43:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2599999999999998e-07 2024-09-17T22:43:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724854+0000", 2024-09-17T22:43:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.26e-06 2024-09-17T22:43:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724871+0000", 2024-09-17T22:43:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6589999999999999e-05 2024-09-17T22:43:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.297 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724877+0000", 2024-09-17T22:43:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.054e-06 2024-09-17T22:43:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752951+0000", 2024-09-17T22:43:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028073747 2024-09-17T22:43:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752993+0000", 2024-09-17T22:43:03.298 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2082000000000001e-05 2024-09-17T22:43:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753058+0000", 2024-09-17T22:43:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5049999999999996e-05 2024-09-17T22:43:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202885 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:43:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.538446444000002, 2024-09-17T22:43:03.299 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028333028, 2024-09-17T22:43:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202885 2024-09-17T22:43:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:43:03.300 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:43:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:43:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3300000000000002e-07 2024-09-17T22:43:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.301 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724892+0000", 2024-09-17T22:43:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2779999999999999e-06 2024-09-17T22:43:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724899+0000", 2024-09-17T22:43:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9110000000000004e-06 2024-09-17T22:43:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724904+0000", 2024-09-17T22:43:03.302 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2360000000000004e-06 2024-09-17T22:43:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753094+0000", 2024-09-17T22:43:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02819033 2024-09-17T22:43:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753142+0000", 2024-09-17T22:43:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.829e-05 2024-09-17T22:43:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.303 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753218+0000", 2024-09-17T22:43:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5450000000000004e-05 2024-09-17T22:43:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202886 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:43:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.538421245999999, 2024-09-17T22:43:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028442512, 2024-09-17T22:43:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.304 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202886 2024-09-17T22:43:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:43:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.305 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:43:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:43:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:43:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724912+0000", 2024-09-17T22:43:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.967e-06 2024-09-17T22:43:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.306 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724918+0000", 2024-09-17T22:43:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8069999999999996e-06 2024-09-17T22:43:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724923+0000", 2024-09-17T22:43:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.386e-06 2024-09-17T22:43:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753244+0000", 2024-09-17T22:43:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028321567999999998 2024-09-17T22:43:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.307 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753289+0000", 2024-09-17T22:43:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4620000000000003e-05 2024-09-17T22:43:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753353+0000", 2024-09-17T22:43:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3795999999999997e-05 2024-09-17T22:43:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202887 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.308 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:43:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.538403332999998, 2024-09-17T22:43:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028565838999999999, 2024-09-17T22:43:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202887 2024-09-17T22:43:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.309 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:43:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:43:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:43:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:43:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.310 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724930+0000", 2024-09-17T22:43:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8980000000000001e-06 2024-09-17T22:43:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724936+0000", 2024-09-17T22:43:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7849999999999997e-06 2024-09-17T22:43:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724942+0000", 2024-09-17T22:43:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9649999999999996e-06 2024-09-17T22:43:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.311 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753380+0000", 2024-09-17T22:43:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028438049 2024-09-17T22:43:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753425+0000", 2024-09-17T22:43:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5163000000000001e-05 2024-09-17T22:43:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753494+0000", 2024-09-17T22:43:03.312 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8588000000000001e-05 2024-09-17T22:43:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202888 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:43:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.538383834999998, 2024-09-17T22:43:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028703314000000001, 2024-09-17T22:43:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.313 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202888 2024-09-17T22:43:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:43:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:43:03.314 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724948+0000", 2024-09-17T22:43:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:43:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724950+0000", 2024-09-17T22:43:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8029999999999999e-06 2024-09-17T22:43:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.315 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724955+0000", 2024-09-17T22:43:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8760000000000002e-06 2024-09-17T22:43:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724960+0000", 2024-09-17T22:43:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5979999999999999e-06 2024-09-17T22:43:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753527+0000", 2024-09-17T22:43:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028566864000000001 2024-09-17T22:43:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.316 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753568+0000", 2024-09-17T22:43:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0775000000000002e-05 2024-09-17T22:43:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753651+0000", 2024-09-17T22:43:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3039000000000002e-05 2024-09-17T22:43:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.317 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202889 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:43:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.538366263, 2024-09-17T22:43:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028820964000000001, 2024-09-17T22:43:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202889 2024-09-17T22:43:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.318 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:43:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:43:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:43:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2599999999999998e-07 2024-09-17T22:43:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.319 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724967+0000", 2024-09-17T22:43:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T22:43:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724973+0000", 2024-09-17T22:43:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6060000000000002e-06 2024-09-17T22:43:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724978+0000", 2024-09-17T22:43:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8230000000000002e-06 2024-09-17T22:43:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.320 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753687+0000", 2024-09-17T22:43:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028709495000000002 2024-09-17T22:43:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753727+0000", 2024-09-17T22:43:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9953999999999997e-05 2024-09-17T22:43:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753786+0000", 2024-09-17T22:43:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8941999999999999e-05 2024-09-17T22:43:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.321 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202890 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:43:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.538348695000003, 2024-09-17T22:43:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028933838, 2024-09-17T22:43:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202890 2024-09-17T22:43:03.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:43:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:43:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:43:03.323 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5999999999999999e-07 2024-09-17T22:43:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724985+0000", 2024-09-17T22:43:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8169999999999999e-06 2024-09-17T22:43:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724991+0000", 2024-09-17T22:43:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:43:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.324 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724995+0000", 2024-09-17T22:43:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4530000000000004e-06 2024-09-17T22:43:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753811+0000", 2024-09-17T22:43:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028815584000000002 2024-09-17T22:43:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753854+0000", 2024-09-17T22:43:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3356999999999998e-05 2024-09-17T22:43:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.325 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753916+0000", 2024-09-17T22:43:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2488000000000002e-05 2024-09-17T22:43:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202891 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:43:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.538331237999998, 2024-09-17T22:43:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029067554999999998, 2024-09-17T22:43:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.326 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202891 2024-09-17T22:43:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:43:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.327 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:43:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:43:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:43:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725002+0000", 2024-09-17T22:43:03.328 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:43:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725008+0000", 2024-09-17T22:43:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6629999999999998e-06 2024-09-17T22:43:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725016+0000", 2024-09-17T22:43:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8269999999999998e-06 2024-09-17T22:43:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.329 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753949+0000", 2024-09-17T22:43:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028932625999999999 2024-09-17T22:43:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753994+0000", 2024-09-17T22:43:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5717000000000003e-05 2024-09-17T22:43:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754068+0000", 2024-09-17T22:43:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3312000000000006e-05 2024-09-17T22:43:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.330 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.331 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.331 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.332 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202892 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.332 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:43:03.332 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.538288530999999, 2024-09-17T22:43:03.332 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029159939999999999, 2024-09-17T22:43:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202892 2024-09-17T22:43:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:43:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:43:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:43:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7999999999999995e-07 2024-09-17T22:43:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.334 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725045+0000", 2024-09-17T22:43:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-17T22:43:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725052+0000", 2024-09-17T22:43:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3629999999999999e-06 2024-09-17T22:43:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725056+0000", 2024-09-17T22:43:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3250000000000001e-06 2024-09-17T22:43:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.335 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754094+0000", 2024-09-17T22:43:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029038270000000001 2024-09-17T22:43:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754134+0000", 2024-09-17T22:43:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9722999999999999e-05 2024-09-17T22:43:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754203+0000", 2024-09-17T22:43:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8524999999999999e-05 2024-09-17T22:43:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.336 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202893 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:43:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.538268954000003, 2024-09-17T22:43:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029341651, 2024-09-17T22:43:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202893 2024-09-17T22:43:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.337 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:43:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:43:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725063+0000", 2024-09-17T22:43:03.338 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6699999999999999e-07 2024-09-17T22:43:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725065+0000", 2024-09-17T22:43:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2120000000000002e-06 2024-09-17T22:43:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725071+0000", 2024-09-17T22:43:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5169999999999996e-06 2024-09-17T22:43:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.339 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725076+0000", 2024-09-17T22:43:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.78e-06 2024-09-17T22:43:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754233+0000", 2024-09-17T22:43:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029156277000000001 2024-09-17T22:43:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754298+0000", 2024-09-17T22:43:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5320000000000005e-05 2024-09-17T22:43:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.340 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754404+0000", 2024-09-17T22:43:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106078 2024-09-17T22:43:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202894 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:43:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.538249075000003, 2024-09-17T22:43:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029456572, 2024-09-17T22:43:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.341 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202894 2024-09-17T22:43:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:43:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.342 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:43:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725083+0000", 2024-09-17T22:43:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5900000000000002e-07 2024-09-17T22:43:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725085+0000", 2024-09-17T22:43:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.979e-06 2024-09-17T22:43:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725091+0000", 2024-09-17T22:43:03.343 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1480000000000003e-06 2024-09-17T22:43:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725095+0000", 2024-09-17T22:43:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1339999999999997e-06 2024-09-17T22:43:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754429+0000", 2024-09-17T22:43:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029333758000000001 2024-09-17T22:43:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.344 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754471+0000", 2024-09-17T22:43:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2274999999999998e-05 2024-09-17T22:43:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754539+0000", 2024-09-17T22:43:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7818999999999996e-05 2024-09-17T22:43:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202895 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.345 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:43:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.538230679000002, 2024-09-17T22:43:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029635578999999999, 2024-09-17T22:43:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202895 2024-09-17T22:43:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:43:03.346 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:43:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:43:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.08e-07 2024-09-17T22:43:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725107+0000", 2024-09-17T22:43:03.347 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9020000000000004e-06 2024-09-17T22:43:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725113+0000", 2024-09-17T22:43:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7400000000000001e-06 2024-09-17T22:43:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725117+0000", 2024-09-17T22:43:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:43:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754567+0000", 2024-09-17T22:43:03.348 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029449411000000002 2024-09-17T22:43:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754638+0000", 2024-09-17T22:43:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1432000000000003e-05 2024-09-17T22:43:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754736+0000", 2024-09-17T22:43:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8066000000000006e-05 2024-09-17T22:43:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.349 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202896 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:43:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.538208947999998, 2024-09-17T22:43:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029760534000000002, 2024-09-17T22:43:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202896 2024-09-17T22:43:03.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:43:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:43:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725123+0000", 2024-09-17T22:43:03.351 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:43:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725125+0000", 2024-09-17T22:43:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.863e-06 2024-09-17T22:43:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725130+0000", 2024-09-17T22:43:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7250000000000002e-06 2024-09-17T22:43:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.352 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725135+0000", 2024-09-17T22:43:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:43:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754763+0000", 2024-09-17T22:43:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029628202999999999 2024-09-17T22:43:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754805+0000", 2024-09-17T22:43:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2370000000000003e-05 2024-09-17T22:43:03.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754883+0000", 2024-09-17T22:43:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7565999999999996e-05 2024-09-17T22:43:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202897 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:43:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 45.538191587999997, 2024-09-17T22:43:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029947972999999999, 2024-09-17T22:43:03.354 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202897 2024-09-17T22:43:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:43:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:43:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:43:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:43:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.356 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725146+0000", 2024-09-17T22:43:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6030000000000004e-06 2024-09-17T22:43:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725152+0000", 2024-09-17T22:43:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9970000000000004e-06 2024-09-17T22:43:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725156+0000", 2024-09-17T22:43:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-06 2024-09-17T22:43:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754909+0000", 2024-09-17T22:43:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029752958999999999 2024-09-17T22:43:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754987+0000", 2024-09-17T22:43:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8208999999999996e-05 2024-09-17T22:43:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.755088+0000", 2024-09-17T22:43:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100348 2024-09-17T22:43:03.358 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.359 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.359 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.359 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.359 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.359 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:03.360 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:43:03.361 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:43:03.358+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:43:03.461 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:03.459+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:03.505 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:03.505 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:03.505 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:43:03.505 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:03.514 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:43:03.597 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:03.595+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:03.657 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:03.657 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:03.657 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:43:03.657 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:43:03.657 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:03.662 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:03.660+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:03.665 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:43:03.763 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:03.761+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:03.811 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:03.811 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:43:03.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:43:03.811 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:43:03.811 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.811 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202977 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.811 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:43:03.811 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.081927149999999, 2024-09-17T22:43:03.811 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023068750999999998, 2024-09-17T22:43:03.811 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.812 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.812 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.812 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.812 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.812 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.812 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202977 2024-09-17T22:43:03.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.812 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.812 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.812 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.812 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:43:03.812 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:43:03.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.813 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725980+0000", 2024-09-17T22:43:03.813 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:43:03.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.813 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.813 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725982+0000", 2024-09-17T22:43:03.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0669999999999999e-06 2024-09-17T22:43:03.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725988+0000", 2024-09-17T22:43:03.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6610000000000003e-06 2024-09-17T22:43:03.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.814 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.814 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.814 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725993+0000", 2024-09-17T22:43:03.814 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:43:03.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748888+0000", 2024-09-17T22:43:03.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022894930000000001 2024-09-17T22:43:03.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748951+0000", 2024-09-17T22:43:03.815 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3638000000000003e-05 2024-09-17T22:43:03.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.815 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.815 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.815 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749048+0000", 2024-09-17T22:43:03.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6343999999999998e-05 2024-09-17T22:43:03.816 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.816 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.816 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.816 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.816 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202978 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2211840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.816 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:43:03.816 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.081906638, 2024-09-17T22:43:03.816 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023238050999999999, 2024-09-17T22:43:03.816 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.816 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.816 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.816 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.817 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.817 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202978 2024-09-17T22:43:03.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.817 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:43:03.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.817 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.817 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.817 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:43:03.817 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:43:03.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.58e-07 2024-09-17T22:43:03.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726002+0000", 2024-09-17T22:43:03.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.193e-06 2024-09-17T22:43:03.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.818 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.818 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.818 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726009+0000", 2024-09-17T22:43:03.818 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8000000000000001e-06 2024-09-17T22:43:03.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726014+0000", 2024-09-17T22:43:03.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9840000000000001e-06 2024-09-17T22:43:03.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749090+0000", 2024-09-17T22:43:03.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023075917000000001 2024-09-17T22:43:03.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.819 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.819 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.819 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749163+0000", 2024-09-17T22:43:03.819 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3219999999999994e-05 2024-09-17T22:43:03.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.820 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.820 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749238+0000", 2024-09-17T22:43:03.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4479000000000003e-05 2024-09-17T22:43:03.820 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.820 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.820 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.820 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.820 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202979 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2342912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.820 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:43:03.820 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.081885186999997, 2024-09-17T22:43:03.820 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023393536999999999, 2024-09-17T22:43:03.820 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.821 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.821 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.821 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.821 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.821 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.821 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202979 2024-09-17T22:43:03.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.821 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.821 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.821 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:43:03.821 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.821 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:43:03.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726022+0000", 2024-09-17T22:43:03.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:43:03.822 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.822 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.822 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.822 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726024+0000", 2024-09-17T22:43:03.822 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1969999999999999e-06 2024-09-17T22:43:03.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726030+0000", 2024-09-17T22:43:03.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.691e-06 2024-09-17T22:43:03.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726035+0000", 2024-09-17T22:43:03.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8029999999999996e-06 2024-09-17T22:43:03.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.823 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.823 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.823 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749280+0000", 2024-09-17T22:43:03.823 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023244668 2024-09-17T22:43:03.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749344+0000", 2024-09-17T22:43:03.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4418000000000006e-05 2024-09-17T22:43:03.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.824 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.824 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749415+0000", 2024-09-17T22:43:03.824 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0272999999999996e-05 2024-09-17T22:43:03.824 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.824 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.824 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.824 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.824 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202980 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2473984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.825 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:43:03.825 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.081864418000002, 2024-09-17T22:43:03.825 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023562221000000001, 2024-09-17T22:43:03.825 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.825 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.825 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.825 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.825 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202980 2024-09-17T22:43:03.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.825 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.825 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:43:03.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:43:03.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.826 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.826 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:43:03.826 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:43:03.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.826 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726044+0000", 2024-09-17T22:43:03.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9489999999999999e-06 2024-09-17T22:43:03.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726066+0000", 2024-09-17T22:43:03.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1508000000000001e-05 2024-09-17T22:43:03.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.827 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.827 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726072+0000", 2024-09-17T22:43:03.827 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3450000000000002e-06 2024-09-17T22:43:03.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.827 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749455+0000", 2024-09-17T22:43:03.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023382824999999999 2024-09-17T22:43:03.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749527+0000", 2024-09-17T22:43:03.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1728000000000002e-05 2024-09-17T22:43:03.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.828 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.828 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.828 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749604+0000", 2024-09-17T22:43:03.828 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7428999999999993e-05 2024-09-17T22:43:03.828 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.829 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.829 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.829 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.829 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202981 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.829 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:43:03.829 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.081826143000001, 2024-09-17T22:43:03.829 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023677800999999998, 2024-09-17T22:43:03.829 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.829 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.829 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.829 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.829 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.829 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.830 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202981 2024-09-17T22:43:03.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.830 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:43:03.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.830 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.830 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.830 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:43:03.830 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.830 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726081+0000", 2024-09-17T22:43:03.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5499999999999998e-07 2024-09-17T22:43:03.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726083+0000", 2024-09-17T22:43:03.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1780000000000002e-06 2024-09-17T22:43:03.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.831 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.831 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.831 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726090+0000", 2024-09-17T22:43:03.831 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8510000000000001e-06 2024-09-17T22:43:03.831 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726101+0000", 2024-09-17T22:43:03.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.133e-05 2024-09-17T22:43:03.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749639+0000", 2024-09-17T22:43:03.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023537679999999998 2024-09-17T22:43:03.832 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.832 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.832 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.832 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749688+0000", 2024-09-17T22:43:03.832 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9548000000000001e-05 2024-09-17T22:43:03.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.833 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.833 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749758+0000", 2024-09-17T22:43:03.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9758999999999994e-05 2024-09-17T22:43:03.833 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.833 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.833 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.833 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.833 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202982 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.833 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:43:03.833 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.081798560000003, 2024-09-17T22:43:03.833 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023841469000000001, 2024-09-17T22:43:03.833 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.834 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.834 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.834 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.834 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202982 2024-09-17T22:43:03.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.834 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.834 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.834 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:43:03.834 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.834 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:43:03.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:43:03.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:43:03.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.835 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.835 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.835 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726110+0000", 2024-09-17T22:43:03.835 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.902e-06 2024-09-17T22:43:03.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726117+0000", 2024-09-17T22:43:03.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6189999999999996e-06 2024-09-17T22:43:03.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726122+0000", 2024-09-17T22:43:03.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.198e-06 2024-09-17T22:43:03.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.836 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.836 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.836 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749795+0000", 2024-09-17T22:43:03.836 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023673397999999998 2024-09-17T22:43:03.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749877+0000", 2024-09-17T22:43:03.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1910999999999996e-05 2024-09-17T22:43:03.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.837 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.837 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749949+0000", 2024-09-17T22:43:03.837 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2001000000000004e-05 2024-09-17T22:43:03.837 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.837 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.837 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.837 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202983 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:43:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.081777500999998, 2024-09-17T22:43:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024014081, 2024-09-17T22:43:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202983 2024-09-17T22:43:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.838 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:43:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:43:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:43:03.839 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:43:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726131+0000", 2024-09-17T22:43:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9429999999999999e-06 2024-09-17T22:43:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726138+0000", 2024-09-17T22:43:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T22:43:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.840 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726143+0000", 2024-09-17T22:43:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0629999999999996e-06 2024-09-17T22:43:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749991+0000", 2024-09-17T22:43:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023847671000000001 2024-09-17T22:43:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750065+0000", 2024-09-17T22:43:03.841 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4569999999999999e-05 2024-09-17T22:43:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750143+0000", 2024-09-17T22:43:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7649000000000001e-05 2024-09-17T22:43:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202984 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:43:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.081756800000001, 2024-09-17T22:43:03.842 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024177605000000001, 2024-09-17T22:43:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202984 2024-09-17T22:43:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:43:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:43:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726150+0000", 2024-09-17T22:43:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T22:43:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726152+0000", 2024-09-17T22:43:03.844 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:43:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726162+0000", 2024-09-17T22:43:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7839999999999998e-06 2024-09-17T22:43:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726167+0000", 2024-09-17T22:43:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5690000000000004e-06 2024-09-17T22:43:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.845 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750184+0000", 2024-09-17T22:43:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024016909999999999 2024-09-17T22:43:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750255+0000", 2024-09-17T22:43:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1137999999999996e-05 2024-09-17T22:43:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750327+0000", 2024-09-17T22:43:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1903000000000006e-05 2024-09-17T22:43:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.846 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.847 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.847 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202985 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.847 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:43:03.847 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.081731413, 2024-09-17T22:43:03.847 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024280500999999999, 2024-09-17T22:43:03.847 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.847 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.847 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.847 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.847 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.847 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.847 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202985 2024-09-17T22:43:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:43:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:43:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.848 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:43:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8100000000000003e-07 2024-09-17T22:43:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726182+0000", 2024-09-17T22:43:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:43:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726190+0000", 2024-09-17T22:43:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:43:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.849 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726195+0000", 2024-09-17T22:43:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2809999999999999e-06 2024-09-17T22:43:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750366+0000", 2024-09-17T22:43:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024171635 2024-09-17T22:43:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750410+0000", 2024-09-17T22:43:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3823999999999999e-05 2024-09-17T22:43:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.850 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750455+0000", 2024-09-17T22:43:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5039e-05 2024-09-17T22:43:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202986 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:43:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.081703832999999, 2024-09-17T22:43:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024453077, 2024-09-17T22:43:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.851 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202986 2024-09-17T22:43:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:43:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.852 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:43:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726203+0000", 2024-09-17T22:43:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T22:43:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726205+0000", 2024-09-17T22:43:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0250000000000001e-06 2024-09-17T22:43:03.853 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726211+0000", 2024-09-17T22:43:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1340000000000001e-06 2024-09-17T22:43:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726222+0000", 2024-09-17T22:43:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1105e-05 2024-09-17T22:43:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750520+0000", 2024-09-17T22:43:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024297462999999998 2024-09-17T22:43:03.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750585+0000", 2024-09-17T22:43:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5940999999999998e-05 2024-09-17T22:43:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750655+0000", 2024-09-17T22:43:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0012000000000007e-05 2024-09-17T22:43:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.855 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202987 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:43:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.081677452999998, 2024-09-17T22:43:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024622569, 2024-09-17T22:43:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202987 2024-09-17T22:43:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.856 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:43:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:43:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:43:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:43:03.857 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726238+0000", 2024-09-17T22:43:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7430000000000008e-06 2024-09-17T22:43:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726245+0000", 2024-09-17T22:43:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6640000000000001e-06 2024-09-17T22:43:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726249+0000", 2024-09-17T22:43:03.858 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9119999999999997e-06 2024-09-17T22:43:03.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750694+0000", 2024-09-17T22:43:03.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024444105000000001 2024-09-17T22:43:03.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750767+0000", 2024-09-17T22:43:03.859 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3577000000000004e-05 2024-09-17T22:43:03.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.859 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.859 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.859 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750851+0000", 2024-09-17T22:43:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4189000000000003e-05 2024-09-17T22:43:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202988 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:43:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.081650590999999, 2024-09-17T22:43:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024808883, 2024-09-17T22:43:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.860 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202988 2024-09-17T22:43:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:43:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.861 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:43:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:43:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5499999999999999e-07 2024-09-17T22:43:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726258+0000", 2024-09-17T22:43:03.862 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8020000000000001e-06 2024-09-17T22:43:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726272+0000", 2024-09-17T22:43:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3828e-05 2024-09-17T22:43:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726277+0000", 2024-09-17T22:43:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2379999999999997e-06 2024-09-17T22:43:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.863 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750895+0000", 2024-09-17T22:43:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024618543 2024-09-17T22:43:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750978+0000", 2024-09-17T22:43:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3030999999999998e-05 2024-09-17T22:43:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751065+0000", 2024-09-17T22:43:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6086000000000002e-05 2024-09-17T22:43:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.864 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202989 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:43:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.081621718000001, 2024-09-17T22:43:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024976788, 2024-09-17T22:43:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202989 2024-09-17T22:43:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.865 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:43:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:43:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726285+0000", 2024-09-17T22:43:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T22:43:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726287+0000", 2024-09-17T22:43:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9860000000000001e-06 2024-09-17T22:43:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726294+0000", 2024-09-17T22:43:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2849999999999997e-06 2024-09-17T22:43:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.867 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726299+0000", 2024-09-17T22:43:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0150000000000003e-06 2024-09-17T22:43:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751109+0000", 2024-09-17T22:43:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024810175 2024-09-17T22:43:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751184+0000", 2024-09-17T22:43:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4640000000000004e-05 2024-09-17T22:43:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.868 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751261+0000", 2024-09-17T22:43:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7317000000000005e-05 2024-09-17T22:43:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202990 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:43:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.081601503000002, 2024-09-17T22:43:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025156767999999999, 2024-09-17T22:43:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.869 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202990 2024-09-17T22:43:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:43:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.870 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:43:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:43:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:43:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726307+0000", 2024-09-17T22:43:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7090000000000001e-06 2024-09-17T22:43:03.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726312+0000", 2024-09-17T22:43:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4909999999999996e-06 2024-09-17T22:43:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726317+0000", 2024-09-17T22:43:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3800000000000004e-06 2024-09-17T22:43:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.872 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751308+0000", 2024-09-17T22:43:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024991143 2024-09-17T22:43:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751385+0000", 2024-09-17T22:43:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6793999999999997e-05 2024-09-17T22:43:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751461+0000", 2024-09-17T22:43:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6891999999999995e-05 2024-09-17T22:43:03.873 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202991 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:43:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.081583469000002, 2024-09-17T22:43:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025289477000000001, 2024-09-17T22:43:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.874 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.875 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.875 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202991 2024-09-17T22:43:03.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.875 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:43:03.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.875 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.875 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.875 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:43:03.875 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:43:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.27e-07 2024-09-17T22:43:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726325+0000", 2024-09-17T22:43:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1459999999999999e-06 2024-09-17T22:43:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.876 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726333+0000", 2024-09-17T22:43:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1289999999999996e-06 2024-09-17T22:43:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726339+0000", 2024-09-17T22:43:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2530000000000004e-06 2024-09-17T22:43:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751499+0000", 2024-09-17T22:43:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025160109 2024-09-17T22:43:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.877 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751543+0000", 2024-09-17T22:43:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4317000000000003e-05 2024-09-17T22:43:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751612+0000", 2024-09-17T22:43:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9096000000000004e-05 2024-09-17T22:43:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202992 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.878 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:43:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.081561405000002, 2024-09-17T22:43:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025420585999999998, 2024-09-17T22:43:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202992 2024-09-17T22:43:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.879 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:43:03.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:43:03.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.880 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.880 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.880 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:43:03.880 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5199999999999998e-07 2024-09-17T22:43:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726347+0000", 2024-09-17T22:43:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9939999999999999e-06 2024-09-17T22:43:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726358+0000", 2024-09-17T22:43:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0421999999999999e-05 2024-09-17T22:43:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.881 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726363+0000", 2024-09-17T22:43:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:43:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751644+0000", 2024-09-17T22:43:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025280727999999999 2024-09-17T22:43:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751712+0000", 2024-09-17T22:43:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8207999999999995e-05 2024-09-17T22:43:03.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751765+0000", 2024-09-17T22:43:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3533999999999999e-05 2024-09-17T22:43:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202993 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:43:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 46.081537414000003, 2024-09-17T22:43:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025516259999999999, 2024-09-17T22:43:03.883 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202993 2024-09-17T22:43:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:43:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.884 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:43:03.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:43:03.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:43:03.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.885 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.885 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.885 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726371+0000", 2024-09-17T22:43:03.885 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T22:43:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726380+0000", 2024-09-17T22:43:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9109999999999999e-06 2024-09-17T22:43:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726385+0000", 2024-09-17T22:43:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7380000000000002e-06 2024-09-17T22:43:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.886 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751785+0000", 2024-09-17T22:43:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025400526 2024-09-17T22:43:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751824+0000", 2024-09-17T22:43:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8989999999999998e-05 2024-09-17T22:43:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751885+0000", 2024-09-17T22:43:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0937000000000003e-05 2024-09-17T22:43:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:215737 3.b 3:d5ee24db:::rbd_data.115f2f67ca35.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-17T22:43:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:43:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.506768719, 2024-09-17T22:43:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025642133000000001, 2024-09-17T22:43:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:43:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 215737 2024-09-17T22:43:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.888 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:43:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:43:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301138+0000", 2024-09-17T22:43:03.889 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8100000000000002e-07 2024-09-17T22:43:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301143+0000", 2024-09-17T22:43:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6999999999999999e-06 2024-09-17T22:43:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301152+0000", 2024-09-17T22:43:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4979999999999995e-06 2024-09-17T22:43:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301159+0000", 2024-09-17T22:43:03.890 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9430000000000003e-06 2024-09-17T22:43:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301201+0000", 2024-09-17T22:43:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1219e-05 2024-09-17T22:43:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301247+0000", 2024-09-17T22:43:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5924999999999998e-05 2024-09-17T22:43:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:43:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301870+0000", 2024-09-17T22:43:03.891 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00062316100000000001 2024-09-17T22:43:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:43:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326658+0000", 2024-09-17T22:43:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024788719000000001 2024-09-17T22:43:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:43:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326694+0000", 2024-09-17T22:43:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5858999999999997e-05 2024-09-17T22:43:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.892 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:43:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326727+0000", 2024-09-17T22:43:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3176999999999998e-05 2024-09-17T22:43:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326780+0000", 2024-09-17T22:43:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2151000000000003e-05 2024-09-17T22:43:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:215738 3.b 3:d5937cde:::rbd_data.115f2f67ca35.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-17T22:43:03.893 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:43:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.506732187000001, 2024-09-17T22:43:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025650267000000001, 2024-09-17T22:43:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:43:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 215738 2024-09-17T22:43:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:43:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:43:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301175+0000", 2024-09-17T22:43:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2600000000000002e-07 2024-09-17T22:43:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.895 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301178+0000", 2024-09-17T22:43:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2049999999999998e-06 2024-09-17T22:43:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301185+0000", 2024-09-17T22:43:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8900000000000001e-06 2024-09-17T22:43:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301190+0000", 2024-09-17T22:43:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6910000000000002e-06 2024-09-17T22:43:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.896 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.304605+0000", 2024-09-17T22:43:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003414594 2024-09-17T22:43:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.304658+0000", 2024-09-17T22:43:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3334000000000001e-05 2024-09-17T22:43:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:43:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.308265+0000", 2024-09-17T22:43:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036068680000000001 2024-09-17T22:43:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.897 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:43:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326688+0000", 2024-09-17T22:43:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018422851000000001 2024-09-17T22:43:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:43:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326785+0000", 2024-09-17T22:43:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7050000000000001e-05 2024-09-17T22:43:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:43:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326806+0000", 2024-09-17T22:43:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1299999999999999e-05 2024-09-17T22:43:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.898 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326824+0000", 2024-09-17T22:43:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7958e-05 2024-09-17T22:43:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop_reply(client.4447.0:215737 3.b e87/23 ondisk, result = 0)", 2024-09-17T22:43:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:43:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 28.504276976, 2024-09-17T22:43:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023035287000000002, 2024-09-17T22:43:03.899 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:43:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:43:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:03.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303630+0000", 2024-09-17T22:43:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2e-07 2024-09-17T22:43:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303633+0000", 2024-09-17T22:43:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7470000000000001e-06 2024-09-17T22:43:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:03.901 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303641+0000", 2024-09-17T22:43:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7979999999999995e-06 2024-09-17T22:43:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303646+0000", 2024-09-17T22:43:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9910000000000002e-06 2024-09-17T22:43:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326647+0000", 2024-09-17T22:43:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023000936999999999 2024-09-17T22:43:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.902 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326654+0000", 2024-09-17T22:43:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2529999999999998e-06 2024-09-17T22:43:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326665+0000", 2024-09-17T22:43:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0040999999999999e-05 2024-09-17T22:43:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:03.903 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:03.904 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:03.904 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:43:03.905 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:03.903+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:03.906 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:43:04.006 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:04.004+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:04.012 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:43:04.014 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:04.014 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:43:04.014 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:43:04.014 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:04.023 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:43:04.030 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:43:04.030 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:43:04.041 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is unset 2024-09-17T22:43:04.050 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:43:04.050 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:43:04.071 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:43:04.072 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:43:04.092 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:43:04.092 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:43:04.107 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:04.105+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:04.111 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:43:04.112 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:43:04.130 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:43:04.131 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:43:04.149 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:43:04.150 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:43:04.152 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:04.152 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:04.153 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:43:04.154 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:43:04.162 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:43:04.208 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:04.205+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:04.305 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:04.305 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:04.309 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:04.307+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:04.314 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:43:04.410 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:04.408+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:04.457 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:04.457 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:04.466 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:43:04.511 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:04.508+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:04.568 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:04.569 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:04.578 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:43:04.612 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:43:04.609+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:43:04.679 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:04.679 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:04.688 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:43:04.713 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:43:04.711+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:43:04.791 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:04.791 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:04.799 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:43:04.843 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:04.841+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:04.901 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:04.901 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:04.910 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:43:04.951 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:04.948+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:05.016 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:05.013+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:05.055 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:05.055 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:05.068 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:43:05.116 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:05.113+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:05.187 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:05.187 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:05.196 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:43:05.217 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:05.214+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:05.317 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:05.315+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:05.340 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:05.340 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:05.349 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:43:05.418 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:05.415+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:05.453 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:05.453 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:05.462 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:43:05.546 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:05.543+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:05.606 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:05.606 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:05.615 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:43:05.620 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:05.618+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:05.721 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:05.719+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:05.758 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:05.758 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:05.767 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:43:05.822 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:05.819+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:05.870 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:05.871 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:05.880 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:43:05.923 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:05.920+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:05.983 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:05.983 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:05.992 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:43:06.035 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:06.032+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:06.043 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:43:06.043 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:43:06.043 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-17T22:43:06.043 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-17T22:43:06.060 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:43:06.103 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:06.104 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:06.114 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:43:06.155 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:43:06.156 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:06.152+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:06.181 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:43:06.184 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:43:06.185 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T22:43:06.185 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-17T22:43:06.226 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:06.224+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:06.261 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:06.261 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:06.271 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:43:06.271 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:43:06.303 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:43:06.304 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:43:06.327 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:06.324+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:06.327 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:43:06.328 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:43:06.345 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:43:06.345 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:43:06.362 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:43:06.362 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:43:06.363 INFO:teuthology.orchestra.run.smithi115.stderr:no valid command found; 10 closest matches: 2024-09-17T22:43:06.363 INFO:teuthology.orchestra.run.smithi115.stderr:0 2024-09-17T22:43:06.363 INFO:teuthology.orchestra.run.smithi115.stderr:1 2024-09-17T22:43:06.363 INFO:teuthology.orchestra.run.smithi115.stderr:2 2024-09-17T22:43:06.363 INFO:teuthology.orchestra.run.smithi115.stderr:abort 2024-09-17T22:43:06.364 INFO:teuthology.orchestra.run.smithi115.stderr:assert 2024-09-17T22:43:06.364 INFO:teuthology.orchestra.run.smithi115.stderr:bench [] [] [] [] 2024-09-17T22:43:06.364 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs debug_inject_read_zeros 2024-09-17T22:43:06.364 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs files list 2024-09-17T22:43:06.364 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs stats 2024-09-17T22:43:06.364 INFO:teuthology.orchestra.run.smithi115.stderr:bluestore allocator dump block 2024-09-17T22:43:06.364 INFO:teuthology.orchestra.run.smithi115.stderr:Error EINVAL: invalid command 2024-09-17T22:43:06.366 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:43:06.366 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T22:43:06.367 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-17T22:43:06.377 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:06.377 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:06.379 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:43:06.379 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:43:06.386 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:43:06.397 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:43:06.397 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:43:06.414 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:43:06.414 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:43:06.417 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:43:06.418 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:43:06.428 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:06.424+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:06.498 INFO:teuthology.orchestra.run.smithi115.stderr:no valid command found; 10 closest matches: 2024-09-17T22:43:06.498 INFO:teuthology.orchestra.run.smithi115.stderr:0 2024-09-17T22:43:06.498 INFO:teuthology.orchestra.run.smithi115.stderr:1 2024-09-17T22:43:06.498 INFO:teuthology.orchestra.run.smithi115.stderr:2 2024-09-17T22:43:06.498 INFO:teuthology.orchestra.run.smithi115.stderr:abort 2024-09-17T22:43:06.499 INFO:teuthology.orchestra.run.smithi115.stderr:assert 2024-09-17T22:43:06.499 INFO:teuthology.orchestra.run.smithi115.stderr:bench [] [] [] [] 2024-09-17T22:43:06.499 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs debug_inject_read_zeros 2024-09-17T22:43:06.499 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs files list 2024-09-17T22:43:06.499 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs stats 2024-09-17T22:43:06.499 INFO:teuthology.orchestra.run.smithi115.stderr:bluestore allocator dump block 2024-09-17T22:43:06.499 INFO:teuthology.orchestra.run.smithi115.stderr:Error EINVAL: invalid command 2024-09-17T22:43:06.504 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:43:06.505 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T22:43:06.505 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-17T22:43:06.529 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:06.525+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:06.530 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:06.530 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:06.530 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:43:06.530 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:43:06.530 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:06.539 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:43:06.629 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:06.627+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:06.635 INFO:teuthology.orchestra.run.smithi115.stderr:no valid command found; 10 closest matches: 2024-09-17T22:43:06.635 INFO:teuthology.orchestra.run.smithi115.stderr:0 2024-09-17T22:43:06.635 INFO:teuthology.orchestra.run.smithi115.stderr:1 2024-09-17T22:43:06.635 INFO:teuthology.orchestra.run.smithi115.stderr:2 2024-09-17T22:43:06.635 INFO:teuthology.orchestra.run.smithi115.stderr:abort 2024-09-17T22:43:06.635 INFO:teuthology.orchestra.run.smithi115.stderr:assert 2024-09-17T22:43:06.635 INFO:teuthology.orchestra.run.smithi115.stderr:bench [] [] [] [] 2024-09-17T22:43:06.635 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs debug_inject_read_zeros 2024-09-17T22:43:06.635 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs files list 2024-09-17T22:43:06.635 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs stats 2024-09-17T22:43:06.636 INFO:teuthology.orchestra.run.smithi115.stderr:bluestore allocator dump block 2024-09-17T22:43:06.636 INFO:teuthology.orchestra.run.smithi115.stderr:Error EINVAL: invalid command 2024-09-17T22:43:06.638 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:43:06.638 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T22:43:06.638 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-17T22:43:06.684 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:06.684 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:43:06.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:43:06.684 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:43:06.684 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.684 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202977 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:06.684 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:43:06.684 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.954762275, 2024-09-17T22:43:06.684 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023068750999999998, 2024-09-17T22:43:06.684 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.684 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.684 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:06.685 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:06.685 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:06.685 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:06.685 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202977 2024-09-17T22:43:06.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.685 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:43:06.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.685 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.685 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.685 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:43:06.685 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725980+0000", 2024-09-17T22:43:06.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:43:06.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725982+0000", 2024-09-17T22:43:06.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0669999999999999e-06 2024-09-17T22:43:06.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.686 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.686 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725988+0000", 2024-09-17T22:43:06.686 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6610000000000003e-06 2024-09-17T22:43:06.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725993+0000", 2024-09-17T22:43:06.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:43:06.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.687 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748888+0000", 2024-09-17T22:43:06.687 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022894930000000001 2024-09-17T22:43:06.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.687 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.687 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748951+0000", 2024-09-17T22:43:06.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3638000000000003e-05 2024-09-17T22:43:06.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.688 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.688 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749048+0000", 2024-09-17T22:43:06.688 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6343999999999998e-05 2024-09-17T22:43:06.688 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.688 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.688 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.688 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.688 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202978 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2211840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:06.688 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:43:06.689 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.954741763000001, 2024-09-17T22:43:06.689 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023238050999999999, 2024-09-17T22:43:06.689 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.689 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.689 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:06.689 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:06.689 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:06.689 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:06.689 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202978 2024-09-17T22:43:06.689 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.689 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.689 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.689 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:43:06.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.690 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.690 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:43:06.690 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.690 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:43:06.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.58e-07 2024-09-17T22:43:06.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726002+0000", 2024-09-17T22:43:06.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.193e-06 2024-09-17T22:43:06.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.691 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.691 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726009+0000", 2024-09-17T22:43:06.691 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8000000000000001e-06 2024-09-17T22:43:06.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.691 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726014+0000", 2024-09-17T22:43:06.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9840000000000001e-06 2024-09-17T22:43:06.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749090+0000", 2024-09-17T22:43:06.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023075917000000001 2024-09-17T22:43:06.692 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.692 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.692 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.692 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749163+0000", 2024-09-17T22:43:06.692 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3219999999999994e-05 2024-09-17T22:43:06.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.693 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.693 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749238+0000", 2024-09-17T22:43:06.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4479000000000003e-05 2024-09-17T22:43:06.693 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.693 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.693 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.693 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.693 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202979 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2342912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:06.693 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:43:06.693 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.954720311999999, 2024-09-17T22:43:06.693 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023393536999999999, 2024-09-17T22:43:06.694 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.694 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.694 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:06.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:06.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:06.694 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:06.694 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202979 2024-09-17T22:43:06.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.694 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.694 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:43:06.694 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.694 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:43:06.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726022+0000", 2024-09-17T22:43:06.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:43:06.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.695 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.695 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.695 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726024+0000", 2024-09-17T22:43:06.695 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1969999999999999e-06 2024-09-17T22:43:06.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726030+0000", 2024-09-17T22:43:06.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.691e-06 2024-09-17T22:43:06.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726035+0000", 2024-09-17T22:43:06.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8029999999999996e-06 2024-09-17T22:43:06.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.696 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.696 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.696 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749280+0000", 2024-09-17T22:43:06.696 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023244668 2024-09-17T22:43:06.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749344+0000", 2024-09-17T22:43:06.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4418000000000006e-05 2024-09-17T22:43:06.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.697 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.697 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749415+0000", 2024-09-17T22:43:06.697 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0272999999999996e-05 2024-09-17T22:43:06.697 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.697 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.697 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.697 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.698 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202980 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2473984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:06.698 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:43:06.698 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.954699542999997, 2024-09-17T22:43:06.698 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023562221000000001, 2024-09-17T22:43:06.698 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.698 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.698 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:06.698 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:06.698 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:06.698 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:06.698 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202980 2024-09-17T22:43:06.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.698 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.698 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.698 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:43:06.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:43:06.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.699 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:43:06.699 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:43:06.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.699 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.699 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726044+0000", 2024-09-17T22:43:06.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9489999999999999e-06 2024-09-17T22:43:06.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726066+0000", 2024-09-17T22:43:06.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1508000000000001e-05 2024-09-17T22:43:06.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.700 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726072+0000", 2024-09-17T22:43:06.700 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3450000000000002e-06 2024-09-17T22:43:06.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.700 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.700 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749455+0000", 2024-09-17T22:43:06.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023382824999999999 2024-09-17T22:43:06.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749527+0000", 2024-09-17T22:43:06.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1728000000000002e-05 2024-09-17T22:43:06.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.701 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.701 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.701 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749604+0000", 2024-09-17T22:43:06.701 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7428999999999993e-05 2024-09-17T22:43:06.701 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.701 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.701 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.702 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.702 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202981 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:06.702 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:43:06.702 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.954661268000002, 2024-09-17T22:43:06.702 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023677800999999998, 2024-09-17T22:43:06.702 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.702 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.702 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:06.702 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:06.702 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:06.702 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:06.702 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202981 2024-09-17T22:43:06.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.702 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:43:06.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:43:06.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.703 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.703 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.703 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726081+0000", 2024-09-17T22:43:06.703 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5499999999999998e-07 2024-09-17T22:43:06.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726083+0000", 2024-09-17T22:43:06.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1780000000000002e-06 2024-09-17T22:43:06.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726090+0000", 2024-09-17T22:43:06.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8510000000000001e-06 2024-09-17T22:43:06.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.704 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.704 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.704 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726101+0000", 2024-09-17T22:43:06.704 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.133e-05 2024-09-17T22:43:06.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.705 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.705 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.705 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749639+0000", 2024-09-17T22:43:06.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023537679999999998 2024-09-17T22:43:06.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.708 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.708 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749688+0000", 2024-09-17T22:43:06.708 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9548000000000001e-05 2024-09-17T22:43:06.708 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.708 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.709 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.709 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749758+0000", 2024-09-17T22:43:06.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9758999999999994e-05 2024-09-17T22:43:06.709 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.709 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.709 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.709 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.709 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202982 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:06.709 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:43:06.709 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.954633684999997, 2024-09-17T22:43:06.709 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023841469000000001, 2024-09-17T22:43:06.709 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.709 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.710 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:06.710 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:06.710 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:06.710 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:06.710 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202982 2024-09-17T22:43:06.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.710 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:43:06.710 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.710 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.710 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.710 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:43:06.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:43:06.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:43:06.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726110+0000", 2024-09-17T22:43:06.711 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.902e-06 2024-09-17T22:43:06.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.711 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.711 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.711 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726117+0000", 2024-09-17T22:43:06.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6189999999999996e-06 2024-09-17T22:43:06.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726122+0000", 2024-09-17T22:43:06.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.198e-06 2024-09-17T22:43:06.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749795+0000", 2024-09-17T22:43:06.712 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023673397999999998 2024-09-17T22:43:06.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.712 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.712 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.712 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749877+0000", 2024-09-17T22:43:06.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1910999999999996e-05 2024-09-17T22:43:06.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.713 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.713 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749949+0000", 2024-09-17T22:43:06.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2001000000000004e-05 2024-09-17T22:43:06.713 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.713 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.713 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.713 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.713 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202983 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:06.713 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:43:06.713 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.954612625999999, 2024-09-17T22:43:06.713 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024014081, 2024-09-17T22:43:06.713 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.714 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.714 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:06.714 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:06.714 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:06.714 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:06.714 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202983 2024-09-17T22:43:06.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.714 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.714 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.714 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:43:06.714 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.714 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:43:06.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:43:06.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:43:06.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.715 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.715 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726131+0000", 2024-09-17T22:43:06.715 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9429999999999999e-06 2024-09-17T22:43:06.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.715 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726138+0000", 2024-09-17T22:43:06.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T22:43:06.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726143+0000", 2024-09-17T22:43:06.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0629999999999996e-06 2024-09-17T22:43:06.716 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.716 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.716 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749991+0000", 2024-09-17T22:43:06.716 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023847671000000001 2024-09-17T22:43:06.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750065+0000", 2024-09-17T22:43:06.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4569999999999999e-05 2024-09-17T22:43:06.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.717 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.717 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750143+0000", 2024-09-17T22:43:06.717 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7649000000000001e-05 2024-09-17T22:43:06.717 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.717 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.717 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.717 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.718 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202984 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:06.718 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:43:06.718 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.954591925000003, 2024-09-17T22:43:06.718 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024177605000000001, 2024-09-17T22:43:06.718 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.718 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.718 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:06.718 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:06.718 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:06.718 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:06.718 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202984 2024-09-17T22:43:06.718 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.718 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.718 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.718 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:43:06.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:43:06.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.719 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.719 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.719 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726150+0000", 2024-09-17T22:43:06.719 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T22:43:06.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726152+0000", 2024-09-17T22:43:06.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:43:06.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726162+0000", 2024-09-17T22:43:06.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7839999999999998e-06 2024-09-17T22:43:06.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.720 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.720 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.720 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726167+0000", 2024-09-17T22:43:06.720 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5690000000000004e-06 2024-09-17T22:43:06.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750184+0000", 2024-09-17T22:43:06.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024016909999999999 2024-09-17T22:43:06.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750255+0000", 2024-09-17T22:43:06.721 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1137999999999996e-05 2024-09-17T22:43:06.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.721 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.721 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750327+0000", 2024-09-17T22:43:06.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1903000000000006e-05 2024-09-17T22:43:06.722 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.722 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.722 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.722 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.722 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.722 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202985 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:06.722 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:43:06.722 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.954566538000002, 2024-09-17T22:43:06.722 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024280500999999999, 2024-09-17T22:43:06.722 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.722 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.722 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:06.723 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:06.723 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:06.723 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:06.723 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202985 2024-09-17T22:43:06.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.723 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:43:06.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.723 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.723 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.723 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:43:06.723 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:43:06.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8100000000000003e-07 2024-09-17T22:43:06.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726182+0000", 2024-09-17T22:43:06.724 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:43:06.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.724 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.724 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.724 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726190+0000", 2024-09-17T22:43:06.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:43:06.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726195+0000", 2024-09-17T22:43:06.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2809999999999999e-06 2024-09-17T22:43:06.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.725 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.725 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750366+0000", 2024-09-17T22:43:06.725 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024171635 2024-09-17T22:43:06.725 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.725 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750410+0000", 2024-09-17T22:43:06.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3823999999999999e-05 2024-09-17T22:43:06.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.726 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.726 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750455+0000", 2024-09-17T22:43:06.726 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5039e-05 2024-09-17T22:43:06.726 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.726 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.726 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.726 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.726 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202986 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:06.726 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:43:06.727 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.954538958000001, 2024-09-17T22:43:06.727 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024453077, 2024-09-17T22:43:06.727 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.727 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.727 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:06.727 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:06.727 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:06.727 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:06.727 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202986 2024-09-17T22:43:06.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.727 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.727 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.727 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:43:06.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:43:06.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.728 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726203+0000", 2024-09-17T22:43:06.728 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T22:43:06.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.728 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.728 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726205+0000", 2024-09-17T22:43:06.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0250000000000001e-06 2024-09-17T22:43:06.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726211+0000", 2024-09-17T22:43:06.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1340000000000001e-06 2024-09-17T22:43:06.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.729 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.729 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.729 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726222+0000", 2024-09-17T22:43:06.729 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1105e-05 2024-09-17T22:43:06.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750520+0000", 2024-09-17T22:43:06.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024297462999999998 2024-09-17T22:43:06.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750585+0000", 2024-09-17T22:43:06.730 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5940999999999998e-05 2024-09-17T22:43:06.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.730 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.730 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.730 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750655+0000", 2024-09-17T22:43:06.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0012000000000007e-05 2024-09-17T22:43:06.731 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.731 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.731 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.731 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202987 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:06.731 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:43:06.731 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.954512577999999, 2024-09-17T22:43:06.731 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024622569, 2024-09-17T22:43:06.731 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.731 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.731 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:06.731 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:06.731 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:06.732 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:06.732 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202987 2024-09-17T22:43:06.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.732 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:43:06.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.732 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.732 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.732 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:43:06.732 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:43:06.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:43:06.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.733 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726238+0000", 2024-09-17T22:43:06.733 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7430000000000008e-06 2024-09-17T22:43:06.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.733 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.733 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726245+0000", 2024-09-17T22:43:06.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6640000000000001e-06 2024-09-17T22:43:06.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726249+0000", 2024-09-17T22:43:06.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9119999999999997e-06 2024-09-17T22:43:06.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.734 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.734 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750694+0000", 2024-09-17T22:43:06.734 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024444105000000001 2024-09-17T22:43:06.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.734 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750767+0000", 2024-09-17T22:43:06.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3577000000000004e-05 2024-09-17T22:43:06.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.735 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.735 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750851+0000", 2024-09-17T22:43:06.735 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4189000000000003e-05 2024-09-17T22:43:06.735 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.735 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.735 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.735 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.735 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.735 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202988 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:06.736 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:43:06.736 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.954485716000001, 2024-09-17T22:43:06.736 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024808883, 2024-09-17T22:43:06.736 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.736 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.736 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:06.736 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:06.736 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:06.736 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:06.736 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202988 2024-09-17T22:43:06.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.736 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:43:06.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:43:06.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.737 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.737 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.737 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:43:06.737 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5499999999999999e-07 2024-09-17T22:43:06.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726258+0000", 2024-09-17T22:43:06.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8020000000000001e-06 2024-09-17T22:43:06.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726272+0000", 2024-09-17T22:43:06.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3828e-05 2024-09-17T22:43:06.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.738 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.738 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.738 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726277+0000", 2024-09-17T22:43:06.738 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2379999999999997e-06 2024-09-17T22:43:06.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750895+0000", 2024-09-17T22:43:06.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024618543 2024-09-17T22:43:06.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750978+0000", 2024-09-17T22:43:06.739 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3030999999999998e-05 2024-09-17T22:43:06.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.739 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.739 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.739 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751065+0000", 2024-09-17T22:43:06.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6086000000000002e-05 2024-09-17T22:43:06.740 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.740 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.740 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.740 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.740 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202989 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:06.740 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:43:06.740 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.954456843000003, 2024-09-17T22:43:06.740 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024976788, 2024-09-17T22:43:06.740 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.740 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.740 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:06.741 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:06.741 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:06.741 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:06.741 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202989 2024-09-17T22:43:06.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.741 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:43:06.741 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.741 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.741 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.741 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:43:06.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726285+0000", 2024-09-17T22:43:06.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T22:43:06.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.742 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.742 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726287+0000", 2024-09-17T22:43:06.742 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9860000000000001e-06 2024-09-17T22:43:06.742 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.742 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726294+0000", 2024-09-17T22:43:06.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2849999999999997e-06 2024-09-17T22:43:06.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726299+0000", 2024-09-17T22:43:06.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0150000000000003e-06 2024-09-17T22:43:06.743 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.743 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.743 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751109+0000", 2024-09-17T22:43:06.743 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024810175 2024-09-17T22:43:06.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751184+0000", 2024-09-17T22:43:06.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4640000000000004e-05 2024-09-17T22:43:06.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.744 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.744 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.744 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751261+0000", 2024-09-17T22:43:06.744 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7317000000000005e-05 2024-09-17T22:43:06.744 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.744 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.744 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.744 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.745 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.745 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202990 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:06.745 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:43:06.745 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.954436628000003, 2024-09-17T22:43:06.745 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025156767999999999, 2024-09-17T22:43:06.745 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.745 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.745 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:06.745 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:06.745 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:06.745 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:06.745 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202990 2024-09-17T22:43:06.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.746 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:43:06.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:43:06.746 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.746 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.746 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.746 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:43:06.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:43:06.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726307+0000", 2024-09-17T22:43:06.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7090000000000001e-06 2024-09-17T22:43:06.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.747 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.747 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726312+0000", 2024-09-17T22:43:06.747 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4909999999999996e-06 2024-09-17T22:43:06.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.747 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726317+0000", 2024-09-17T22:43:06.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3800000000000004e-06 2024-09-17T22:43:06.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751308+0000", 2024-09-17T22:43:06.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024991143 2024-09-17T22:43:06.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.748 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.748 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.748 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751385+0000", 2024-09-17T22:43:06.748 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6793999999999997e-05 2024-09-17T22:43:06.748 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.749 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.749 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751461+0000", 2024-09-17T22:43:06.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6891999999999995e-05 2024-09-17T22:43:06.749 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.749 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.749 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.749 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.749 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202991 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:06.749 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:43:06.749 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.954418594000003, 2024-09-17T22:43:06.749 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025289477000000001, 2024-09-17T22:43:06.750 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.750 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.750 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:06.750 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:06.750 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:06.750 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:06.750 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202991 2024-09-17T22:43:06.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.750 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.750 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.750 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:43:06.750 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.750 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:43:06.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:43:06.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.27e-07 2024-09-17T22:43:06.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.751 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.751 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726325+0000", 2024-09-17T22:43:06.751 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1459999999999999e-06 2024-09-17T22:43:06.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.751 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726333+0000", 2024-09-17T22:43:06.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1289999999999996e-06 2024-09-17T22:43:06.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726339+0000", 2024-09-17T22:43:06.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2530000000000004e-06 2024-09-17T22:43:06.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.752 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.752 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.752 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751499+0000", 2024-09-17T22:43:06.752 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025160109 2024-09-17T22:43:06.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751543+0000", 2024-09-17T22:43:06.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4317000000000003e-05 2024-09-17T22:43:06.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.753 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.753 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751612+0000", 2024-09-17T22:43:06.753 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9096000000000004e-05 2024-09-17T22:43:06.753 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.753 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.753 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.753 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.754 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202992 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:06.754 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:43:06.754 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.954396529999997, 2024-09-17T22:43:06.754 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025420585999999998, 2024-09-17T22:43:06.754 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.754 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.754 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:06.754 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:06.754 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:06.754 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:06.754 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202992 2024-09-17T22:43:06.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.754 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.754 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:43:06.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:43:06.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.755 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.755 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.755 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:43:06.755 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5199999999999998e-07 2024-09-17T22:43:06.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726347+0000", 2024-09-17T22:43:06.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9939999999999999e-06 2024-09-17T22:43:06.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726358+0000", 2024-09-17T22:43:06.756 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0421999999999999e-05 2024-09-17T22:43:06.756 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.756 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.756 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.756 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726363+0000", 2024-09-17T22:43:06.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:43:06.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751644+0000", 2024-09-17T22:43:06.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025280727999999999 2024-09-17T22:43:06.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.757 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.757 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751712+0000", 2024-09-17T22:43:06.757 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8207999999999995e-05 2024-09-17T22:43:06.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.757 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.758 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.758 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751765+0000", 2024-09-17T22:43:06.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3533999999999999e-05 2024-09-17T22:43:06.758 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.758 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.758 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.758 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.758 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202993 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:06.758 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:43:06.758 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 48.954372538999998, 2024-09-17T22:43:06.758 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025516259999999999, 2024-09-17T22:43:06.758 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.758 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.759 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:06.759 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:06.759 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:06.759 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:06.759 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202993 2024-09-17T22:43:06.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.759 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.759 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.759 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:43:06.759 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:43:06.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:43:06.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:43:06.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.760 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.760 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.760 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726371+0000", 2024-09-17T22:43:06.760 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T22:43:06.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726380+0000", 2024-09-17T22:43:06.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9109999999999999e-06 2024-09-17T22:43:06.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726385+0000", 2024-09-17T22:43:06.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7380000000000002e-06 2024-09-17T22:43:06.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.761 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.761 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.761 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751785+0000", 2024-09-17T22:43:06.761 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025400526 2024-09-17T22:43:06.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751824+0000", 2024-09-17T22:43:06.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8989999999999998e-05 2024-09-17T22:43:06.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.762 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.762 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751885+0000", 2024-09-17T22:43:06.762 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0937000000000003e-05 2024-09-17T22:43:06.762 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.762 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.762 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.762 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.763 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:215737 3.b 3:d5ee24db:::rbd_data.115f2f67ca35.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-17T22:43:06.763 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:43:06.763 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.379603844000002, 2024-09-17T22:43:06.763 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025642133000000001, 2024-09-17T22:43:06.763 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.763 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.763 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:43:06.763 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:06.763 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:06.763 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:06.763 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 215737 2024-09-17T22:43:06.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.763 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.763 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:43:06.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:43:06.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.764 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.764 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.764 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301138+0000", 2024-09-17T22:43:06.764 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8100000000000002e-07 2024-09-17T22:43:06.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301143+0000", 2024-09-17T22:43:06.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6999999999999999e-06 2024-09-17T22:43:06.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301152+0000", 2024-09-17T22:43:06.765 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4979999999999995e-06 2024-09-17T22:43:06.765 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.765 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.765 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.765 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301159+0000", 2024-09-17T22:43:06.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9430000000000003e-06 2024-09-17T22:43:06.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301201+0000", 2024-09-17T22:43:06.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1219e-05 2024-09-17T22:43:06.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301247+0000", 2024-09-17T22:43:06.766 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5924999999999998e-05 2024-09-17T22:43:06.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.766 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.766 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:43:06.766 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301870+0000", 2024-09-17T22:43:06.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00062316100000000001 2024-09-17T22:43:06.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:43:06.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326658+0000", 2024-09-17T22:43:06.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024788719000000001 2024-09-17T22:43:06.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:43:06.767 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326694+0000", 2024-09-17T22:43:06.767 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5858999999999997e-05 2024-09-17T22:43:06.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.767 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.767 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:43:06.768 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326727+0000", 2024-09-17T22:43:06.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3176999999999998e-05 2024-09-17T22:43:06.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.768 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.768 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326780+0000", 2024-09-17T22:43:06.768 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2151000000000003e-05 2024-09-17T22:43:06.768 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.768 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.768 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.768 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.768 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:215738 3.b 3:d5937cde:::rbd_data.115f2f67ca35.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-17T22:43:06.768 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:43:06.768 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.379567311999999, 2024-09-17T22:43:06.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025650267000000001, 2024-09-17T22:43:06.769 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.769 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.769 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:43:06.769 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:06.769 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:06.769 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:06.769 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 215738 2024-09-17T22:43:06.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.769 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.769 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.769 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.769 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:43:06.769 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:43:06.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301175+0000", 2024-09-17T22:43:06.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2600000000000002e-07 2024-09-17T22:43:06.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.770 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.770 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.770 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301178+0000", 2024-09-17T22:43:06.770 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2049999999999998e-06 2024-09-17T22:43:06.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301185+0000", 2024-09-17T22:43:06.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8900000000000001e-06 2024-09-17T22:43:06.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301190+0000", 2024-09-17T22:43:06.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6910000000000002e-06 2024-09-17T22:43:06.771 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.771 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.771 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.771 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.304605+0000", 2024-09-17T22:43:06.771 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003414594 2024-09-17T22:43:06.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.304658+0000", 2024-09-17T22:43:06.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3334000000000001e-05 2024-09-17T22:43:06.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:43:06.772 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.308265+0000", 2024-09-17T22:43:06.772 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036068680000000001 2024-09-17T22:43:06.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.772 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.772 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:43:06.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326688+0000", 2024-09-17T22:43:06.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018422851000000001 2024-09-17T22:43:06.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:43:06.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326785+0000", 2024-09-17T22:43:06.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7050000000000001e-05 2024-09-17T22:43:06.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.773 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.773 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:43:06.773 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326806+0000", 2024-09-17T22:43:06.773 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1299999999999999e-05 2024-09-17T22:43:06.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.774 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.774 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326824+0000", 2024-09-17T22:43:06.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7958e-05 2024-09-17T22:43:06.774 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.774 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.774 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.774 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.774 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop_reply(client.4447.0:215737 3.b e87/23 ondisk, result = 0)", 2024-09-17T22:43:06.774 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:43:06.774 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 31.377112101000002, 2024-09-17T22:43:06.774 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023035287000000002, 2024-09-17T22:43:06.774 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:06.775 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:06.775 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:06.775 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:06.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:06.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:43:06.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.775 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:06.775 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:43:06.775 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:06.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.775 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:06.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303630+0000", 2024-09-17T22:43:06.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2e-07 2024-09-17T22:43:06.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:06.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303633+0000", 2024-09-17T22:43:06.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7470000000000001e-06 2024-09-17T22:43:06.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.776 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.776 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:06.776 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303641+0000", 2024-09-17T22:43:06.776 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7979999999999995e-06 2024-09-17T22:43:06.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:06.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303646+0000", 2024-09-17T22:43:06.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9910000000000002e-06 2024-09-17T22:43:06.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:06.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326647+0000", 2024-09-17T22:43:06.777 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023000936999999999 2024-09-17T22:43:06.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.777 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.777 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:06.777 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326654+0000", 2024-09-17T22:43:06.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2529999999999998e-06 2024-09-17T22:43:06.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:06.778 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:06.778 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:06.778 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326665+0000", 2024-09-17T22:43:06.778 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0040999999999999e-05 2024-09-17T22:43:06.778 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.778 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.778 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.778 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:06.778 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:06.778 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:06.780 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:43:06.780 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:06.777+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:06.811 INFO:teuthology.orchestra.run.smithi115.stderr:no valid command found; 10 closest matches: 2024-09-17T22:43:06.811 INFO:teuthology.orchestra.run.smithi115.stderr:0 2024-09-17T22:43:06.811 INFO:teuthology.orchestra.run.smithi115.stderr:1 2024-09-17T22:43:06.811 INFO:teuthology.orchestra.run.smithi115.stderr:2 2024-09-17T22:43:06.811 INFO:teuthology.orchestra.run.smithi115.stderr:abort 2024-09-17T22:43:06.811 INFO:teuthology.orchestra.run.smithi115.stderr:assert 2024-09-17T22:43:06.811 INFO:teuthology.orchestra.run.smithi115.stderr:bench [] [] [] [] 2024-09-17T22:43:06.811 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs debug_inject_read_zeros 2024-09-17T22:43:06.811 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs files list 2024-09-17T22:43:06.811 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs stats 2024-09-17T22:43:06.812 INFO:teuthology.orchestra.run.smithi115.stderr:bluestore allocator dump block 2024-09-17T22:43:06.812 INFO:teuthology.orchestra.run.smithi115.stderr:Error EINVAL: invalid command 2024-09-17T22:43:06.814 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:43:06.814 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T22:43:06.814 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-17T22:43:06.880 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:43:06.877+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:43:06.885 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:43:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:43:06.885 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:06.894 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:43:06.981 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:06.978+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:06.990 INFO:teuthology.orchestra.run.smithi115.stderr:no valid command found; 10 closest matches: 2024-09-17T22:43:06.990 INFO:teuthology.orchestra.run.smithi115.stderr:0 2024-09-17T22:43:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:1 2024-09-17T22:43:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:2 2024-09-17T22:43:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:abort 2024-09-17T22:43:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:assert 2024-09-17T22:43:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:bench [] [] [] [] 2024-09-17T22:43:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs debug_inject_read_zeros 2024-09-17T22:43:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs files list 2024-09-17T22:43:06.991 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs stats 2024-09-17T22:43:06.992 INFO:teuthology.orchestra.run.smithi115.stderr:bluestore allocator dump block 2024-09-17T22:43:06.992 INFO:teuthology.orchestra.run.smithi115.stderr:Error EINVAL: invalid command 2024-09-17T22:43:06.994 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:43:06.995 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T22:43:06.995 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-17T22:43:07.044 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:07.044 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:43:07.044 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:43:07.044 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:43:07.044 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:07.054 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:43:07.066 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is set 2024-09-17T22:43:07.082 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:07.080+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:07.144 INFO:teuthology.orchestra.run.smithi115.stderr:no valid command found; 10 closest matches: 2024-09-17T22:43:07.144 INFO:teuthology.orchestra.run.smithi115.stderr:0 2024-09-17T22:43:07.144 INFO:teuthology.orchestra.run.smithi115.stderr:1 2024-09-17T22:43:07.144 INFO:teuthology.orchestra.run.smithi115.stderr:2 2024-09-17T22:43:07.144 INFO:teuthology.orchestra.run.smithi115.stderr:abort 2024-09-17T22:43:07.144 INFO:teuthology.orchestra.run.smithi115.stderr:assert 2024-09-17T22:43:07.145 INFO:teuthology.orchestra.run.smithi115.stderr:bench [] [] [] [] 2024-09-17T22:43:07.145 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs debug_inject_read_zeros 2024-09-17T22:43:07.145 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs files list 2024-09-17T22:43:07.145 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs stats 2024-09-17T22:43:07.145 INFO:teuthology.orchestra.run.smithi115.stderr:bluestore allocator dump block 2024-09-17T22:43:07.145 INFO:teuthology.orchestra.run.smithi115.stderr:Error EINVAL: invalid command 2024-09-17T22:43:07.147 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:43:07.147 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T22:43:07.148 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-17T22:43:07.174 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:07.174 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:43:07.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:43:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:43:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:43:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:43:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.812597662, 2024-09-17T22:43:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.249158453, 2024-09-17T22:43:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:43:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:43:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:43:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-17T22:43:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:43:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:43:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358680+0000", 2024-09-17T22:43:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.592e-06 2024-09-17T22:43:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358688+0000", 2024-09-17T22:43:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1610000000000003e-06 2024-09-17T22:43:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358697+0000", 2024-09-17T22:43:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2580000000000001e-06 2024-09-17T22:43:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358706+0000", 2024-09-17T22:43:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1640000000000007e-06 2024-09-17T22:43:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358766+0000", 2024-09-17T22:43:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9893999999999999e-05 2024-09-17T22:43:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:43:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358778+0000", 2024-09-17T22:43:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1148999999999999e-05 2024-09-17T22:43:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606347+0000", 2024-09-17T22:43:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.247569971 2024-09-17T22:43:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606533+0000", 2024-09-17T22:43:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018599400000000001 2024-09-17T22:43:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:43:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606752+0000", 2024-09-17T22:43:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000218373 2024-09-17T22:43:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:43:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607455+0000", 2024-09-17T22:43:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070311799999999997 2024-09-17T22:43:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:43:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607735+0000", 2024-09-17T22:43:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027972300000000001 2024-09-17T22:43:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:43:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607793+0000", 2024-09-17T22:43:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8529999999999997e-05 2024-09-17T22:43:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607837+0000", 2024-09-17T22:43:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3526000000000003e-05 2024-09-17T22:43:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186749 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 11960320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:43:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.747956340000002, 2024-09-17T22:43:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029226166000000001, 2024-09-17T22:43:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186749 2024-09-17T22:43:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:43:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:43:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:43:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-09-17T22:43:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423328+0000", 2024-09-17T22:43:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6790000000000005e-06 2024-09-17T22:43:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423334+0000", 2024-09-17T22:43:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0009999999999998e-06 2024-09-17T22:43:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423338+0000", 2024-09-17T22:43:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6419999999999998e-06 2024-09-17T22:43:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452464+0000", 2024-09-17T22:43:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029125389000000002 2024-09-17T22:43:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452501+0000", 2024-09-17T22:43:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7656e-05 2024-09-17T22:43:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452546+0000", 2024-09-17T22:43:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4468999999999997e-05 2024-09-17T22:43:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186750 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12050432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:43:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.74793055, 2024-09-17T22:43:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029307863999999999, 2024-09-17T22:43:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186750 2024-09-17T22:43:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:43:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:43:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423346+0000", 2024-09-17T22:43:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.84e-07 2024-09-17T22:43:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423348+0000", 2024-09-17T22:43:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7540000000000001e-06 2024-09-17T22:43:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423353+0000", 2024-09-17T22:43:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:43:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423362+0000", 2024-09-17T22:43:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9220000000000003e-06 2024-09-17T22:43:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452571+0000", 2024-09-17T22:43:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029209174000000001 2024-09-17T22:43:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452612+0000", 2024-09-17T22:43:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1363999999999999e-05 2024-09-17T22:43:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452653+0000", 2024-09-17T22:43:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0893999999999998e-05 2024-09-17T22:43:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186751 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12140544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:43:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.74790806, 2024-09-17T22:43:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029387488, 2024-09-17T22:43:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186751 2024-09-17T22:43:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:43:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:43:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:43:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1399999999999997e-07 2024-09-17T22:43:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423375+0000", 2024-09-17T22:43:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8889999999999996e-06 2024-09-17T22:43:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423381+0000", 2024-09-17T22:43:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5690000000000004e-06 2024-09-17T22:43:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423386+0000", 2024-09-17T22:43:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9559999999999996e-06 2024-09-17T22:43:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452679+0000", 2024-09-17T22:43:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029292820000000001 2024-09-17T22:43:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452718+0000", 2024-09-17T22:43:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8961000000000002e-05 2024-09-17T22:43:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452755+0000", 2024-09-17T22:43:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7879000000000001e-05 2024-09-17T22:43:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186752 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12230656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:43:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.747876808999997, 2024-09-17T22:43:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029463269, 2024-09-17T22:43:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186752 2024-09-17T22:43:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:43:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:43:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423400+0000", 2024-09-17T22:43:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:43:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423401+0000", 2024-09-17T22:43:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3599999999999999e-06 2024-09-17T22:43:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423404+0000", 2024-09-17T22:43:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5999999999999998e-06 2024-09-17T22:43:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423407+0000", 2024-09-17T22:43:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6869999999999999e-06 2024-09-17T22:43:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452782+0000", 2024-09-17T22:43:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029375107000000001 2024-09-17T22:43:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452821+0000", 2024-09-17T22:43:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9122999999999998e-05 2024-09-17T22:43:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452862+0000", 2024-09-17T22:43:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1019000000000001e-05 2024-09-17T22:43:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186753 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:43:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.747864786999997, 2024-09-17T22:43:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029553297999999999, 2024-09-17T22:43:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186753 2024-09-17T22:43:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:43:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:43:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:43:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.98e-07 2024-09-17T22:43:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423412+0000", 2024-09-17T22:43:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8700000000000004e-07 2024-09-17T22:43:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423415+0000", 2024-09-17T22:43:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:43:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423418+0000", 2024-09-17T22:43:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.283e-06 2024-09-17T22:43:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452893+0000", 2024-09-17T22:43:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029475881999999998 2024-09-17T22:43:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452928+0000", 2024-09-17T22:43:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4109000000000002e-05 2024-09-17T22:43:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452964+0000", 2024-09-17T22:43:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6937000000000002e-05 2024-09-17T22:43:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186754 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12410880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:43:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.747855649999998, 2024-09-17T22:43:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029656021000000001, 2024-09-17T22:43:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186754 2024-09-17T22:43:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:43:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:43:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423421+0000", 2024-09-17T22:43:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9999999999999999e-07 2024-09-17T22:43:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423425+0000", 2024-09-17T22:43:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6519999999999997e-06 2024-09-17T22:43:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423428+0000", 2024-09-17T22:43:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1370000000000002e-06 2024-09-17T22:43:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423431+0000", 2024-09-17T22:43:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4090000000000001e-06 2024-09-17T22:43:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452992+0000", 2024-09-17T22:43:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029561545000000002 2024-09-17T22:43:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453031+0000", 2024-09-17T22:43:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8834000000000001e-05 2024-09-17T22:43:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453076+0000", 2024-09-17T22:43:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5244000000000003e-05 2024-09-17T22:43:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186755 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12500992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:43:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.747842368999997, 2024-09-17T22:43:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029745649999999998, 2024-09-17T22:43:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186755 2024-09-17T22:43:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:43:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:43:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:43:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.17e-07 2024-09-17T22:43:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423435+0000", 2024-09-17T22:43:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:43:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423438+0000", 2024-09-17T22:43:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8210000000000002e-06 2024-09-17T22:43:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423443+0000", 2024-09-17T22:43:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1070000000000004e-06 2024-09-17T22:43:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453103+0000", 2024-09-17T22:43:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029660399 2024-09-17T22:43:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453141+0000", 2024-09-17T22:43:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.748e-05 2024-09-17T22:43:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453179+0000", 2024-09-17T22:43:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8671000000000003e-05 2024-09-17T22:43:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187439 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8437760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:43:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.630405271999997, 2024-09-17T22:43:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029278048000000001, 2024-09-17T22:43:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187439 2024-09-17T22:43:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:43:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:43:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:43:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6199999999999999e-07 2024-09-17T22:43:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540872+0000", 2024-09-17T22:43:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.09e-07 2024-09-17T22:43:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540875+0000", 2024-09-17T22:43:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8990000000000001e-06 2024-09-17T22:43:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540877+0000", 2024-09-17T22:43:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1789999999999998e-06 2024-09-17T22:43:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570074+0000", 2024-09-17T22:43:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029197358 2024-09-17T22:43:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570111+0000", 2024-09-17T22:43:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6170000000000001e-05 2024-09-17T22:43:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570149+0000", 2024-09-17T22:43:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8170999999999998e-05 2024-09-17T22:43:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187440 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8527872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:43:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.630389802000003, 2024-09-17T22:43:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029359758999999999, 2024-09-17T22:43:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187440 2024-09-17T22:43:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:43:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:43:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:43:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2700000000000001e-07 2024-09-17T22:43:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540888+0000", 2024-09-17T22:43:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-17T22:43:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540893+0000", 2024-09-17T22:43:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9110000000000001e-06 2024-09-17T22:43:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540895+0000", 2024-09-17T22:43:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.712e-06 2024-09-17T22:43:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570168+0000", 2024-09-17T22:43:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029273114999999999 2024-09-17T22:43:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570208+0000", 2024-09-17T22:43:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9227999999999998e-05 2024-09-17T22:43:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570246+0000", 2024-09-17T22:43:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8253000000000001e-05 2024-09-17T22:43:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187441 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8617984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:43:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.630376837999997, 2024-09-17T22:43:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029442211999999999, 2024-09-17T22:43:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187441 2024-09-17T22:43:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:43:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:43:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:43:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:43:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540900+0000", 2024-09-17T22:43:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7999999999999993e-07 2024-09-17T22:43:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540903+0000", 2024-09-17T22:43:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9129999999999999e-06 2024-09-17T22:43:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540909+0000", 2024-09-17T22:43:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4269999999999999e-06 2024-09-17T22:43:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570267+0000", 2024-09-17T22:43:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029358308 2024-09-17T22:43:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570304+0000", 2024-09-17T22:43:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6860999999999998e-05 2024-09-17T22:43:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570341+0000", 2024-09-17T22:43:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7509000000000003e-05 2024-09-17T22:43:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187442 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8708096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:43:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.630364131, 2024-09-17T22:43:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029522606999999999, 2024-09-17T22:43:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187442 2024-09-17T22:43:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:43:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:43:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:43:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.22e-07 2024-09-17T22:43:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540917+0000", 2024-09-17T22:43:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999998e-06 2024-09-17T22:43:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540920+0000", 2024-09-17T22:43:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4989999999999999e-06 2024-09-17T22:43:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540923+0000", 2024-09-17T22:43:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3870000000000002e-06 2024-09-17T22:43:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570361+0000", 2024-09-17T22:43:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029438345000000001 2024-09-17T22:43:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570397+0000", 2024-09-17T22:43:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5429000000000002e-05 2024-09-17T22:43:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570434+0000", 2024-09-17T22:43:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7824999999999998e-05 2024-09-17T22:43:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187443 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8798208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:43:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.630350204000003, 2024-09-17T22:43:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029603640000000001, 2024-09-17T22:43:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187443 2024-09-17T22:43:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:43:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:43:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:43:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.05e-07 2024-09-17T22:43:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540927+0000", 2024-09-17T22:43:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:43:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540930+0000", 2024-09-17T22:43:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9229999999999998e-06 2024-09-17T22:43:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540932+0000", 2024-09-17T22:43:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.294e-06 2024-09-17T22:43:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570454+0000", 2024-09-17T22:43:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029522230999999999 2024-09-17T22:43:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570491+0000", 2024-09-17T22:43:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6835999999999998e-05 2024-09-17T22:43:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570529+0000", 2024-09-17T22:43:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8167999999999998e-05 2024-09-17T22:43:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187444 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8888320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:43:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.63034073, 2024-09-17T22:43:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029687584999999999, 2024-09-17T22:43:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187444 2024-09-17T22:43:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:43:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:43:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:43:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-09-17T22:43:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540936+0000", 2024-09-17T22:43:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3900000000000003e-07 2024-09-17T22:43:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540939+0000", 2024-09-17T22:43:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.994e-06 2024-09-17T22:43:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540942+0000", 2024-09-17T22:43:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1739999999999999e-06 2024-09-17T22:43:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570549+0000", 2024-09-17T22:43:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029607652000000002 2024-09-17T22:43:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570585+0000", 2024-09-17T22:43:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5775000000000002e-05 2024-09-17T22:43:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570623+0000", 2024-09-17T22:43:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7839999999999997e-05 2024-09-17T22:43:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187445 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8978432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:43:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.630331841999997, 2024-09-17T22:43:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029772837999999999, 2024-09-17T22:43:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187445 2024-09-17T22:43:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:43:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:43:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:43:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:43:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540945+0000", 2024-09-17T22:43:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3699999999999999e-07 2024-09-17T22:43:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540948+0000", 2024-09-17T22:43:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.898e-06 2024-09-17T22:43:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540950+0000", 2024-09-17T22:43:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2249999999999999e-06 2024-09-17T22:43:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570643+0000", 2024-09-17T22:43:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029692463999999998 2024-09-17T22:43:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570680+0000", 2024-09-17T22:43:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6772e-05 2024-09-17T22:43:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570717+0000", 2024-09-17T22:43:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7438999999999998e-05 2024-09-17T22:43:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187446 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9068544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:43:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.630323103999999, 2024-09-17T22:43:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029856321000000002, 2024-09-17T22:43:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187446 2024-09-17T22:43:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:43:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:43:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:43:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:43:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540958+0000", 2024-09-17T22:43:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3449999999999996e-06 2024-09-17T22:43:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540962+0000", 2024-09-17T22:43:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3529999999999999e-06 2024-09-17T22:43:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540964+0000", 2024-09-17T22:43:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3810000000000002e-06 2024-09-17T22:43:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570737+0000", 2024-09-17T22:43:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029772824 2024-09-17T22:43:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570770+0000", 2024-09-17T22:43:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2835e-05 2024-09-17T22:43:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570809+0000", 2024-09-17T22:43:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9379999999999999e-05 2024-09-17T22:43:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.246 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.246 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187447 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9158656~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:07.246 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:43:07.246 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.630309136000001, 2024-09-17T22:43:07.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02993262, 2024-09-17T22:43:07.246 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.246 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.246 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.246 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.246 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.246 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.246 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187447 2024-09-17T22:43:07.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.247 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:43:07.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:43:07.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:43:07.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-09-17T22:43:07.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540968+0000", 2024-09-17T22:43:07.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6500000000000008e-07 2024-09-17T22:43:07.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540971+0000", 2024-09-17T22:43:07.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9869999999999999e-06 2024-09-17T22:43:07.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540973+0000", 2024-09-17T22:43:07.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3120000000000001e-06 2024-09-17T22:43:07.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570829+0000", 2024-09-17T22:43:07.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029855742000000001 2024-09-17T22:43:07.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570861+0000", 2024-09-17T22:43:07.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2152e-05 2024-09-17T22:43:07.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570899+0000", 2024-09-17T22:43:07.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8241999999999997e-05 2024-09-17T22:43:07.250 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.250 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.250 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.250 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187448 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9248768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:07.250 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:43:07.250 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.630300106, 2024-09-17T22:43:07.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030014865000000002, 2024-09-17T22:43:07.250 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.250 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.250 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.250 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.251 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187448 2024-09-17T22:43:07.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.251 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:43:07.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:43:07.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:43:07.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9600000000000001e-07 2024-09-17T22:43:07.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540977+0000", 2024-09-17T22:43:07.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.16e-07 2024-09-17T22:43:07.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540980+0000", 2024-09-17T22:43:07.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.892e-06 2024-09-17T22:43:07.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540982+0000", 2024-09-17T22:43:07.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3410000000000001e-06 2024-09-17T22:43:07.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570919+0000", 2024-09-17T22:43:07.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029936800999999999 2024-09-17T22:43:07.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570953+0000", 2024-09-17T22:43:07.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3939000000000002e-05 2024-09-17T22:43:07.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570991+0000", 2024-09-17T22:43:07.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7780000000000001e-05 2024-09-17T22:43:07.254 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.254 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.254 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.254 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187449 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9338880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:07.254 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:43:07.254 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.630290926999997, 2024-09-17T22:43:07.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030110238000000001, 2024-09-17T22:43:07.254 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.254 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.254 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.254 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.255 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.255 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.255 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187449 2024-09-17T22:43:07.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.255 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:43:07.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:43:07.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:43:07.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1500000000000001e-07 2024-09-17T22:43:07.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540986+0000", 2024-09-17T22:43:07.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9500000000000001e-07 2024-09-17T22:43:07.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540989+0000", 2024-09-17T22:43:07.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7949999999999999e-06 2024-09-17T22:43:07.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540994+0000", 2024-09-17T22:43:07.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0359999999999997e-06 2024-09-17T22:43:07.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571011+0000", 2024-09-17T22:43:07.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030016915000000002 2024-09-17T22:43:07.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571056+0000", 2024-09-17T22:43:07.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5457999999999997e-05 2024-09-17T22:43:07.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571095+0000", 2024-09-17T22:43:07.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8924000000000002e-05 2024-09-17T22:43:07.258 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.258 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.258 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.258 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187450 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9428992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:07.258 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:43:07.258 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.630279280000003, 2024-09-17T22:43:07.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030188204999999999, 2024-09-17T22:43:07.259 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.259 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.259 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.259 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.259 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.259 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.259 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187450 2024-09-17T22:43:07.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.259 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:43:07.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:43:07.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:43:07.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.16e-07 2024-09-17T22:43:07.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540998+0000", 2024-09-17T22:43:07.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9000000000000005e-07 2024-09-17T22:43:07.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.541001+0000", 2024-09-17T22:43:07.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0680000000000001e-06 2024-09-17T22:43:07.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.541006+0000", 2024-09-17T22:43:07.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4720000000000003e-06 2024-09-17T22:43:07.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571110+0000", 2024-09-17T22:43:07.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030103431 2024-09-17T22:43:07.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571147+0000", 2024-09-17T22:43:07.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6748999999999997e-05 2024-09-17T22:43:07.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571185+0000", 2024-09-17T22:43:07.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8278999999999997e-05 2024-09-17T22:43:07.262 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.262 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.262 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.262 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.262 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.262 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:07.264 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:43:07.264 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:07.261+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:07.281 INFO:teuthology.orchestra.run.smithi115.stderr:no valid command found; 10 closest matches: 2024-09-17T22:43:07.281 INFO:teuthology.orchestra.run.smithi115.stderr:0 2024-09-17T22:43:07.281 INFO:teuthology.orchestra.run.smithi115.stderr:1 2024-09-17T22:43:07.282 INFO:teuthology.orchestra.run.smithi115.stderr:2 2024-09-17T22:43:07.282 INFO:teuthology.orchestra.run.smithi115.stderr:abort 2024-09-17T22:43:07.282 INFO:teuthology.orchestra.run.smithi115.stderr:assert 2024-09-17T22:43:07.282 INFO:teuthology.orchestra.run.smithi115.stderr:bench [] [] [] [] 2024-09-17T22:43:07.282 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs debug_inject_read_zeros 2024-09-17T22:43:07.282 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs files list 2024-09-17T22:43:07.282 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs stats 2024-09-17T22:43:07.282 INFO:teuthology.orchestra.run.smithi115.stderr:bluestore allocator dump block 2024-09-17T22:43:07.282 INFO:teuthology.orchestra.run.smithi115.stderr:Error EINVAL: invalid command 2024-09-17T22:43:07.284 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:43:07.284 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T22:43:07.365 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:07.361+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:07.384 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:07.384 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:43:07.384 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:43:07.384 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:07.393 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:43:07.465 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:07.463+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:07.495 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:07.496 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:43:07.496 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:43:07.496 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:43:07.496 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:07.504 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:43:07.588 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:07.586+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:07.649 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:07.649 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:43:07.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:43:07.649 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:43:07.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.649 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145246 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3850240~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.649 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:43:07.649 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.316037230999996, 2024-09-17T22:43:07.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052644991000000002, 2024-09-17T22:43:07.649 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.650 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.650 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.650 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145246 2024-09-17T22:43:07.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.650 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:43:07.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:43:07.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329899+0000", 2024-09-17T22:43:07.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-09-17T22:43:07.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329901+0000", 2024-09-17T22:43:07.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1619999999999998e-06 2024-09-17T22:43:07.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329907+0000", 2024-09-17T22:43:07.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9059999999999999e-06 2024-09-17T22:43:07.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329911+0000", 2024-09-17T22:43:07.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7419999999999997e-06 2024-09-17T22:43:07.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382383+0000", 2024-09-17T22:43:07.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052471583000000002 2024-09-17T22:43:07.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382457+0000", 2024-09-17T22:43:07.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4309999999999998e-05 2024-09-17T22:43:07.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382543+0000", 2024-09-17T22:43:07.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5918e-05 2024-09-17T22:43:07.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.653 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.654 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145247 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3946496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.654 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:43:07.654 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.316018580999994, 2024-09-17T22:43:07.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052805863000000001, 2024-09-17T22:43:07.654 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.654 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.654 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.654 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145247 2024-09-17T22:43:07.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.654 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:43:07.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:43:07.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:43:07.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3200000000000001e-07 2024-09-17T22:43:07.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329919+0000", 2024-09-17T22:43:07.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.888e-06 2024-09-17T22:43:07.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329925+0000", 2024-09-17T22:43:07.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5369999999999997e-06 2024-09-17T22:43:07.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329929+0000", 2024-09-17T22:43:07.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.639e-06 2024-09-17T22:43:07.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382570+0000", 2024-09-17T22:43:07.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052641213999999999 2024-09-17T22:43:07.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382636+0000", 2024-09-17T22:43:07.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6008999999999998e-05 2024-09-17T22:43:07.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382723+0000", 2024-09-17T22:43:07.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6243999999999996e-05 2024-09-17T22:43:07.658 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.658 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.658 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.658 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145248 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3856384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.658 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:43:07.658 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.316000532999993, 2024-09-17T22:43:07.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053002858, 2024-09-17T22:43:07.658 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.658 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.658 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145248 2024-09-17T22:43:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:43:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:43:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:43:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.13e-07 2024-09-17T22:43:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329937+0000", 2024-09-17T22:43:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9300000000000002e-06 2024-09-17T22:43:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329943+0000", 2024-09-17T22:43:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-09-17T22:43:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329948+0000", 2024-09-17T22:43:07.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8579999999999999e-06 2024-09-17T22:43:07.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382763+0000", 2024-09-17T22:43:07.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052814537000000002 2024-09-17T22:43:07.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382840+0000", 2024-09-17T22:43:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6879999999999996e-05 2024-09-17T22:43:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382938+0000", 2024-09-17T22:43:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8159999999999995e-05 2024-09-17T22:43:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145249 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3952640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:43:07.662 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.315981632000003, 2024-09-17T22:43:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053188602000000001, 2024-09-17T22:43:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145249 2024-09-17T22:43:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:43:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:43:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:43:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3700000000000001e-07 2024-09-17T22:43:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329956+0000", 2024-09-17T22:43:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1430000000000001e-06 2024-09-17T22:43:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329962+0000", 2024-09-17T22:43:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8270000000000003e-06 2024-09-17T22:43:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329967+0000", 2024-09-17T22:43:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7249999999999997e-06 2024-09-17T22:43:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382969+0000", 2024-09-17T22:43:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053001952999999997 2024-09-17T22:43:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383040+0000", 2024-09-17T22:43:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1272999999999993e-05 2024-09-17T22:43:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383142+0000", 2024-09-17T22:43:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010234399999999999 2024-09-17T22:43:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145250 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3862528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:43:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.315962833, 2024-09-17T22:43:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053359537999999998, 2024-09-17T22:43:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145250 2024-09-17T22:43:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:43:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:43:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:43:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:43:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329975+0000", 2024-09-17T22:43:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2199999999999999e-06 2024-09-17T22:43:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329981+0000", 2024-09-17T22:43:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1020000000000002e-06 2024-09-17T22:43:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329991+0000", 2024-09-17T22:43:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6800000000000005e-06 2024-09-17T22:43:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383171+0000", 2024-09-17T22:43:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053180142 2024-09-17T22:43:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383242+0000", 2024-09-17T22:43:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1310000000000007e-05 2024-09-17T22:43:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383332+0000", 2024-09-17T22:43:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9703999999999994e-05 2024-09-17T22:43:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145251 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3958784~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:43:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.315937133000006, 2024-09-17T22:43:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053438537000000001, 2024-09-17T22:43:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145251 2024-09-17T22:43:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:43:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:43:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329999+0000", 2024-09-17T22:43:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-07 2024-09-17T22:43:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330001+0000", 2024-09-17T22:43:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:43:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330006+0000", 2024-09-17T22:43:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000004e-06 2024-09-17T22:43:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330011+0000", 2024-09-17T22:43:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4039999999999997e-06 2024-09-17T22:43:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383350+0000", 2024-09-17T22:43:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053339011999999998 2024-09-17T22:43:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383387+0000", 2024-09-17T22:43:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7085999999999997e-05 2024-09-17T22:43:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383437+0000", 2024-09-17T22:43:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9988000000000003e-05 2024-09-17T22:43:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145252 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3868672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:43:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.315918686000003, 2024-09-17T22:43:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053526724999999997, 2024-09-17T22:43:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145252 2024-09-17T22:43:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:43:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:43:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331054+0000", 2024-09-17T22:43:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010371670000000001 2024-09-17T22:43:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331059+0000", 2024-09-17T22:43:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1680000000000003e-06 2024-09-17T22:43:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331075+0000", 2024-09-17T22:43:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6028e-05 2024-09-17T22:43:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331084+0000", 2024-09-17T22:43:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3980000000000005e-06 2024-09-17T22:43:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383464+0000", 2024-09-17T22:43:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052379851999999998 2024-09-17T22:43:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383499+0000", 2024-09-17T22:43:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4610000000000002e-05 2024-09-17T22:43:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383543+0000", 2024-09-17T22:43:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4502000000000002e-05 2024-09-17T22:43:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145255 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3971072~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:43:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.314788594000007, 2024-09-17T22:43:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052677527000000002, 2024-09-17T22:43:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145255 2024-09-17T22:43:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:43:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:43:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:43:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8800000000000002e-07 2024-09-17T22:43:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331150+0000", 2024-09-17T22:43:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1160000000000002e-06 2024-09-17T22:43:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331156+0000", 2024-09-17T22:43:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7469999999999999e-06 2024-09-17T22:43:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331162+0000", 2024-09-17T22:43:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-17T22:43:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383750+0000", 2024-09-17T22:43:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052588441999999999 2024-09-17T22:43:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383782+0000", 2024-09-17T22:43:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1610999999999999e-05 2024-09-17T22:43:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383824+0000", 2024-09-17T22:43:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2512000000000002e-05 2024-09-17T22:43:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145256 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3880960~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:43:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.314767173000007, 2024-09-17T22:43:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052744775000000001, 2024-09-17T22:43:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145256 2024-09-17T22:43:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:43:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:43:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331169+0000", 2024-09-17T22:43:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:43:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331175+0000", 2024-09-17T22:43:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5660000000000003e-06 2024-09-17T22:43:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331182+0000", 2024-09-17T22:43:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0169999999999999e-06 2024-09-17T22:43:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331188+0000", 2024-09-17T22:43:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2750000000000003e-06 2024-09-17T22:43:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383839+0000", 2024-09-17T22:43:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052651728000000002 2024-09-17T22:43:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383872+0000", 2024-09-17T22:43:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2347e-05 2024-09-17T22:43:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383913+0000", 2024-09-17T22:43:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1362000000000001e-05 2024-09-17T22:43:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145257 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3977216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:43:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.314741124999998, 2024-09-17T22:43:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052809598999999999, 2024-09-17T22:43:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145257 2024-09-17T22:43:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:43:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:43:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333051+0000", 2024-09-17T22:43:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018567189999999999 2024-09-17T22:43:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333056+0000", 2024-09-17T22:43:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-17T22:43:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333067+0000", 2024-09-17T22:43:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1704e-05 2024-09-17T22:43:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333074+0000", 2024-09-17T22:43:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1280000000000002e-06 2024-09-17T22:43:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383927+0000", 2024-09-17T22:43:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050853094000000001 2024-09-17T22:43:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383960+0000", 2024-09-17T22:43:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2891000000000001e-05 2024-09-17T22:43:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.384004+0000", 2024-09-17T22:43:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3565e-05 2024-09-17T22:43:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146168 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2467840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:43:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.251252504000007, 2024-09-17T22:43:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052652501999999997, 2024-09-17T22:43:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146168 2024-09-17T22:43:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:43:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:43:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394989+0000", 2024-09-17T22:43:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030618300000000002 2024-09-17T22:43:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394994+0000", 2024-09-17T22:43:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.566e-06 2024-09-17T22:43:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395003+0000", 2024-09-17T22:43:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3619999999999995e-06 2024-09-17T22:43:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395009+0000", 2024-09-17T22:43:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3589999999999996e-06 2024-09-17T22:43:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447180+0000", 2024-09-17T22:43:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052170643000000003 2024-09-17T22:43:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447251+0000", 2024-09-17T22:43:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1382000000000002e-05 2024-09-17T22:43:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447335+0000", 2024-09-17T22:43:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4006999999999997e-05 2024-09-17T22:43:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146172 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2480128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:43:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.250897619, 2024-09-17T22:43:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052714629999999998, 2024-09-17T22:43:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146172 2024-09-17T22:43:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:43:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:43:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395486+0000", 2024-09-17T22:43:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044787000000000001 2024-09-17T22:43:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395491+0000", 2024-09-17T22:43:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8550000000000001e-06 2024-09-17T22:43:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395503+0000", 2024-09-17T22:43:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2271e-05 2024-09-17T22:43:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395510+0000", 2024-09-17T22:43:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6320000000000008e-06 2024-09-17T22:43:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447565+0000", 2024-09-17T22:43:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052055016000000003 2024-09-17T22:43:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447644+0000", 2024-09-17T22:43:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8306000000000005e-05 2024-09-17T22:43:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447752+0000", 2024-09-17T22:43:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010868 2024-09-17T22:43:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146181 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2504704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:43:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.250030229000004, 2024-09-17T22:43:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052850394000000002, 2024-09-17T22:43:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146181 2024-09-17T22:43:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:43:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:43:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396210+0000", 2024-09-17T22:43:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000305214 2024-09-17T22:43:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396214+0000", 2024-09-17T22:43:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9389999999999999e-06 2024-09-17T22:43:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396228+0000", 2024-09-17T22:43:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3365e-05 2024-09-17T22:43:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396235+0000", 2024-09-17T22:43:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2559999999999996e-06 2024-09-17T22:43:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448568+0000", 2024-09-17T22:43:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052333084000000002 2024-09-17T22:43:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448646+0000", 2024-09-17T22:43:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8287000000000004e-05 2024-09-17T22:43:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448756+0000", 2024-09-17T22:43:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109249 2024-09-17T22:43:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146183 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2510848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:43:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.249689681999996, 2024-09-17T22:43:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052725453999999998, 2024-09-17T22:43:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146183 2024-09-17T22:43:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:43:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:43:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396508+0000", 2024-09-17T22:43:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026271299999999998 2024-09-17T22:43:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396512+0000", 2024-09-17T22:43:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8039999999999999e-06 2024-09-17T22:43:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396522+0000", 2024-09-17T22:43:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9289999999999993e-06 2024-09-17T22:43:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396528+0000", 2024-09-17T22:43:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2990000000000002e-06 2024-09-17T22:43:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448798+0000", 2024-09-17T22:43:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052269985999999997 2024-09-17T22:43:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448871+0000", 2024-09-17T22:43:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2581000000000002e-05 2024-09-17T22:43:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448971+0000", 2024-09-17T22:43:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100142 2024-09-17T22:43:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146391 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:43:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.241962189999995, 2024-09-17T22:43:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052666719000000001, 2024-09-17T22:43:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146391 2024-09-17T22:43:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:43:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:43:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404098+0000", 2024-09-17T22:43:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000124509 2024-09-17T22:43:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404107+0000", 2024-09-17T22:43:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4220000000000006e-06 2024-09-17T22:43:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404117+0000", 2024-09-17T22:43:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011e-05 2024-09-17T22:43:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404123+0000", 2024-09-17T22:43:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1530000000000002e-06 2024-09-17T22:43:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456481+0000", 2024-09-17T22:43:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052357923000000001 2024-09-17T22:43:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456551+0000", 2024-09-17T22:43:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9869000000000005e-05 2024-09-17T22:43:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456640+0000", 2024-09-17T22:43:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8733000000000006e-05 2024-09-17T22:43:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146393 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2748416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:43:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.241802127, 2024-09-17T22:43:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052739239, 2024-09-17T22:43:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146393 2024-09-17T22:43:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:43:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:43:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404443+0000", 2024-09-17T22:43:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030977199999999999 2024-09-17T22:43:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404447+0000", 2024-09-17T22:43:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6370000000000001e-06 2024-09-17T22:43:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404469+0000", 2024-09-17T22:43:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2739999999999999e-05 2024-09-17T22:43:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404477+0000", 2024-09-17T22:43:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6559999999999992e-06 2024-09-17T22:43:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456734+0000", 2024-09-17T22:43:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052256889000000001 2024-09-17T22:43:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456796+0000", 2024-09-17T22:43:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2485999999999998e-05 2024-09-17T22:43:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456873+0000", 2024-09-17T22:43:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6058999999999998e-05 2024-09-17T22:43:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146397 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2760704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:43:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.241426363000002, 2024-09-17T22:43:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052788928999999998, 2024-09-17T22:43:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146397 2024-09-17T22:43:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:43:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:43:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404763+0000", 2024-09-17T22:43:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025351399999999999 2024-09-17T22:43:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404767+0000", 2024-09-17T22:43:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:43:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404777+0000", 2024-09-17T22:43:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9310000000000003e-06 2024-09-17T22:43:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404783+0000", 2024-09-17T22:43:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.336e-06 2024-09-17T22:43:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457126+0000", 2024-09-17T22:43:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052342975999999999 2024-09-17T22:43:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457200+0000", 2024-09-17T22:43:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4146000000000005e-05 2024-09-17T22:43:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457298+0000", 2024-09-17T22:43:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7807999999999996e-05 2024-09-17T22:43:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146399 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2766848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:43:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.241136221999994, 2024-09-17T22:43:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052829779, 2024-09-17T22:43:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146399 2024-09-17T22:43:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:43:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:43:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404800+0000", 2024-09-17T22:43:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8500000000000001e-07 2024-09-17T22:43:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404802+0000", 2024-09-17T22:43:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5349999999999999e-06 2024-09-17T22:43:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404811+0000", 2024-09-17T22:43:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9390000000000003e-06 2024-09-17T22:43:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404817+0000", 2024-09-17T22:43:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.524e-06 2024-09-17T22:43:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457461+0000", 2024-09-17T22:43:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052643784999999998 2024-09-17T22:43:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457539+0000", 2024-09-17T22:43:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8589000000000003e-05 2024-09-17T22:43:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457629+0000", 2024-09-17T22:43:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9821999999999995e-05 2024-09-17T22:43:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146401 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2772992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:43:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.241110974999998, 2024-09-17T22:43:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052910077, 2024-09-17T22:43:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146401 2024-09-17T22:43:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:43:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:43:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405106+0000", 2024-09-17T22:43:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000281286 2024-09-17T22:43:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405110+0000", 2024-09-17T22:43:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1459999999999998e-06 2024-09-17T22:43:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405120+0000", 2024-09-17T22:43:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0546e-05 2024-09-17T22:43:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405127+0000", 2024-09-17T22:43:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4300000000000003e-06 2024-09-17T22:43:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457647+0000", 2024-09-17T22:43:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052520229000000002 2024-09-17T22:43:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457684+0000", 2024-09-17T22:43:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6609999999999997e-05 2024-09-17T22:43:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457735+0000", 2024-09-17T22:43:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.083e-05 2024-09-17T22:43:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146402 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2779136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:43:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.240797563000001, 2024-09-17T22:43:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052689812000000003, 2024-09-17T22:43:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146402 2024-09-17T22:43:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:43:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:43:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405365+0000", 2024-09-17T22:43:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022744599999999999 2024-09-17T22:43:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405370+0000", 2024-09-17T22:43:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2769999999999999e-06 2024-09-17T22:43:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405380+0000", 2024-09-17T22:43:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0349e-05 2024-09-17T22:43:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405386+0000", 2024-09-17T22:43:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5169999999999996e-06 2024-09-17T22:43:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457750+0000", 2024-09-17T22:43:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052363448999999999 2024-09-17T22:43:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457784+0000", 2024-09-17T22:43:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4363999999999998e-05 2024-09-17T22:43:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457828+0000", 2024-09-17T22:43:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999999e-05 2024-09-17T22:43:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:07.739 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:07.740 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:43:07.741 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:07.737+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:07.841 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:07.838+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:07.883 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:07.883 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:43:07.883 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:43:07.883 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:07.892 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:43:07.942 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:07.939+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:07.994 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:07.994 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:43:07.994 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:43:07.994 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:43:07.995 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:08.003 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:43:08.043 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:43:08.040+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:43:08.114 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:08.114 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:43:08.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:43:08.114 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:43:08.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.115 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4229.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-17T22:43:08.115 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:43:08.115 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 118.716686618, 2024-09-17T22:43:08.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25708877200000002, 2024-09-17T22:43:08.115 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.115 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.115 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.115 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.115 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4229", 2024-09-17T22:43:08.115 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/3080427038", 2024-09-17T22:43:08.115 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2 2024-09-17T22:43:08.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.115 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:43:08.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:43:08.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394611+0000", 2024-09-17T22:43:08.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3149999999999999e-06 2024-09-17T22:43:08.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394619+0000", 2024-09-17T22:43:08.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240000000000003e-06 2024-09-17T22:43:08.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394632+0000", 2024-09-17T22:43:08.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2921e-05 2024-09-17T22:43:08.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394639+0000", 2024-09-17T22:43:08.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-09-17T22:43:08.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394703+0000", 2024-09-17T22:43:08.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4407999999999997e-05 2024-09-17T22:43:08.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:43:08.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394711+0000", 2024-09-17T22:43:08.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5939999999999996e-06 2024-09-17T22:43:08.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651545+0000", 2024-09-17T22:43:08.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25683409200000001 2024-09-17T22:43:08.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651698+0000", 2024-09-17T22:43:08.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000153678 2024-09-17T22:43:08.119 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.119 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.119 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.120 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38736 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:08.120 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:43:08.120 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.334090613000001, 2024-09-17T22:43:08.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051395418999999998, 2024-09-17T22:43:08.120 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.120 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.120 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:08.120 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.120 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:08.120 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.120 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38736 2024-09-17T22:43:08.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.120 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:43:08.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:43:08.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:43:08.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6899999999999998e-07 2024-09-17T22:43:08.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777212+0000", 2024-09-17T22:43:08.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1240000000000002e-06 2024-09-17T22:43:08.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777219+0000", 2024-09-17T22:43:08.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5409999999999997e-06 2024-09-17T22:43:08.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777224+0000", 2024-09-17T22:43:08.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:43:08.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.826131+0000", 2024-09-17T22:43:08.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048907369999999999 2024-09-17T22:43:08.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:08.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828457+0000", 2024-09-17T22:43:08.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023256209999999999 2024-09-17T22:43:08.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828601+0000", 2024-09-17T22:43:08.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014438799999999999 2024-09-17T22:43:08.123 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.124 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.124 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38752 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 673792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:08.124 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:43:08.124 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.334066542000002, 2024-09-17T22:43:08.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051582301999999997, 2024-09-17T22:43:08.124 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.124 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.124 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:08.124 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.124 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:08.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.125 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38752 2024-09-17T22:43:08.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.125 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:43:08.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:43:08.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:43:08.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3000000000000001e-07 2024-09-17T22:43:08.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777232+0000", 2024-09-17T22:43:08.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9709999999999998e-06 2024-09-17T22:43:08.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777238+0000", 2024-09-17T22:43:08.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7209999999999999e-06 2024-09-17T22:43:08.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777243+0000", 2024-09-17T22:43:08.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5210000000000004e-06 2024-09-17T22:43:08.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828633+0000", 2024-09-17T22:43:08.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051390915000000002 2024-09-17T22:43:08.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:08.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828711+0000", 2024-09-17T22:43:08.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7474999999999999e-05 2024-09-17T22:43:08.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828812+0000", 2024-09-17T22:43:08.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010116900000000001 2024-09-17T22:43:08.128 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.128 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.128 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.128 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38753 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:08.128 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:43:08.128 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.334047268999996, 2024-09-17T22:43:08.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051770444999999998, 2024-09-17T22:43:08.129 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.129 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.129 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:08.129 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.129 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:08.129 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.129 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38753 2024-09-17T22:43:08.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.129 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:43:08.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:43:08.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:43:08.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:43:08.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777251+0000", 2024-09-17T22:43:08.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8059999999999999e-06 2024-09-17T22:43:08.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777257+0000", 2024-09-17T22:43:08.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2480000000000001e-06 2024-09-17T22:43:08.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777262+0000", 2024-09-17T22:43:08.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000004e-06 2024-09-17T22:43:08.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828842+0000", 2024-09-17T22:43:08.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051580203999999998 2024-09-17T22:43:08.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:08.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828917+0000", 2024-09-17T22:43:08.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4980000000000004e-05 2024-09-17T22:43:08.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829019+0000", 2024-09-17T22:43:08.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102248 2024-09-17T22:43:08.133 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.133 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.136 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.136 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38766 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:08.136 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:43:08.137 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.334028283999999, 2024-09-17T22:43:08.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051958839999999999, 2024-09-17T22:43:08.137 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.137 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.137 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:08.137 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.137 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:08.137 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.137 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38766 2024-09-17T22:43:08.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.137 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:43:08.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:43:08.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777269+0000", 2024-09-17T22:43:08.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:43:08.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777271+0000", 2024-09-17T22:43:08.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-17T22:43:08.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777276+0000", 2024-09-17T22:43:08.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:43:08.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777281+0000", 2024-09-17T22:43:08.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:43:08.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829055+0000", 2024-09-17T22:43:08.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051773570999999997 2024-09-17T22:43:08.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:08.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829128+0000", 2024-09-17T22:43:08.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3131000000000002e-05 2024-09-17T22:43:08.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829227+0000", 2024-09-17T22:43:08.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8800000000000003e-05 2024-09-17T22:43:08.140 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.140 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.141 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.141 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38767 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:08.141 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:43:08.141 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.334008300999997, 2024-09-17T22:43:08.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137018, 2024-09-17T22:43:08.141 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.141 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.141 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:08.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:08.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.142 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38767 2024-09-17T22:43:08.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.142 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:43:08.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:43:08.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:43:08.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:43:08.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777290+0000", 2024-09-17T22:43:08.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.88e-06 2024-09-17T22:43:08.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777296+0000", 2024-09-17T22:43:08.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-09-17T22:43:08.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777300+0000", 2024-09-17T22:43:08.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4510000000000002e-06 2024-09-17T22:43:08.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829257+0000", 2024-09-17T22:43:08.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051956975000000002 2024-09-17T22:43:08.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:08.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829328+0000", 2024-09-17T22:43:08.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1026999999999996e-05 2024-09-17T22:43:08.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829425+0000", 2024-09-17T22:43:08.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.679e-05 2024-09-17T22:43:08.145 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.145 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.145 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.145 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38784 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 675840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:08.145 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:43:08.145 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.333989846999998, 2024-09-17T22:43:08.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052344522999999997, 2024-09-17T22:43:08.146 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.146 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.146 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:08.146 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.146 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:08.146 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.146 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38784 2024-09-17T22:43:08.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.146 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:43:08.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:43:08.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777307+0000", 2024-09-17T22:43:08.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:43:08.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777309+0000", 2024-09-17T22:43:08.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.063e-06 2024-09-17T22:43:08.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777314+0000", 2024-09-17T22:43:08.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-17T22:43:08.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777319+0000", 2024-09-17T22:43:08.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999996e-06 2024-09-17T22:43:08.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829456+0000", 2024-09-17T22:43:08.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137151 2024-09-17T22:43:08.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:08.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829527+0000", 2024-09-17T22:43:08.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1588000000000006e-05 2024-09-17T22:43:08.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829651+0000", 2024-09-17T22:43:08.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012365 2024-09-17T22:43:08.150 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.150 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.150 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.150 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38785 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 749568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:08.150 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:43:08.150 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.333971731999995, 2024-09-17T22:43:08.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052546572999999999, 2024-09-17T22:43:08.150 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.150 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.150 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:08.150 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.150 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:08.151 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.151 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38785 2024-09-17T22:43:08.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.151 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:43:08.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:43:08.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:43:08.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:43:08.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777327+0000", 2024-09-17T22:43:08.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:43:08.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777333+0000", 2024-09-17T22:43:08.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7309999999999998e-06 2024-09-17T22:43:08.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777337+0000", 2024-09-17T22:43:08.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:43:08.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829686+0000", 2024-09-17T22:43:08.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052349159999999999 2024-09-17T22:43:08.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:08.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829766+0000", 2024-09-17T22:43:08.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9326999999999995e-05 2024-09-17T22:43:08.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829871+0000", 2024-09-17T22:43:08.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000105466 2024-09-17T22:43:08.154 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.154 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.154 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.154 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38798 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:08.154 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:43:08.154 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.333338123999994, 2024-09-17T22:43:08.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052117564999999998, 2024-09-17T22:43:08.155 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.155 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.155 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:08.155 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.155 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:08.155 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.155 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38798 2024-09-17T22:43:08.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.155 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:43:08.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:43:08.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777959+0000", 2024-09-17T22:43:08.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9599999999999999e-07 2024-09-17T22:43:08.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777963+0000", 2024-09-17T22:43:08.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2660000000000003e-06 2024-09-17T22:43:08.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777971+0000", 2024-09-17T22:43:08.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3489999999999992e-06 2024-09-17T22:43:08.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777978+0000", 2024-09-17T22:43:08.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:43:08.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829906+0000", 2024-09-17T22:43:08.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051927635999999999 2024-09-17T22:43:08.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:08.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829986+0000", 2024-09-17T22:43:08.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0444000000000004e-05 2024-09-17T22:43:08.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830076+0000", 2024-09-17T22:43:08.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9473000000000002e-05 2024-09-17T22:43:08.158 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.158 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.158 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.159 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38799 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 882688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:08.159 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:43:08.159 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.333306684999997, 2024-09-17T22:43:08.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052359606000000003, 2024-09-17T22:43:08.159 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.159 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.159 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:08.159 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.159 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:08.159 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.159 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38799 2024-09-17T22:43:08.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.159 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:43:08.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:43:08.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:43:08.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8299999999999997e-07 2024-09-17T22:43:08.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777992+0000", 2024-09-17T22:43:08.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.176e-06 2024-09-17T22:43:08.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777998+0000", 2024-09-17T22:43:08.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.173e-06 2024-09-17T22:43:08.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778003+0000", 2024-09-17T22:43:08.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6349999999999997e-06 2024-09-17T22:43:08.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830112+0000", 2024-09-17T22:43:08.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052109366999999997 2024-09-17T22:43:08.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:08.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830192+0000", 2024-09-17T22:43:08.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9814000000000005e-05 2024-09-17T22:43:08.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830349+0000", 2024-09-17T22:43:08.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015695800000000001 2024-09-17T22:43:08.163 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.163 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.163 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.163 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38814 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:08.163 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:43:08.163 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.333023280999996, 2024-09-17T22:43:08.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052274339000000003, 2024-09-17T22:43:08.163 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.163 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.163 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:08.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.164 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:08.164 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.164 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38814 2024-09-17T22:43:08.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.164 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:43:08.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:43:08.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:43:08.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:43:08.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778276+0000", 2024-09-17T22:43:08.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2029999999999999e-06 2024-09-17T22:43:08.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778282+0000", 2024-09-17T22:43:08.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0059999999999998e-06 2024-09-17T22:43:08.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778287+0000", 2024-09-17T22:43:08.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0039999999999999e-06 2024-09-17T22:43:08.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830382+0000", 2024-09-17T22:43:08.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052095379999999997 2024-09-17T22:43:08.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:08.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830455+0000", 2024-09-17T22:43:08.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3127000000000006e-05 2024-09-17T22:43:08.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830547+0000", 2024-09-17T22:43:08.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2207999999999995e-05 2024-09-17T22:43:08.167 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.167 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.167 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.167 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38828 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 163840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:08.167 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:43:08.167 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.332840872999995, 2024-09-17T22:43:08.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052466039999999999, 2024-09-17T22:43:08.168 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.168 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.168 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:08.168 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.168 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:08.168 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.168 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38828 2024-09-17T22:43:08.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.168 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:43:08.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:43:08.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778456+0000", 2024-09-17T22:43:08.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7599999999999996e-07 2024-09-17T22:43:08.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778460+0000", 2024-09-17T22:43:08.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8229999999999997e-06 2024-09-17T22:43:08.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778470+0000", 2024-09-17T22:43:08.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8199999999999992e-06 2024-09-17T22:43:08.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778476+0000", 2024-09-17T22:43:08.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2589999999999997e-06 2024-09-17T22:43:08.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830578+0000", 2024-09-17T22:43:08.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052102273999999997 2024-09-17T22:43:08.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:08.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830667+0000", 2024-09-17T22:43:08.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8701000000000003e-05 2024-09-17T22:43:08.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830921+0000", 2024-09-17T22:43:08.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025438700000000002 2024-09-17T22:43:08.171 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.171 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.171 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.171 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38829 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:08.171 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:43:08.172 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.332809475999994, 2024-09-17T22:43:08.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052648882000000001, 2024-09-17T22:43:08.172 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.172 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.172 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:08.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:08.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.172 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38829 2024-09-17T22:43:08.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.172 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:43:08.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:43:08.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:43:08.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7e-07 2024-09-17T22:43:08.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778490+0000", 2024-09-17T22:43:08.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:43:08.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778495+0000", 2024-09-17T22:43:08.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9329999999999998e-06 2024-09-17T22:43:08.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778500+0000", 2024-09-17T22:43:08.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9509999999999997e-06 2024-09-17T22:43:08.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830958+0000", 2024-09-17T22:43:08.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05245785 2024-09-17T22:43:08.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:08.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831051+0000", 2024-09-17T22:43:08.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2989e-05 2024-09-17T22:43:08.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831136+0000", 2024-09-17T22:43:08.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4448000000000002e-05 2024-09-17T22:43:08.175 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.175 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.175 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.176 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38891 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:08.176 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:43:08.176 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.329497418000003, 2024-09-17T22:43:08.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050847934999999997, 2024-09-17T22:43:08.176 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.176 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.176 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:08.176 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.176 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:08.176 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.176 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38891 2024-09-17T22:43:08.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.176 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:43:08.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:43:08.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:43:08.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-09-17T22:43:08.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781802+0000", 2024-09-17T22:43:08.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:43:08.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781810+0000", 2024-09-17T22:43:08.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8019999999999992e-06 2024-09-17T22:43:08.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781816+0000", 2024-09-17T22:43:08.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.285e-06 2024-09-17T22:43:08.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832485+0000", 2024-09-17T22:43:08.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050668462999999997 2024-09-17T22:43:08.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:08.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832565+0000", 2024-09-17T22:43:08.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9692999999999997e-05 2024-09-17T22:43:08.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832647+0000", 2024-09-17T22:43:08.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2224000000000004e-05 2024-09-17T22:43:08.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.179 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.180 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.180 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38892 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:08.180 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:43:08.180 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.329469169000006, 2024-09-17T22:43:08.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050946120999999997, 2024-09-17T22:43:08.180 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.180 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.180 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:08.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:08.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.180 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38892 2024-09-17T22:43:08.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.181 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:43:08.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:43:08.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781828+0000", 2024-09-17T22:43:08.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5400000000000002e-07 2024-09-17T22:43:08.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781830+0000", 2024-09-17T22:43:08.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1959999999999998e-06 2024-09-17T22:43:08.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781837+0000", 2024-09-17T22:43:08.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8310000000000003e-06 2024-09-17T22:43:08.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781843+0000", 2024-09-17T22:43:08.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1e-06 2024-09-17T22:43:08.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832674+0000", 2024-09-17T22:43:08.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050831182000000003 2024-09-17T22:43:08.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:08.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832725+0000", 2024-09-17T22:43:08.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1588000000000001e-05 2024-09-17T22:43:08.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832773+0000", 2024-09-17T22:43:08.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7769999999999998e-05 2024-09-17T22:43:08.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.184 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.184 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38909 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 36864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:08.184 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:43:08.184 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.329446399000005, 2024-09-17T22:43:08.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051046518999999999, 2024-09-17T22:43:08.184 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.184 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.184 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:08.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:08.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.185 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38909 2024-09-17T22:43:08.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.185 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:43:08.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:43:08.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:43:08.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:43:08.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781852+0000", 2024-09-17T22:43:08.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0590000000000001e-06 2024-09-17T22:43:08.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781859+0000", 2024-09-17T22:43:08.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3380000000000001e-06 2024-09-17T22:43:08.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781864+0000", 2024-09-17T22:43:08.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:43:08.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832804+0000", 2024-09-17T22:43:08.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050939614000000001 2024-09-17T22:43:08.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:08.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832845+0000", 2024-09-17T22:43:08.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1014999999999999e-05 2024-09-17T22:43:08.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832896+0000", 2024-09-17T22:43:08.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1836999999999999e-05 2024-09-17T22:43:08.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.188 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.188 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38910 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:08.188 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:43:08.189 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.329426088999995, 2024-09-17T22:43:08.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051216818999999997, 2024-09-17T22:43:08.189 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.189 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.189 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:08.189 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.189 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:08.189 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.189 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38910 2024-09-17T22:43:08.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.189 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:43:08.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:43:08.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781871+0000", 2024-09-17T22:43:08.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:43:08.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781873+0000", 2024-09-17T22:43:08.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:43:08.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781879+0000", 2024-09-17T22:43:08.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0700000000000003e-06 2024-09-17T22:43:08.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781884+0000", 2024-09-17T22:43:08.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-09-17T22:43:08.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832932+0000", 2024-09-17T22:43:08.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051048219999999998 2024-09-17T22:43:08.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:08.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833001+0000", 2024-09-17T22:43:08.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8504999999999995e-05 2024-09-17T22:43:08.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833087+0000", 2024-09-17T22:43:08.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6464000000000004e-05 2024-09-17T22:43:08.192 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.192 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.192 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.193 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38923 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:08.193 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:43:08.193 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.329405769000005, 2024-09-17T22:43:08.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051379573999999997, 2024-09-17T22:43:08.193 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.193 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.193 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:08.193 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.193 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:08.193 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.193 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38923 2024-09-17T22:43:08.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.193 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:43:08.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:43:08.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:43:08.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:43:08.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781893+0000", 2024-09-17T22:43:08.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-09-17T22:43:08.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781900+0000", 2024-09-17T22:43:08.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3860000000000004e-06 2024-09-17T22:43:08.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781904+0000", 2024-09-17T22:43:08.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9300000000000002e-06 2024-09-17T22:43:08.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833115+0000", 2024-09-17T22:43:08.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051210845999999997 2024-09-17T22:43:08.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:08.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833188+0000", 2024-09-17T22:43:08.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2276999999999999e-05 2024-09-17T22:43:08.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833270+0000", 2024-09-17T22:43:08.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2501000000000002e-05 2024-09-17T22:43:08.197 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.197 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.197 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.197 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38941 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:08.197 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:43:08.198 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.328613234000002, 2024-09-17T22:43:08.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050969459000000002, 2024-09-17T22:43:08.198 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.198 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.198 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:08.198 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.198 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:08.198 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.198 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38941 2024-09-17T22:43:08.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.198 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:43:08.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:43:08.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782684+0000", 2024-09-17T22:43:08.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:43:08.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782686+0000", 2024-09-17T22:43:08.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.396e-06 2024-09-17T22:43:08.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782693+0000", 2024-09-17T22:43:08.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1999999999999997e-06 2024-09-17T22:43:08.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782699+0000", 2024-09-17T22:43:08.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7910000000000001e-06 2024-09-17T22:43:08.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833471+0000", 2024-09-17T22:43:08.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050772333000000003 2024-09-17T22:43:08.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:08.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833536+0000", 2024-09-17T22:43:08.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4380000000000004e-05 2024-09-17T22:43:08.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833653+0000", 2024-09-17T22:43:08.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011691799999999999 2024-09-17T22:43:08.202 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.202 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.202 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.202 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:151189 3.14 3:2b696b13:::rbd_data.115f14f2d1d5.0000000000000009:e [read 3127296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.202 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:43:08.202 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 78.398392489000003, 2024-09-17T22:43:08.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.054254700000000003, 2024-09-17T22:43:08.202 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:08.202 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:08.202 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:08.202 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:08.203 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:08.203 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.203 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 151189 2024-09-17T22:43:08.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.203 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:08.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:08.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:43:08.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:08.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:43:08.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:08.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:08.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713051+0000", 2024-09-17T22:43:08.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014722100000000001 2024-09-17T22:43:08.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:08.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713055+0000", 2024-09-17T22:43:08.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.16e-06 2024-09-17T22:43:08.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:08.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713065+0000", 2024-09-17T22:43:08.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8390000000000002e-06 2024-09-17T22:43:08.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713073+0000", 2024-09-17T22:43:08.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0390000000000005e-06 2024-09-17T22:43:08.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:08.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762644+0000", 2024-09-17T22:43:08.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049571160000000003 2024-09-17T22:43:08.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:08.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762739+0000", 2024-09-17T22:43:08.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4565e-05 2024-09-17T22:43:08.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:08.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:08.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:08.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.767159+0000", 2024-09-17T22:43:08.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044197159999999997 2024-09-17T22:43:08.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.206 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:08.206 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:08.206 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:08.208 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:43:08.208 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:08.205+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:08.309 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:08.306+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:08.312 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:08.312 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:08.312 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:43:08.312 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:08.321 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:43:08.409 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:08.406+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:08.419 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:43:08.424 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:08.424 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:08.424 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:43:08.424 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:43:08.424 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:08.433 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:43:08.510 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:08.507+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:08.522 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:43:08.538 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:43:08.539 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:43:08.555 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:43:08.555 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:43:08.571 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:43:08.571 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:43:08.578 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:08.578 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:43:08.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:43:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:43:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158673 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:43:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333908054000005, 2024-09-17T22:43:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039400614, 2024-09-17T22:43:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158673 2024-09-17T22:43:08.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:43:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:43:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:43:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.04e-07 2024-09-17T22:43:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241563+0000", 2024-09-17T22:43:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5799999999999998e-07 2024-09-17T22:43:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241566+0000", 2024-09-17T22:43:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8279999999999999e-06 2024-09-17T22:43:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241568+0000", 2024-09-17T22:43:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2709999999999999e-06 2024-09-17T22:43:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280954+0000", 2024-09-17T22:43:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039386051999999998 2024-09-17T22:43:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280962+0000", 2024-09-17T22:43:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3010000000000007e-06 2024-09-17T22:43:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.582 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158675 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:43:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333898762999993, 2024-09-17T22:43:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039408033000000002, 2024-09-17T22:43:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158675 2024-09-17T22:43:08.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:43:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:43:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:43:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.36e-07 2024-09-17T22:43:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241572+0000", 2024-09-17T22:43:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6599999999999994e-07 2024-09-17T22:43:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241575+0000", 2024-09-17T22:43:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8389999999999998e-06 2024-09-17T22:43:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241577+0000", 2024-09-17T22:43:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4119999999999999e-06 2024-09-17T22:43:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280975+0000", 2024-09-17T22:43:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039397205999999997 2024-09-17T22:43:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280979+0000", 2024-09-17T22:43:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.374e-06 2024-09-17T22:43:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.586 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158677 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:43:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333876320000002, 2024-09-17T22:43:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039409307999999997, 2024-09-17T22:43:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158677 2024-09-17T22:43:08.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:43:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:43:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241594+0000", 2024-09-17T22:43:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:43:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241595+0000", 2024-09-17T22:43:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2729999999999999e-06 2024-09-17T22:43:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241602+0000", 2024-09-17T22:43:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:43:08.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241606+0000", 2024-09-17T22:43:08.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3019999999999999e-06 2024-09-17T22:43:08.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280994+0000", 2024-09-17T22:43:08.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039388484000000001 2024-09-17T22:43:08.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281003+0000", 2024-09-17T22:43:08.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5459999999999992e-06 2024-09-17T22:43:08.591 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.591 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.591 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.591 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158679 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.591 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:43:08.591 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333860642000005, 2024-09-17T22:43:08.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039413403999999999, 2024-09-17T22:43:08.591 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.591 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.591 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.591 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158679 2024-09-17T22:43:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:43:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:43:08.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:43:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4999999999999999e-07 2024-09-17T22:43:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241613+0000", 2024-09-17T22:43:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.207e-06 2024-09-17T22:43:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241616+0000", 2024-09-17T22:43:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.275e-06 2024-09-17T22:43:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241618+0000", 2024-09-17T22:43:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:43:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281017+0000", 2024-09-17T22:43:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039398385000000001 2024-09-17T22:43:08.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281022+0000", 2024-09-17T22:43:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.896e-06 2024-09-17T22:43:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158681 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:43:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333848463999999, 2024-09-17T22:43:08.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039425292000000001, 2024-09-17T22:43:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158681 2024-09-17T22:43:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:43:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:43:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:43:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:43:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241622+0000", 2024-09-17T22:43:08.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:43:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241625+0000", 2024-09-17T22:43:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9629999999999998e-06 2024-09-17T22:43:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241628+0000", 2024-09-17T22:43:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4339999999999999e-06 2024-09-17T22:43:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281039+0000", 2024-09-17T22:43:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039410672000000001 2024-09-17T22:43:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281047+0000", 2024-09-17T22:43:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9969999999999998e-06 2024-09-17T22:43:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158682 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:43:08.599 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333839179999998, 2024-09-17T22:43:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039436455000000002, 2024-09-17T22:43:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158682 2024-09-17T22:43:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:43:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:43:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:43:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:43:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241632+0000", 2024-09-17T22:43:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:43:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241635+0000", 2024-09-17T22:43:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8940000000000002e-06 2024-09-17T22:43:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241637+0000", 2024-09-17T22:43:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3659999999999999e-06 2024-09-17T22:43:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281061+0000", 2024-09-17T22:43:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039423664999999997 2024-09-17T22:43:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281067+0000", 2024-09-17T22:43:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.285e-06 2024-09-17T22:43:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.603 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158684 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:43:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333829833999999, 2024-09-17T22:43:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462980000000002, 2024-09-17T22:43:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158684 2024-09-17T22:43:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:43:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:43:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:43:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:43:08.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241641+0000", 2024-09-17T22:43:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:43:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241644+0000", 2024-09-17T22:43:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8870000000000001e-06 2024-09-17T22:43:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241646+0000", 2024-09-17T22:43:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3530000000000002e-06 2024-09-17T22:43:08.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281093+0000", 2024-09-17T22:43:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039446947000000003 2024-09-17T22:43:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281103+0000", 2024-09-17T22:43:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6299999999999993e-06 2024-09-17T22:43:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158686 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:43:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333820664000001, 2024-09-17T22:43:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039474214000000001, 2024-09-17T22:43:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158686 2024-09-17T22:43:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:43:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:43:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:43:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:43:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241650+0000", 2024-09-17T22:43:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2999999999999999e-07 2024-09-17T22:43:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241653+0000", 2024-09-17T22:43:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9689999999999999e-06 2024-09-17T22:43:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241655+0000", 2024-09-17T22:43:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3460000000000001e-06 2024-09-17T22:43:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281118+0000", 2024-09-17T22:43:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462876000000001 2024-09-17T22:43:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281123+0000", 2024-09-17T22:43:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8840000000000002e-06 2024-09-17T22:43:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.611 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158688 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:43:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333811432999994, 2024-09-17T22:43:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039490458999999999, 2024-09-17T22:43:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158688 2024-09-17T22:43:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.612 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:43:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:43:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:43:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:43:08.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241659+0000", 2024-09-17T22:43:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3699999999999999e-07 2024-09-17T22:43:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241662+0000", 2024-09-17T22:43:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8499999999999998e-06 2024-09-17T22:43:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241665+0000", 2024-09-17T22:43:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.356e-06 2024-09-17T22:43:08.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281141+0000", 2024-09-17T22:43:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039476031000000002 2024-09-17T22:43:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281149+0000", 2024-09-17T22:43:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0709999999999995e-06 2024-09-17T22:43:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.615 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158690 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:43:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333802374000001, 2024-09-17T22:43:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039502378999999997, 2024-09-17T22:43:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158690 2024-09-17T22:43:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:43:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:43:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241668+0000", 2024-09-17T22:43:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:43:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241669+0000", 2024-09-17T22:43:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1009999999999999e-06 2024-09-17T22:43:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241672+0000", 2024-09-17T22:43:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9059999999999998e-06 2024-09-17T22:43:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241674+0000", 2024-09-17T22:43:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4439999999999998e-06 2024-09-17T22:43:08.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281162+0000", 2024-09-17T22:43:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039488189 2024-09-17T22:43:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281170+0000", 2024-09-17T22:43:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5310000000000004e-06 2024-09-17T22:43:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158692 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:43:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333792940999999, 2024-09-17T22:43:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039531064999999997, 2024-09-17T22:43:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158692 2024-09-17T22:43:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:43:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:43:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:43:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:43:08.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241678+0000", 2024-09-17T22:43:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0800000000000003e-07 2024-09-17T22:43:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241681+0000", 2024-09-17T22:43:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8729999999999999e-06 2024-09-17T22:43:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241683+0000", 2024-09-17T22:43:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.447e-06 2024-09-17T22:43:08.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281199+0000", 2024-09-17T22:43:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039515388999999998 2024-09-17T22:43:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281208+0000", 2024-09-17T22:43:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2399999999999996e-06 2024-09-17T22:43:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.623 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158694 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:43:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333783769999997, 2024-09-17T22:43:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039543986000000003, 2024-09-17T22:43:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158694 2024-09-17T22:43:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:43:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:43:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:43:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:43:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241687+0000", 2024-09-17T22:43:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:43:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241691+0000", 2024-09-17T22:43:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.811e-06 2024-09-17T22:43:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241693+0000", 2024-09-17T22:43:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.385e-06 2024-09-17T22:43:08.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281223+0000", 2024-09-17T22:43:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039529332 2024-09-17T22:43:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281230+0000", 2024-09-17T22:43:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2139999999999999e-06 2024-09-17T22:43:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158696 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:43:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333773382000004, 2024-09-17T22:43:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039570458000000003, 2024-09-17T22:43:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158696 2024-09-17T22:43:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.628 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:43:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:43:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:43:08.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:43:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:43:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.33e-07 2024-09-17T22:43:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241700+0000", 2024-09-17T22:43:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8949999999999998e-06 2024-09-17T22:43:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241703+0000", 2024-09-17T22:43:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3159999999999999e-06 2024-09-17T22:43:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281257+0000", 2024-09-17T22:43:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039553862000000002 2024-09-17T22:43:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281267+0000", 2024-09-17T22:43:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.024e-05 2024-09-17T22:43:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158698 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:43:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333763304000001, 2024-09-17T22:43:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039576493999999997, 2024-09-17T22:43:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.632 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158698 2024-09-17T22:43:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:43:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:43:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241707+0000", 2024-09-17T22:43:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:43:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241708+0000", 2024-09-17T22:43:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5000000000000001e-07 2024-09-17T22:43:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241711+0000", 2024-09-17T22:43:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9670000000000001e-06 2024-09-17T22:43:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241713+0000", 2024-09-17T22:43:08.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2689999999999998e-06 2024-09-17T22:43:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281278+0000", 2024-09-17T22:43:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039565206999999998 2024-09-17T22:43:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281283+0000", 2024-09-17T22:43:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:43:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158700 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:43:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333754092999996, 2024-09-17T22:43:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039604292999999999, 2024-09-17T22:43:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158700 2024-09-17T22:43:08.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:43:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:43:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:43:08.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:43:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241717+0000", 2024-09-17T22:43:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.78e-07 2024-09-17T22:43:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241720+0000", 2024-09-17T22:43:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8119999999999999e-06 2024-09-17T22:43:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241722+0000", 2024-09-17T22:43:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-17T22:43:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281311+0000", 2024-09-17T22:43:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039588894999999999 2024-09-17T22:43:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281320+0000", 2024-09-17T22:43:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9989999999999997e-06 2024-09-17T22:43:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.639 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158702 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:43:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333742771000004, 2024-09-17T22:43:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039616142, 2024-09-17T22:43:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.640 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158702 2024-09-17T22:43:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:43:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:43:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:43:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4400000000000001e-07 2024-09-17T22:43:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241728+0000", 2024-09-17T22:43:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0839999999999999e-06 2024-09-17T22:43:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241731+0000", 2024-09-17T22:43:08.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0690000000000002e-06 2024-09-17T22:43:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241734+0000", 2024-09-17T22:43:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4159999999999998e-06 2024-09-17T22:43:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281334+0000", 2024-09-17T22:43:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039600110000000001 2024-09-17T22:43:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281343+0000", 2024-09-17T22:43:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2189999999999993e-06 2024-09-17T22:43:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158704 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:43:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333732906999998, 2024-09-17T22:43:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039641485999999997, 2024-09-17T22:43:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.644 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158704 2024-09-17T22:43:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:43:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.645 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:43:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:43:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:43:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241738+0000", 2024-09-17T22:43:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0589999999999999e-06 2024-09-17T22:43:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241741+0000", 2024-09-17T22:43:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.943e-06 2024-09-17T22:43:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241744+0000", 2024-09-17T22:43:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4819999999999999e-06 2024-09-17T22:43:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281370+0000", 2024-09-17T22:43:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039625661 2024-09-17T22:43:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281378+0000", 2024-09-17T22:43:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1310000000000006e-06 2024-09-17T22:43:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158705 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:43:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333722356999999, 2024-09-17T22:43:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039652637999999997, 2024-09-17T22:43:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.648 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158705 2024-09-17T22:43:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:43:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:43:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:43:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-07 2024-09-17T22:43:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:43:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0599999999999999e-07 2024-09-17T22:43:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.650 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241751+0000", 2024-09-17T22:43:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.926e-06 2024-09-17T22:43:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241754+0000", 2024-09-17T22:43:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4360000000000001e-06 2024-09-17T22:43:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281392+0000", 2024-09-17T22:43:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039638566 2024-09-17T22:43:08.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.652 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.652 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281400+0000", 2024-09-17T22:43:08.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.571e-06 2024-09-17T22:43:08.652 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.652 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.652 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.652 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.652 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158707 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.652 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:43:08.652 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333701255999998, 2024-09-17T22:43:08.652 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039654809999999999, 2024-09-17T22:43:08.652 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.653 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.653 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.653 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.653 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.653 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.653 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158707 2024-09-17T22:43:08.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.653 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.653 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.653 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.653 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:43:08.653 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.653 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:43:08.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241769+0000", 2024-09-17T22:43:08.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4299999999999999e-07 2024-09-17T22:43:08.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.654 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.654 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.654 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241770+0000", 2024-09-17T22:43:08.654 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3769999999999999e-06 2024-09-17T22:43:08.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241773+0000", 2024-09-17T22:43:08.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2109999999999998e-06 2024-09-17T22:43:08.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241776+0000", 2024-09-17T22:43:08.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5069999999999999e-06 2024-09-17T22:43:08.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.655 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.655 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.655 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281415+0000", 2024-09-17T22:43:08.655 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039639499000000002 2024-09-17T22:43:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281423+0000", 2024-09-17T22:43:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8730000000000007e-06 2024-09-17T22:43:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158709 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:43:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 78.333690646999997, 2024-09-17T22:43:08.656 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039661937000000001, 2024-09-17T22:43:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158709 2024-09-17T22:43:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:43:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:43:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242843+0000", 2024-09-17T22:43:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010634780000000001 2024-09-17T22:43:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242845+0000", 2024-09-17T22:43:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7209999999999999e-06 2024-09-17T22:43:08.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242851+0000", 2024-09-17T22:43:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6949999999999997e-06 2024-09-17T22:43:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242855+0000", 2024-09-17T22:43:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7960000000000002e-06 2024-09-17T22:43:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281434+0000", 2024-09-17T22:43:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038579519 2024-09-17T22:43:08.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281441+0000", 2024-09-17T22:43:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7279999999999998e-06 2024-09-17T22:43:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:08.660 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:08.660 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:08.661 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:43:08.661 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:43:08.662 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:43:08.662 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:43:08.663 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:43:08.663 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:43:08.663 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:43:08.664 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:43:08.664 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:43:08.664 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:43:08.665 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:43:08.665 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:08.659+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:08.762 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:08.760+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:08.769 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:08.769 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:08.778 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:43:08.863 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:08.861+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:08.880 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:08.880 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:08.889 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:43:08.964 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:08.961+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:08.992 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:08.992 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:09.001 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:43:09.085 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:43:09.082+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:43:09.087 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T22:43:09.149 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:09.149 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:09.159 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:43:09.166 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:09.163+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:09.267 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:09.263+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:09.302 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:09.302 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:09.311 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:43:09.396 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:43:09.392+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:43:09.455 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:09.455 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:09.464 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:43:09.505 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:43:09.502+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:43:09.569 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:43:09.566+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:43:09.608 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:09.608 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:09.617 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:43:09.670 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:09.667+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:09.761 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:09.761 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:09.773 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:43:09.774 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:43:09.771+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:43:09.875 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:09.872+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:09.876 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:09.877 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:09.885 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:43:09.976 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:09.973+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:09.993 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:09.993 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:10.002 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:43:10.076 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is set 2024-09-17T22:43:10.077 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:10.073+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:10.115 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:10.115 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:10.125 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:43:10.178 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:10.175+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:10.270 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:10.270 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:10.279 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:10.276+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:10.280 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:43:10.379 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:10.376+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:10.384 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:10.384 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:10.392 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:43:10.480 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:10.477+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:10.535 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:10.536 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:10.544 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:43:10.581 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:10.578+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:10.647 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:10.647 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:10.656 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:43:10.666 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:43:10.699 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:10.697+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:10.761 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:10.761 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:10.770 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:43:10.805 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:43:10.805 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:10.803+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:10.822 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:43:10.822 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:43:10.839 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:43:10.839 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:43:10.855 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:43:10.856 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:43:10.873 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:43:10.873 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:43:10.877 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:10.877 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:43:10.877 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:43:10.877 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:43:10.877 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:10.883 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:43:10.880+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:43:10.886 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:43:10.890 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:43:10.891 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:43:10.908 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:43:10.908 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:43:10.925 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:43:10.925 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:43:10.928 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:43:10.928 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:43:10.983 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:10.981+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:10.995 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:10.995 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:43:10.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:43:10.995 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:43:10.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:10.996 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4229.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e19)", 2024-09-17T22:43:10.996 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:43:10.996 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 121.597832357, 2024-09-17T22:43:10.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25708877200000002, 2024-09-17T22:43:10.996 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:10.996 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:10.996 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-09-17T22:43:10.996 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:10.996 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4229", 2024-09-17T22:43:10.996 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/3080427038", 2024-09-17T22:43:10.996 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2 2024-09-17T22:43:10.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:10.997 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:10.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:10.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:10.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:43:10.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:10.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:10.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:10.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:10.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394610+0000", 2024-09-17T22:43:10.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:10.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:10.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:10.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:10.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394611+0000", 2024-09-17T22:43:10.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3149999999999999e-06 2024-09-17T22:43:10.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:10.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:10.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:10.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394619+0000", 2024-09-17T22:43:10.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3240000000000003e-06 2024-09-17T22:43:10.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:10.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:10.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:10.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394632+0000", 2024-09-17T22:43:10.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2921e-05 2024-09-17T22:43:10.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:10.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:10.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:10.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394639+0000", 2024-09-17T22:43:10.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-09-17T22:43:10.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:10.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:10.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:10.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394703+0000", 2024-09-17T22:43:10.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4407999999999997e-05 2024-09-17T22:43:10.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:10.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:10.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:43:10.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.394711+0000", 2024-09-17T22:43:10.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5939999999999996e-06 2024-09-17T22:43:10.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651545+0000", 2024-09-17T22:43:11.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.25683409200000001 2024-09-17T22:43:11.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:09.651698+0000", 2024-09-17T22:43:11.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000153678 2024-09-17T22:43:11.000 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.000 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.000 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.001 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38736 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 878592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:11.001 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:43:11.001 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.21523635200001, 2024-09-17T22:43:11.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051395418999999998, 2024-09-17T22:43:11.001 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:11.001 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:11.001 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:11.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:11.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:11.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.001 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38736 2024-09-17T22:43:11.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.001 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:11.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:11.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:43:11.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:11.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:43:11.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:11.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777206+0000", 2024-09-17T22:43:11.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6899999999999998e-07 2024-09-17T22:43:11.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:11.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777212+0000", 2024-09-17T22:43:11.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1240000000000002e-06 2024-09-17T22:43:11.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:11.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777219+0000", 2024-09-17T22:43:11.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5409999999999997e-06 2024-09-17T22:43:11.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777224+0000", 2024-09-17T22:43:11.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:43:11.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.826131+0000", 2024-09-17T22:43:11.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048907369999999999 2024-09-17T22:43:11.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:11.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828457+0000", 2024-09-17T22:43:11.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023256209999999999 2024-09-17T22:43:11.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828601+0000", 2024-09-17T22:43:11.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014438799999999999 2024-09-17T22:43:11.005 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.005 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.005 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.005 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38752 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 673792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:11.005 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:43:11.005 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.21521228100001, 2024-09-17T22:43:11.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051582301999999997, 2024-09-17T22:43:11.005 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:11.006 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:11.006 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:11.006 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:11.006 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:11.006 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.006 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38752 2024-09-17T22:43:11.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.006 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:11.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:11.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:43:11.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:11.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:43:11.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:11.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777230+0000", 2024-09-17T22:43:11.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3000000000000001e-07 2024-09-17T22:43:11.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:11.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777232+0000", 2024-09-17T22:43:11.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9709999999999998e-06 2024-09-17T22:43:11.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:11.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777238+0000", 2024-09-17T22:43:11.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7209999999999999e-06 2024-09-17T22:43:11.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777243+0000", 2024-09-17T22:43:11.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5210000000000004e-06 2024-09-17T22:43:11.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828633+0000", 2024-09-17T22:43:11.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051390915000000002 2024-09-17T22:43:11.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:11.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828711+0000", 2024-09-17T22:43:11.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7474999999999999e-05 2024-09-17T22:43:11.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828812+0000", 2024-09-17T22:43:11.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010116900000000001 2024-09-17T22:43:11.009 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.009 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.010 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.010 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38753 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:11.010 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:43:11.010 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.215193008, 2024-09-17T22:43:11.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051770444999999998, 2024-09-17T22:43:11.010 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:11.010 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:11.010 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:11.010 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:11.010 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:11.010 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.010 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38753 2024-09-17T22:43:11.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.011 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:11.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:11.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:43:11.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:11.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:43:11.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:11.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777249+0000", 2024-09-17T22:43:11.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9900000000000001e-07 2024-09-17T22:43:11.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:11.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777251+0000", 2024-09-17T22:43:11.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8059999999999999e-06 2024-09-17T22:43:11.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:11.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777257+0000", 2024-09-17T22:43:11.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2480000000000001e-06 2024-09-17T22:43:11.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777262+0000", 2024-09-17T22:43:11.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000004e-06 2024-09-17T22:43:11.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828842+0000", 2024-09-17T22:43:11.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051580203999999998 2024-09-17T22:43:11.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:11.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.828917+0000", 2024-09-17T22:43:11.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4980000000000004e-05 2024-09-17T22:43:11.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829019+0000", 2024-09-17T22:43:11.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000102248 2024-09-17T22:43:11.014 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.014 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.014 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.014 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38766 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 806912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:11.014 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:43:11.014 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.215174023, 2024-09-17T22:43:11.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051958839999999999, 2024-09-17T22:43:11.015 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:11.015 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:11.015 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:11.015 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:11.015 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:11.015 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.015 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38766 2024-09-17T22:43:11.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.015 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:11.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:11.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:43:11.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:11.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777268+0000", 2024-09-17T22:43:11.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:11.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777269+0000", 2024-09-17T22:43:11.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-09-17T22:43:11.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:11.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777271+0000", 2024-09-17T22:43:11.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0600000000000002e-06 2024-09-17T22:43:11.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:11.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777276+0000", 2024-09-17T22:43:11.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:43:11.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777281+0000", 2024-09-17T22:43:11.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.006e-06 2024-09-17T22:43:11.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829055+0000", 2024-09-17T22:43:11.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051773570999999997 2024-09-17T22:43:11.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:11.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829128+0000", 2024-09-17T22:43:11.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3131000000000002e-05 2024-09-17T22:43:11.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829227+0000", 2024-09-17T22:43:11.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8800000000000003e-05 2024-09-17T22:43:11.019 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.019 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.019 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.019 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38767 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:11.019 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:43:11.019 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.21515404, 2024-09-17T22:43:11.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137018, 2024-09-17T22:43:11.019 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:11.019 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:11.019 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:11.019 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:11.020 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:11.020 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.020 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38767 2024-09-17T22:43:11.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.020 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:11.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:11.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:43:11.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:11.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:43:11.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:11.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777288+0000", 2024-09-17T22:43:11.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-09-17T22:43:11.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:11.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777290+0000", 2024-09-17T22:43:11.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.88e-06 2024-09-17T22:43:11.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:11.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777296+0000", 2024-09-17T22:43:11.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-09-17T22:43:11.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777300+0000", 2024-09-17T22:43:11.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4510000000000002e-06 2024-09-17T22:43:11.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829257+0000", 2024-09-17T22:43:11.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051956975000000002 2024-09-17T22:43:11.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:11.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829328+0000", 2024-09-17T22:43:11.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1026999999999996e-05 2024-09-17T22:43:11.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829425+0000", 2024-09-17T22:43:11.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.679e-05 2024-09-17T22:43:11.023 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.023 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.023 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.023 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38784 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 675840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:11.023 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:43:11.024 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.215135586, 2024-09-17T22:43:11.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052344522999999997, 2024-09-17T22:43:11.024 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:11.024 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:11.024 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:11.024 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:11.024 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:11.024 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.024 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38784 2024-09-17T22:43:11.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.024 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:11.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:11.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:43:11.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:11.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777306+0000", 2024-09-17T22:43:11.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:11.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777307+0000", 2024-09-17T22:43:11.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:43:11.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:11.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777309+0000", 2024-09-17T22:43:11.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.063e-06 2024-09-17T22:43:11.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:11.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777314+0000", 2024-09-17T22:43:11.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.339e-06 2024-09-17T22:43:11.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777319+0000", 2024-09-17T22:43:11.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999996e-06 2024-09-17T22:43:11.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829456+0000", 2024-09-17T22:43:11.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052137151 2024-09-17T22:43:11.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:11.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829527+0000", 2024-09-17T22:43:11.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1588000000000006e-05 2024-09-17T22:43:11.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829651+0000", 2024-09-17T22:43:11.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012365 2024-09-17T22:43:11.028 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.028 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.028 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.028 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38785 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 749568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:11.028 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:43:11.028 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.215117471, 2024-09-17T22:43:11.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052546572999999999, 2024-09-17T22:43:11.028 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:11.028 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:11.028 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:11.029 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:11.029 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:11.029 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.029 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38785 2024-09-17T22:43:11.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.029 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:11.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:11.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:43:11.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:11.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:43:11.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:11.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777325+0000", 2024-09-17T22:43:11.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1699999999999999e-07 2024-09-17T22:43:11.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:11.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777327+0000", 2024-09-17T22:43:11.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0329999999999998e-06 2024-09-17T22:43:11.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:11.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777333+0000", 2024-09-17T22:43:11.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7309999999999998e-06 2024-09-17T22:43:11.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777337+0000", 2024-09-17T22:43:11.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:43:11.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829686+0000", 2024-09-17T22:43:11.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052349159999999999 2024-09-17T22:43:11.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:11.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829766+0000", 2024-09-17T22:43:11.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9326999999999995e-05 2024-09-17T22:43:11.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829871+0000", 2024-09-17T22:43:11.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000105466 2024-09-17T22:43:11.032 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.032 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.032 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38798 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:43:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.214483863, 2024-09-17T22:43:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052117564999999998, 2024-09-17T22:43:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38798 2024-09-17T22:43:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.033 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:11.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:11.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:43:11.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:11.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777958+0000", 2024-09-17T22:43:11.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:11.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777959+0000", 2024-09-17T22:43:11.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9599999999999999e-07 2024-09-17T22:43:11.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:11.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777963+0000", 2024-09-17T22:43:11.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2660000000000003e-06 2024-09-17T22:43:11.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:11.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777971+0000", 2024-09-17T22:43:11.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3489999999999992e-06 2024-09-17T22:43:11.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777978+0000", 2024-09-17T22:43:11.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7009999999999999e-06 2024-09-17T22:43:11.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829906+0000", 2024-09-17T22:43:11.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051927635999999999 2024-09-17T22:43:11.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:11.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.829986+0000", 2024-09-17T22:43:11.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0444000000000004e-05 2024-09-17T22:43:11.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830076+0000", 2024-09-17T22:43:11.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9473000000000002e-05 2024-09-17T22:43:11.037 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.037 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.037 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.037 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38799 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 882688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:11.037 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:43:11.037 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.214452424, 2024-09-17T22:43:11.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052359606000000003, 2024-09-17T22:43:11.037 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:11.037 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:11.038 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:11.038 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:11.038 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:11.038 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.038 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38799 2024-09-17T22:43:11.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.038 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:11.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:11.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:43:11.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:11.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:43:11.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:11.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777990+0000", 2024-09-17T22:43:11.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8299999999999997e-07 2024-09-17T22:43:11.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:11.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777992+0000", 2024-09-17T22:43:11.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.176e-06 2024-09-17T22:43:11.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:11.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.777998+0000", 2024-09-17T22:43:11.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.173e-06 2024-09-17T22:43:11.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778003+0000", 2024-09-17T22:43:11.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6349999999999997e-06 2024-09-17T22:43:11.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830112+0000", 2024-09-17T22:43:11.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052109366999999997 2024-09-17T22:43:11.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:11.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830192+0000", 2024-09-17T22:43:11.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9814000000000005e-05 2024-09-17T22:43:11.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830349+0000", 2024-09-17T22:43:11.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015695800000000001 2024-09-17T22:43:11.041 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.041 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.041 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.042 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38814 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 104448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:11.042 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:43:11.042 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.21416902, 2024-09-17T22:43:11.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052274339000000003, 2024-09-17T22:43:11.042 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:11.042 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:11.042 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:11.042 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:11.042 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:11.042 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.042 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38814 2024-09-17T22:43:11.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.042 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:11.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:11.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:43:11.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:11.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:43:11.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:11.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778273+0000", 2024-09-17T22:43:11.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-09-17T22:43:11.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:11.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778276+0000", 2024-09-17T22:43:11.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2029999999999999e-06 2024-09-17T22:43:11.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:11.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778282+0000", 2024-09-17T22:43:11.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0059999999999998e-06 2024-09-17T22:43:11.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778287+0000", 2024-09-17T22:43:11.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0039999999999999e-06 2024-09-17T22:43:11.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830382+0000", 2024-09-17T22:43:11.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052095379999999997 2024-09-17T22:43:11.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:11.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830455+0000", 2024-09-17T22:43:11.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3127000000000006e-05 2024-09-17T22:43:11.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830547+0000", 2024-09-17T22:43:11.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2207999999999995e-05 2024-09-17T22:43:11.046 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.046 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.046 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.046 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38828 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 163840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:11.046 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:43:11.046 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.213986612, 2024-09-17T22:43:11.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052466039999999999, 2024-09-17T22:43:11.046 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:11.046 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:11.047 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:11.047 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:11.047 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:11.047 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.047 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38828 2024-09-17T22:43:11.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.047 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:11.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:11.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:43:11.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:11.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778455+0000", 2024-09-17T22:43:11.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:11.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778456+0000", 2024-09-17T22:43:11.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7599999999999996e-07 2024-09-17T22:43:11.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:11.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778460+0000", 2024-09-17T22:43:11.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8229999999999997e-06 2024-09-17T22:43:11.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:11.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778470+0000", 2024-09-17T22:43:11.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8199999999999992e-06 2024-09-17T22:43:11.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778476+0000", 2024-09-17T22:43:11.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2589999999999997e-06 2024-09-17T22:43:11.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830578+0000", 2024-09-17T22:43:11.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052102273999999997 2024-09-17T22:43:11.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:11.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830667+0000", 2024-09-17T22:43:11.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8701000000000003e-05 2024-09-17T22:43:11.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830921+0000", 2024-09-17T22:43:11.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025438700000000002 2024-09-17T22:43:11.050 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.050 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.050 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.050 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38829 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 237568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:11.051 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:43:11.051 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.213955215, 2024-09-17T22:43:11.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052648882000000001, 2024-09-17T22:43:11.051 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:11.051 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:11.051 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:11.051 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:11.051 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:11.051 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.051 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38829 2024-09-17T22:43:11.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.051 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:11.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:11.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:43:11.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:11.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:43:11.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:11.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778487+0000", 2024-09-17T22:43:11.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7e-07 2024-09-17T22:43:11.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:11.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778490+0000", 2024-09-17T22:43:11.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:43:11.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:11.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778495+0000", 2024-09-17T22:43:11.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9329999999999998e-06 2024-09-17T22:43:11.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.778500+0000", 2024-09-17T22:43:11.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9509999999999997e-06 2024-09-17T22:43:11.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.830958+0000", 2024-09-17T22:43:11.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.05245785 2024-09-17T22:43:11.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:11.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831051+0000", 2024-09-17T22:43:11.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2989e-05 2024-09-17T22:43:11.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.831136+0000", 2024-09-17T22:43:11.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4448000000000002e-05 2024-09-17T22:43:11.055 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.055 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.055 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.055 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38891 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:11.055 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:43:11.055 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.21064315700001, 2024-09-17T22:43:11.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050847934999999997, 2024-09-17T22:43:11.055 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:11.055 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:11.055 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:11.055 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:11.056 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:11.056 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.056 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38891 2024-09-17T22:43:11.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.056 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:11.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:11.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:43:11.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:11.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:43:11.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:11.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781799+0000", 2024-09-17T22:43:11.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-09-17T22:43:11.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:11.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781802+0000", 2024-09-17T22:43:11.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9110000000000002e-06 2024-09-17T22:43:11.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:11.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781810+0000", 2024-09-17T22:43:11.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8019999999999992e-06 2024-09-17T22:43:11.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781816+0000", 2024-09-17T22:43:11.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.285e-06 2024-09-17T22:43:11.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832485+0000", 2024-09-17T22:43:11.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050668462999999997 2024-09-17T22:43:11.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:11.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832565+0000", 2024-09-17T22:43:11.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9692999999999997e-05 2024-09-17T22:43:11.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832647+0000", 2024-09-17T22:43:11.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2224000000000004e-05 2024-09-17T22:43:11.059 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.059 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.059 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.059 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38892 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 241664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:11.060 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:43:11.060 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.210614908, 2024-09-17T22:43:11.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050946120999999997, 2024-09-17T22:43:11.060 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:11.060 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:11.060 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:11.060 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:11.060 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:11.060 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.060 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38892 2024-09-17T22:43:11.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.060 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:11.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:11.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:43:11.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:11.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781827+0000", 2024-09-17T22:43:11.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:11.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781828+0000", 2024-09-17T22:43:11.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5400000000000002e-07 2024-09-17T22:43:11.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:11.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781830+0000", 2024-09-17T22:43:11.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1959999999999998e-06 2024-09-17T22:43:11.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:11.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781837+0000", 2024-09-17T22:43:11.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8310000000000003e-06 2024-09-17T22:43:11.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781843+0000", 2024-09-17T22:43:11.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1e-06 2024-09-17T22:43:11.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832674+0000", 2024-09-17T22:43:11.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050831182000000003 2024-09-17T22:43:11.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:11.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832725+0000", 2024-09-17T22:43:11.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1588000000000001e-05 2024-09-17T22:43:11.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832773+0000", 2024-09-17T22:43:11.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7769999999999998e-05 2024-09-17T22:43:11.063 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.063 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.063 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.064 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38909 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 36864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:11.064 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:43:11.064 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.210592138, 2024-09-17T22:43:11.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051046518999999999, 2024-09-17T22:43:11.064 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:11.064 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:11.064 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:11.064 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:11.064 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:11.064 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.064 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38909 2024-09-17T22:43:11.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.065 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:11.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:11.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:43:11.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:11.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:43:11.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:11.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781850+0000", 2024-09-17T22:43:11.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:43:11.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:11.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781852+0000", 2024-09-17T22:43:11.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0590000000000001e-06 2024-09-17T22:43:11.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:11.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781859+0000", 2024-09-17T22:43:11.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3380000000000001e-06 2024-09-17T22:43:11.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781864+0000", 2024-09-17T22:43:11.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2159999999999997e-06 2024-09-17T22:43:11.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832804+0000", 2024-09-17T22:43:11.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050939614000000001 2024-09-17T22:43:11.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:11.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832845+0000", 2024-09-17T22:43:11.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1014999999999999e-05 2024-09-17T22:43:11.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832896+0000", 2024-09-17T22:43:11.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1836999999999999e-05 2024-09-17T22:43:11.068 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.068 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.068 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.068 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38910 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 110592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:11.068 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:43:11.068 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.210571828, 2024-09-17T22:43:11.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051216818999999997, 2024-09-17T22:43:11.068 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:11.068 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:11.068 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:11.069 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:11.069 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:11.069 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.069 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38910 2024-09-17T22:43:11.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.069 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:11.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:11.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:43:11.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:11.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781870+0000", 2024-09-17T22:43:11.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:11.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781871+0000", 2024-09-17T22:43:11.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-09-17T22:43:11.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:11.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781873+0000", 2024-09-17T22:43:11.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8899999999999999e-06 2024-09-17T22:43:11.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:11.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781879+0000", 2024-09-17T22:43:11.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0700000000000003e-06 2024-09-17T22:43:11.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781884+0000", 2024-09-17T22:43:11.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-09-17T22:43:11.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.832932+0000", 2024-09-17T22:43:11.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051048219999999998 2024-09-17T22:43:11.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:11.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833001+0000", 2024-09-17T22:43:11.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8504999999999995e-05 2024-09-17T22:43:11.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833087+0000", 2024-09-17T22:43:11.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6464000000000004e-05 2024-09-17T22:43:11.072 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.072 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.072 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.072 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38923 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 169984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:11.072 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:43:11.073 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.21055150799999, 2024-09-17T22:43:11.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051379573999999997, 2024-09-17T22:43:11.073 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:11.073 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:11.073 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:11.073 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:11.073 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:11.073 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.073 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38923 2024-09-17T22:43:11.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.073 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:11.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:11.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:43:11.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:11.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:43:11.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:11.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781891+0000", 2024-09-17T22:43:11.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:43:11.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:11.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781893+0000", 2024-09-17T22:43:11.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-09-17T22:43:11.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:11.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781900+0000", 2024-09-17T22:43:11.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3860000000000004e-06 2024-09-17T22:43:11.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.781904+0000", 2024-09-17T22:43:11.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9300000000000002e-06 2024-09-17T22:43:11.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833115+0000", 2024-09-17T22:43:11.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.051210845999999997 2024-09-17T22:43:11.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:11.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833188+0000", 2024-09-17T22:43:11.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2276999999999999e-05 2024-09-17T22:43:11.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833270+0000", 2024-09-17T22:43:11.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2501000000000002e-05 2024-09-17T22:43:11.077 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.077 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.077 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.077 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:38941 3.a 3:52b94c5e:::rbd_data.115f1ca54530.0000000000000003:4 [read 38912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31)", 2024-09-17T22:43:11.077 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:43:11.077 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 101.20975897300001, 2024-09-17T22:43:11.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050969459000000002, 2024-09-17T22:43:11.077 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:11.077 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:11.077 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:11.077 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:11.077 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:11.078 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.078 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 38941 2024-09-17T22:43:11.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.078 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:11.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:11.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:43:11.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:11.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782683+0000", 2024-09-17T22:43:11.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:11.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782684+0000", 2024-09-17T22:43:11.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-09-17T22:43:11.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:11.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782686+0000", 2024-09-17T22:43:11.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.396e-06 2024-09-17T22:43:11.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:11.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782693+0000", 2024-09-17T22:43:11.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1999999999999997e-06 2024-09-17T22:43:11.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.782699+0000", 2024-09-17T22:43:11.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7910000000000001e-06 2024-09-17T22:43:11.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833471+0000", 2024-09-17T22:43:11.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050772333000000003 2024-09-17T22:43:11.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:11.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833536+0000", 2024-09-17T22:43:11.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4380000000000004e-05 2024-09-17T22:43:11.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:29.833653+0000", 2024-09-17T22:43:11.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011691799999999999 2024-09-17T22:43:11.081 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.081 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.081 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.081 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:151189 3.14 3:2b696b13:::rbd_data.115f14f2d1d5.0000000000000009:e [read 3127296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.081 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:43:11.082 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.279538228000007, 2024-09-17T22:43:11.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.054254700000000003, 2024-09-17T22:43:11.082 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:11.082 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:11.082 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:11.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:11.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:11.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.082 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 151189 2024-09-17T22:43:11.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.082 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:11.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:11.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:43:11.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:11.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.712904+0000", 2024-09-17T22:43:11.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:11.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:11.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713051+0000", 2024-09-17T22:43:11.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00014722100000000001 2024-09-17T22:43:11.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:11.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713055+0000", 2024-09-17T22:43:11.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.16e-06 2024-09-17T22:43:11.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:11.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713065+0000", 2024-09-17T22:43:11.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8390000000000002e-06 2024-09-17T22:43:11.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.713073+0000", 2024-09-17T22:43:11.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0390000000000005e-06 2024-09-17T22:43:11.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:11.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762644+0000", 2024-09-17T22:43:11.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049571160000000003 2024-09-17T22:43:11.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:11.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.762739+0000", 2024-09-17T22:43:11.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4565e-05 2024-09-17T22:43:11.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:11.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:11.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:11.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.767159+0000", 2024-09-17T22:43:11.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044197159999999997 2024-09-17T22:43:11.085 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.085 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.086 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.086 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:11.086 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:11.086 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:11.087 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:43:11.088 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:11.084+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:11.187 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:11.184+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:11.206 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:11.206 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:11.207 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:43:11.207 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:11.215 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:43:11.299 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:11.296+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:11.360 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:11.360 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:11.360 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:43:11.360 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:43:11.360 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:11.369 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:43:11.411 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:11.408+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:11.489 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:11.487+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:11.514 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:11.515 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:43:11.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:43:11.515 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:43:11.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.515 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158673 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.515 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:43:11.515 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270425036000006, 2024-09-17T22:43:11.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039400614, 2024-09-17T22:43:11.515 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.515 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.515 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.515 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.515 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.516 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.516 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158673 2024-09-17T22:43:11.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.516 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:43:11.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:43:11.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241562+0000", 2024-09-17T22:43:11.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.04e-07 2024-09-17T22:43:11.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241563+0000", 2024-09-17T22:43:11.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5799999999999998e-07 2024-09-17T22:43:11.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.517 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241566+0000", 2024-09-17T22:43:11.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8279999999999999e-06 2024-09-17T22:43:11.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241568+0000", 2024-09-17T22:43:11.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2709999999999999e-06 2024-09-17T22:43:11.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280954+0000", 2024-09-17T22:43:11.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039386051999999998 2024-09-17T22:43:11.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280962+0000", 2024-09-17T22:43:11.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3010000000000007e-06 2024-09-17T22:43:11.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.519 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.519 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.519 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158675 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.519 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:43:11.519 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270415744999994, 2024-09-17T22:43:11.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039408033000000002, 2024-09-17T22:43:11.519 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.519 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.519 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.519 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.519 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.519 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.520 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158675 2024-09-17T22:43:11.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.520 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:43:11.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:43:11.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241571+0000", 2024-09-17T22:43:11.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.36e-07 2024-09-17T22:43:11.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241572+0000", 2024-09-17T22:43:11.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6599999999999994e-07 2024-09-17T22:43:11.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241575+0000", 2024-09-17T22:43:11.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8389999999999998e-06 2024-09-17T22:43:11.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241577+0000", 2024-09-17T22:43:11.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4119999999999999e-06 2024-09-17T22:43:11.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280975+0000", 2024-09-17T22:43:11.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039397205999999997 2024-09-17T22:43:11.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.522 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.522 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280979+0000", 2024-09-17T22:43:11.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.374e-06 2024-09-17T22:43:11.523 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.523 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.523 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.523 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158677 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.523 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:43:11.523 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270393302000002, 2024-09-17T22:43:11.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039409307999999997, 2024-09-17T22:43:11.523 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.523 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.523 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.523 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.524 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.524 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.524 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158677 2024-09-17T22:43:11.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.524 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:43:11.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241593+0000", 2024-09-17T22:43:11.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241594+0000", 2024-09-17T22:43:11.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2800000000000002e-07 2024-09-17T22:43:11.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241595+0000", 2024-09-17T22:43:11.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.2729999999999999e-06 2024-09-17T22:43:11.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241602+0000", 2024-09-17T22:43:11.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:43:11.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241606+0000", 2024-09-17T22:43:11.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3019999999999999e-06 2024-09-17T22:43:11.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.280994+0000", 2024-09-17T22:43:11.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039388484000000001 2024-09-17T22:43:11.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281003+0000", 2024-09-17T22:43:11.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5459999999999992e-06 2024-09-17T22:43:11.527 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.527 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.527 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.527 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158679 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.527 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:43:11.527 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270377624000005, 2024-09-17T22:43:11.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039413403999999999, 2024-09-17T22:43:11.527 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.527 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.527 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.528 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.528 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.528 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.528 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158679 2024-09-17T22:43:11.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.528 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:43:11.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:43:11.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241609+0000", 2024-09-17T22:43:11.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4999999999999999e-07 2024-09-17T22:43:11.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241613+0000", 2024-09-17T22:43:11.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.207e-06 2024-09-17T22:43:11.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241616+0000", 2024-09-17T22:43:11.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.275e-06 2024-09-17T22:43:11.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241618+0000", 2024-09-17T22:43:11.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3910000000000001e-06 2024-09-17T22:43:11.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281017+0000", 2024-09-17T22:43:11.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039398385000000001 2024-09-17T22:43:11.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.531 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.531 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281022+0000", 2024-09-17T22:43:11.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.896e-06 2024-09-17T22:43:11.531 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.531 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.531 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.531 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158681 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.531 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:43:11.531 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270365446, 2024-09-17T22:43:11.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039425292000000001, 2024-09-17T22:43:11.532 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.532 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.532 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.532 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.532 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.532 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.532 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158681 2024-09-17T22:43:11.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.532 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:43:11.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:43:11.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241621+0000", 2024-09-17T22:43:11.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4299999999999999e-07 2024-09-17T22:43:11.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241622+0000", 2024-09-17T22:43:11.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:43:11.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241625+0000", 2024-09-17T22:43:11.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9629999999999998e-06 2024-09-17T22:43:11.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241628+0000", 2024-09-17T22:43:11.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4339999999999999e-06 2024-09-17T22:43:11.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281039+0000", 2024-09-17T22:43:11.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039410672000000001 2024-09-17T22:43:11.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281047+0000", 2024-09-17T22:43:11.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.9969999999999998e-06 2024-09-17T22:43:11.535 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.535 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.535 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.535 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158682 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.536 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:43:11.536 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270356161999999, 2024-09-17T22:43:11.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039436455000000002, 2024-09-17T22:43:11.536 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.536 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.536 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.536 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.536 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.536 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.536 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158682 2024-09-17T22:43:11.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.536 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.536 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:43:11.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:43:11.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241631+0000", 2024-09-17T22:43:11.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:43:11.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241632+0000", 2024-09-17T22:43:11.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:43:11.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241635+0000", 2024-09-17T22:43:11.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8940000000000002e-06 2024-09-17T22:43:11.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241637+0000", 2024-09-17T22:43:11.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3659999999999999e-06 2024-09-17T22:43:11.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281061+0000", 2024-09-17T22:43:11.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039423664999999997 2024-09-17T22:43:11.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281067+0000", 2024-09-17T22:43:11.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.285e-06 2024-09-17T22:43:11.539 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.539 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.539 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.540 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158684 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.540 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:43:11.540 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270346816, 2024-09-17T22:43:11.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462980000000002, 2024-09-17T22:43:11.540 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.540 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.540 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.540 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.540 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.540 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.540 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158684 2024-09-17T22:43:11.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.540 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:43:11.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:43:11.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.541 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241640+0000", 2024-09-17T22:43:11.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:43:11.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241641+0000", 2024-09-17T22:43:11.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:43:11.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241644+0000", 2024-09-17T22:43:11.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8870000000000001e-06 2024-09-17T22:43:11.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241646+0000", 2024-09-17T22:43:11.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3530000000000002e-06 2024-09-17T22:43:11.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281093+0000", 2024-09-17T22:43:11.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039446947000000003 2024-09-17T22:43:11.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281103+0000", 2024-09-17T22:43:11.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6299999999999993e-06 2024-09-17T22:43:11.544 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.544 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.544 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.545 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158686 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.545 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:43:11.545 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270337646000002, 2024-09-17T22:43:11.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039474214000000001, 2024-09-17T22:43:11.545 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.545 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.545 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.545 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158686 2024-09-17T22:43:11.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.546 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:43:11.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:43:11.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241649+0000", 2024-09-17T22:43:11.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:43:11.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241650+0000", 2024-09-17T22:43:11.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2999999999999999e-07 2024-09-17T22:43:11.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241653+0000", 2024-09-17T22:43:11.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9689999999999999e-06 2024-09-17T22:43:11.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241655+0000", 2024-09-17T22:43:11.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3460000000000001e-06 2024-09-17T22:43:11.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281118+0000", 2024-09-17T22:43:11.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039462876000000001 2024-09-17T22:43:11.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281123+0000", 2024-09-17T22:43:11.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8840000000000002e-06 2024-09-17T22:43:11.549 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.549 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.549 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.549 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158688 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.549 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:43:11.549 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270328414999994, 2024-09-17T22:43:11.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039490458999999999, 2024-09-17T22:43:11.550 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.550 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.550 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.550 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158688 2024-09-17T22:43:11.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.550 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.550 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:43:11.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:43:11.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241658+0000", 2024-09-17T22:43:11.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:43:11.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241659+0000", 2024-09-17T22:43:11.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3699999999999999e-07 2024-09-17T22:43:11.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241662+0000", 2024-09-17T22:43:11.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8499999999999998e-06 2024-09-17T22:43:11.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241665+0000", 2024-09-17T22:43:11.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.356e-06 2024-09-17T22:43:11.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281141+0000", 2024-09-17T22:43:11.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039476031000000002 2024-09-17T22:43:11.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281149+0000", 2024-09-17T22:43:11.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0709999999999995e-06 2024-09-17T22:43:11.553 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.553 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.553 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.554 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158690 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.554 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:43:11.554 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270319356000002, 2024-09-17T22:43:11.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039502378999999997, 2024-09-17T22:43:11.554 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.554 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.554 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.554 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.554 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.554 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.554 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158690 2024-09-17T22:43:11.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.554 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:43:11.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241667+0000", 2024-09-17T22:43:11.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241668+0000", 2024-09-17T22:43:11.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:43:11.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241669+0000", 2024-09-17T22:43:11.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1009999999999999e-06 2024-09-17T22:43:11.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241672+0000", 2024-09-17T22:43:11.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9059999999999998e-06 2024-09-17T22:43:11.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241674+0000", 2024-09-17T22:43:11.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4439999999999998e-06 2024-09-17T22:43:11.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281162+0000", 2024-09-17T22:43:11.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039488189 2024-09-17T22:43:11.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281170+0000", 2024-09-17T22:43:11.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5310000000000004e-06 2024-09-17T22:43:11.557 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.557 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.557 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.558 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158692 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.558 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:43:11.558 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270309922999999, 2024-09-17T22:43:11.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039531064999999997, 2024-09-17T22:43:11.558 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.558 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.558 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.558 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.558 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.558 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.558 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158692 2024-09-17T22:43:11.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.559 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:43:11.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:43:11.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241677+0000", 2024-09-17T22:43:11.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:43:11.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241678+0000", 2024-09-17T22:43:11.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0800000000000003e-07 2024-09-17T22:43:11.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241681+0000", 2024-09-17T22:43:11.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8729999999999999e-06 2024-09-17T22:43:11.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241683+0000", 2024-09-17T22:43:11.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.447e-06 2024-09-17T22:43:11.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281199+0000", 2024-09-17T22:43:11.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039515388999999998 2024-09-17T22:43:11.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281208+0000", 2024-09-17T22:43:11.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2399999999999996e-06 2024-09-17T22:43:11.561 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.562 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.562 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.562 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158694 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.562 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:43:11.562 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270300751999997, 2024-09-17T22:43:11.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039543986000000003, 2024-09-17T22:43:11.562 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.562 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.562 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.562 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.562 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.563 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.563 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158694 2024-09-17T22:43:11.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.563 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:43:11.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:43:11.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241686+0000", 2024-09-17T22:43:11.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0900000000000001e-07 2024-09-17T22:43:11.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241687+0000", 2024-09-17T22:43:11.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.035e-06 2024-09-17T22:43:11.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241691+0000", 2024-09-17T22:43:11.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.811e-06 2024-09-17T22:43:11.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241693+0000", 2024-09-17T22:43:11.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.385e-06 2024-09-17T22:43:11.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281223+0000", 2024-09-17T22:43:11.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039529332 2024-09-17T22:43:11.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281230+0000", 2024-09-17T22:43:11.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2139999999999999e-06 2024-09-17T22:43:11.566 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.566 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.566 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.566 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158696 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.566 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:43:11.566 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270290364000005, 2024-09-17T22:43:11.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039570458000000003, 2024-09-17T22:43:11.566 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.566 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.566 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.566 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.566 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.566 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.567 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158696 2024-09-17T22:43:11.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.567 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:43:11.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241696+0000", 2024-09-17T22:43:11.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:43:11.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:43:11.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241697+0000", 2024-09-17T22:43:11.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.33e-07 2024-09-17T22:43:11.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241700+0000", 2024-09-17T22:43:11.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8949999999999998e-06 2024-09-17T22:43:11.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241703+0000", 2024-09-17T22:43:11.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3159999999999999e-06 2024-09-17T22:43:11.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281257+0000", 2024-09-17T22:43:11.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039553862000000002 2024-09-17T22:43:11.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281267+0000", 2024-09-17T22:43:11.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.024e-05 2024-09-17T22:43:11.569 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.570 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.570 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.570 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158698 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.570 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:43:11.570 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270280286000002, 2024-09-17T22:43:11.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039576493999999997, 2024-09-17T22:43:11.570 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.570 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.570 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.570 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.571 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.571 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.571 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158698 2024-09-17T22:43:11.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.571 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:43:11.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241706+0000", 2024-09-17T22:43:11.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241707+0000", 2024-09-17T22:43:11.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1199999999999999e-07 2024-09-17T22:43:11.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241708+0000", 2024-09-17T22:43:11.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.5000000000000001e-07 2024-09-17T22:43:11.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241711+0000", 2024-09-17T22:43:11.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9670000000000001e-06 2024-09-17T22:43:11.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241713+0000", 2024-09-17T22:43:11.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2689999999999998e-06 2024-09-17T22:43:11.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281278+0000", 2024-09-17T22:43:11.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039565206999999998 2024-09-17T22:43:11.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281283+0000", 2024-09-17T22:43:11.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8890000000000001e-06 2024-09-17T22:43:11.574 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.574 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.574 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.574 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158700 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.574 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:43:11.574 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270271074999997, 2024-09-17T22:43:11.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039604292999999999, 2024-09-17T22:43:11.574 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.574 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.574 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.575 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.575 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.575 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.575 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158700 2024-09-17T22:43:11.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.575 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:43:11.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:43:11.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241716+0000", 2024-09-17T22:43:11.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0800000000000001e-07 2024-09-17T22:43:11.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241717+0000", 2024-09-17T22:43:11.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.78e-07 2024-09-17T22:43:11.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241720+0000", 2024-09-17T22:43:11.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.8119999999999999e-06 2024-09-17T22:43:11.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241722+0000", 2024-09-17T22:43:11.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.401e-06 2024-09-17T22:43:11.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281311+0000", 2024-09-17T22:43:11.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039588894999999999 2024-09-17T22:43:11.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281320+0000", 2024-09-17T22:43:11.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9989999999999997e-06 2024-09-17T22:43:11.578 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.578 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.578 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.578 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158702 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.578 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:43:11.578 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270259753000005, 2024-09-17T22:43:11.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039616142, 2024-09-17T22:43:11.578 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.578 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.578 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.578 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.579 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.579 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.579 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158702 2024-09-17T22:43:11.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.579 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:43:11.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:43:11.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241727+0000", 2024-09-17T22:43:11.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4400000000000001e-07 2024-09-17T22:43:11.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241728+0000", 2024-09-17T22:43:11.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0839999999999999e-06 2024-09-17T22:43:11.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241731+0000", 2024-09-17T22:43:11.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0690000000000002e-06 2024-09-17T22:43:11.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241734+0000", 2024-09-17T22:43:11.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4159999999999998e-06 2024-09-17T22:43:11.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281334+0000", 2024-09-17T22:43:11.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039600110000000001 2024-09-17T22:43:11.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281343+0000", 2024-09-17T22:43:11.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2189999999999993e-06 2024-09-17T22:43:11.582 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.582 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.582 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.582 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158704 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.582 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:43:11.582 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270249888999999, 2024-09-17T22:43:11.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039641485999999997, 2024-09-17T22:43:11.582 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.583 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.583 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.583 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.583 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158704 2024-09-17T22:43:11.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.583 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:43:11.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:43:11.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241737+0000", 2024-09-17T22:43:11.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1e-07 2024-09-17T22:43:11.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241738+0000", 2024-09-17T22:43:11.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0589999999999999e-06 2024-09-17T22:43:11.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241741+0000", 2024-09-17T22:43:11.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.943e-06 2024-09-17T22:43:11.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241744+0000", 2024-09-17T22:43:11.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4819999999999999e-06 2024-09-17T22:43:11.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281370+0000", 2024-09-17T22:43:11.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039625661 2024-09-17T22:43:11.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281378+0000", 2024-09-17T22:43:11.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1310000000000006e-06 2024-09-17T22:43:11.586 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.586 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.586 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.586 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158705 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.586 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:43:11.586 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270239339, 2024-09-17T22:43:11.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039652637999999997, 2024-09-17T22:43:11.586 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.586 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.586 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.587 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.587 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.587 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.587 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158705 2024-09-17T22:43:11.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.587 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:43:11.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241747+0000", 2024-09-17T22:43:11.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:43:11.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3300000000000001e-07 2024-09-17T22:43:11.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241748+0000", 2024-09-17T22:43:11.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0599999999999999e-07 2024-09-17T22:43:11.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241751+0000", 2024-09-17T22:43:11.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.926e-06 2024-09-17T22:43:11.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241754+0000", 2024-09-17T22:43:11.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4360000000000001e-06 2024-09-17T22:43:11.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281392+0000", 2024-09-17T22:43:11.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039638566 2024-09-17T22:43:11.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281400+0000", 2024-09-17T22:43:11.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.571e-06 2024-09-17T22:43:11.590 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.590 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.590 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.590 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158707 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.591 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:43:11.591 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270218237999998, 2024-09-17T22:43:11.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039654809999999999, 2024-09-17T22:43:11.591 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.591 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.591 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.591 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.591 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.591 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.591 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158707 2024-09-17T22:43:11.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.591 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:43:11.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241768+0000", 2024-09-17T22:43:11.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241769+0000", 2024-09-17T22:43:11.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4299999999999999e-07 2024-09-17T22:43:11.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241770+0000", 2024-09-17T22:43:11.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3769999999999999e-06 2024-09-17T22:43:11.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241773+0000", 2024-09-17T22:43:11.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2109999999999998e-06 2024-09-17T22:43:11.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241776+0000", 2024-09-17T22:43:11.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5069999999999999e-06 2024-09-17T22:43:11.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281415+0000", 2024-09-17T22:43:11.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039639499000000002 2024-09-17T22:43:11.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281423+0000", 2024-09-17T22:43:11.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8730000000000007e-06 2024-09-17T22:43:11.594 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.595 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.595 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.595 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:158709 3.1b 3.f135dd7b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:11.595 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:43:11.595 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.270207628999998, 2024-09-17T22:43:11.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.039661937000000001, 2024-09-17T22:43:11.595 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:11.595 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:11.595 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "reached pg", 2024-09-17T22:43:11.595 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:11.595 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:11.596 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:11.596 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 158709 2024-09-17T22:43:11.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.596 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:11.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:11.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:43:11.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:11.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.241779+0000", 2024-09-17T22:43:11.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:11.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:11.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242843+0000", 2024-09-17T22:43:11.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0010634780000000001 2024-09-17T22:43:11.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:11.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242845+0000", 2024-09-17T22:43:11.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7209999999999999e-06 2024-09-17T22:43:11.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:11.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242851+0000", 2024-09-17T22:43:11.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6949999999999997e-06 2024-09-17T22:43:11.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:11.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.242855+0000", 2024-09-17T22:43:11.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7960000000000002e-06 2024-09-17T22:43:11.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:11.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281434+0000", 2024-09-17T22:43:11.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.038579519 2024-09-17T22:43:11.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:11.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:11.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:11.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.281441+0000", 2024-09-17T22:43:11.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7279999999999998e-06 2024-09-17T22:43:11.599 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.599 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.599 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.599 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:11.599 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:11.599 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:11.599 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:43:11.600 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:11.598+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:11.702 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:43:11.699+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:43:11.704 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:11.704 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:11.704 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:43:11.704 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:11.713 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:43:11.803 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:11.801+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:11.856 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:11.857 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:11.857 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:43:11.857 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:43:11.857 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:11.865 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:43:11.906 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:11.904+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:12.005 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:12.003+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:12.010 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:12.010 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:43:12.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:43:12.010 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:43:12.010 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.010 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157229 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4073472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.010 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:43:12.010 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.860866959999996, 2024-09-17T22:43:12.010 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041957959000000003, 2024-09-17T22:43:12.010 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.011 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.011 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.011 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.011 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.011 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.011 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157229 2024-09-17T22:43:12.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.011 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.011 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:43:12.011 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.011 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146206+0000", 2024-09-17T22:43:12.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146207+0000", 2024-09-17T22:43:12.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0299999999999999e-07 2024-09-17T22:43:12.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.012 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.012 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.012 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146215+0000", 2024-09-17T22:43:12.012 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3059999999999998e-06 2024-09-17T22:43:12.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146222+0000", 2024-09-17T22:43:12.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1180000000000003e-06 2024-09-17T22:43:12.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146227+0000", 2024-09-17T22:43:12.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1749999999999998e-06 2024-09-17T22:43:12.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.013 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.013 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.013 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.187985+0000", 2024-09-17T22:43:12.013 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041758391999999998 2024-09-17T22:43:12.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188065+0000", 2024-09-17T22:43:12.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0290000000000005e-05 2024-09-17T22:43:12.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.014 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.014 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188164+0000", 2024-09-17T22:43:12.014 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9074999999999996e-05 2024-09-17T22:43:12.014 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.014 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.014 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.014 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.015 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157231 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4079616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.015 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:43:12.015 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.860729336000006, 2024-09-17T22:43:12.015 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042034581000000001, 2024-09-17T22:43:12.015 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.015 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.015 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.015 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.015 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157231 2024-09-17T22:43:12.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.015 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:43:12.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146344+0000", 2024-09-17T22:43:12.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.016 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.016 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.016 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146345+0000", 2024-09-17T22:43:12.016 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5599999999999995e-07 2024-09-17T22:43:12.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146347+0000", 2024-09-17T22:43:12.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.357e-06 2024-09-17T22:43:12.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146352+0000", 2024-09-17T22:43:12.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4070000000000001e-06 2024-09-17T22:43:12.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.017 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.017 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.017 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146361+0000", 2024-09-17T22:43:12.017 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0219999999999993e-06 2024-09-17T22:43:12.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188199+0000", 2024-09-17T22:43:12.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041838130000000001 2024-09-17T22:43:12.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188285+0000", 2024-09-17T22:43:12.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.5160000000000005e-05 2024-09-17T22:43:12.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.018 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.018 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.018 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188379+0000", 2024-09-17T22:43:12.018 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.3949000000000005e-05 2024-09-17T22:43:12.018 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.019 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.019 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.019 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.019 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157233 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4085760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.019 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:43:12.019 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.860513186999995, 2024-09-17T22:43:12.019 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042008960999999997, 2024-09-17T22:43:12.019 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.019 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.019 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.019 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.020 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157233 2024-09-17T22:43:12.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.020 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:43:12.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.020 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146560+0000", 2024-09-17T22:43:12.020 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.020 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146561+0000", 2024-09-17T22:43:12.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6300000000000005e-07 2024-09-17T22:43:12.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146564+0000", 2024-09-17T22:43:12.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9330000000000001e-06 2024-09-17T22:43:12.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.021 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146575+0000", 2024-09-17T22:43:12.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0891e-05 2024-09-17T22:43:12.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.021 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146579+0000", 2024-09-17T22:43:12.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.284e-06 2024-09-17T22:43:12.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188407+0000", 2024-09-17T22:43:12.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041828390999999999 2024-09-17T22:43:12.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188476+0000", 2024-09-17T22:43:12.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9129999999999997e-05 2024-09-17T22:43:12.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188569+0000", 2024-09-17T22:43:12.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2669000000000003e-05 2024-09-17T22:43:12.023 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.023 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.023 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.023 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157235 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4091904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.023 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:43:12.023 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.860319189999998, 2024-09-17T22:43:12.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042047517, 2024-09-17T22:43:12.023 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.023 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.023 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.024 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.024 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157235 2024-09-17T22:43:12.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.024 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:43:12.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146754+0000", 2024-09-17T22:43:12.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146755+0000", 2024-09-17T22:43:12.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0900000000000001e-07 2024-09-17T22:43:12.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146758+0000", 2024-09-17T22:43:12.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.399e-06 2024-09-17T22:43:12.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146768+0000", 2024-09-17T22:43:12.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4730000000000006e-06 2024-09-17T22:43:12.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146774+0000", 2024-09-17T22:43:12.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6939999999999998e-06 2024-09-17T22:43:12.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188616+0000", 2024-09-17T22:43:12.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041841481999999999 2024-09-17T22:43:12.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.026 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.026 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188700+0000", 2024-09-17T22:43:12.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4227999999999994e-05 2024-09-17T22:43:12.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188802+0000", 2024-09-17T22:43:12.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000101532 2024-09-17T22:43:12.027 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.027 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.027 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.027 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157237 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4098048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.027 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:43:12.027 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.860288918999998, 2024-09-17T22:43:12.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165452999999999, 2024-09-17T22:43:12.028 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.028 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.028 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.028 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.028 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157237 2024-09-17T22:43:12.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.028 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:43:12.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146784+0000", 2024-09-17T22:43:12.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146785+0000", 2024-09-17T22:43:12.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6800000000000001e-07 2024-09-17T22:43:12.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146787+0000", 2024-09-17T22:43:12.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.046e-06 2024-09-17T22:43:12.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146794+0000", 2024-09-17T22:43:12.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9249999999999998e-06 2024-09-17T22:43:12.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146799+0000", 2024-09-17T22:43:12.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9289999999999997e-06 2024-09-17T22:43:12.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188844+0000", 2024-09-17T22:43:12.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042045386999999997 2024-09-17T22:43:12.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188888+0000", 2024-09-17T22:43:12.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3760000000000001e-05 2024-09-17T22:43:12.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.031 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.031 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188950+0000", 2024-09-17T22:43:12.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1938000000000002e-05 2024-09-17T22:43:12.031 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.031 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.031 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.031 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157239 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4104192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.032 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:43:12.032 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.860260533000002, 2024-09-17T22:43:12.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042347193999999998, 2024-09-17T22:43:12.032 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.032 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.032 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.032 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.032 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.032 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.032 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157239 2024-09-17T22:43:12.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.032 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:43:12.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:43:12.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146813+0000", 2024-09-17T22:43:12.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:43:12.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146816+0000", 2024-09-17T22:43:12.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4700000000000001e-06 2024-09-17T22:43:12.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146822+0000", 2024-09-17T22:43:12.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.579e-06 2024-09-17T22:43:12.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146826+0000", 2024-09-17T22:43:12.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3780000000000001e-06 2024-09-17T22:43:12.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.188991+0000", 2024-09-17T22:43:12.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042165553000000001 2024-09-17T22:43:12.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189068+0000", 2024-09-17T22:43:12.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6548e-05 2024-09-17T22:43:12.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189160+0000", 2024-09-17T22:43:12.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.2163000000000005e-05 2024-09-17T22:43:12.035 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.035 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.035 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.036 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157242 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4110336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.036 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:43:12.036 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.860222746999995, 2024-09-17T22:43:12.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042499783999999999, 2024-09-17T22:43:12.036 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.036 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.036 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.036 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.036 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.036 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.036 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157242 2024-09-17T22:43:12.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.036 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:43:12.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:43:12.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146851+0000", 2024-09-17T22:43:12.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:43:12.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146852+0000", 2024-09-17T22:43:12.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.37e-06 2024-09-17T22:43:12.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146856+0000", 2024-09-17T22:43:12.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1690000000000002e-06 2024-09-17T22:43:12.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146863+0000", 2024-09-17T22:43:12.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9920000000000001e-06 2024-09-17T22:43:12.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189193+0000", 2024-09-17T22:43:12.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042329355999999999 2024-09-17T22:43:12.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189246+0000", 2024-09-17T22:43:12.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3643000000000001e-05 2024-09-17T22:43:12.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.039 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.039 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.039 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189350+0000", 2024-09-17T22:43:12.039 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00010394 2024-09-17T22:43:12.039 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.039 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.039 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.040 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157244 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4116480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.040 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:43:12.040 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.860203536, 2024-09-17T22:43:12.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042675684999999998, 2024-09-17T22:43:12.040 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.040 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.040 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.040 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.040 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.040 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.040 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157244 2024-09-17T22:43:12.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.041 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:43:12.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:43:12.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146870+0000", 2024-09-17T22:43:12.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1e-07 2024-09-17T22:43:12.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146871+0000", 2024-09-17T22:43:12.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.189e-06 2024-09-17T22:43:12.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.042 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146877+0000", 2024-09-17T22:43:12.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8100000000000003e-06 2024-09-17T22:43:12.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.042 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146880+0000", 2024-09-17T22:43:12.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.83e-06 2024-09-17T22:43:12.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189388+0000", 2024-09-17T22:43:12.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042507637000000001 2024-09-17T22:43:12.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189443+0000", 2024-09-17T22:43:12.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5742999999999998e-05 2024-09-17T22:43:12.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189545+0000", 2024-09-17T22:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102166 2024-09-17T22:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157246 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4122624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.860174483999998, 2024-09-17T22:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042847083000000001, 2024-09-17T22:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.044 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157246 2024-09-17T22:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:43:12.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146899+0000", 2024-09-17T22:43:12.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.0100000000000001e-07 2024-09-17T22:43:12.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146901+0000", 2024-09-17T22:43:12.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.531e-06 2024-09-17T22:43:12.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146905+0000", 2024-09-17T22:43:12.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0759999999999996e-06 2024-09-17T22:43:12.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146908+0000", 2024-09-17T22:43:12.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.083e-06 2024-09-17T22:43:12.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189585+0000", 2024-09-17T22:43:12.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042677009000000002 2024-09-17T22:43:12.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.047 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.047 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189656+0000", 2024-09-17T22:43:12.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0712999999999999e-05 2024-09-17T22:43:12.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189746+0000", 2024-09-17T22:43:12.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.0370000000000004e-05 2024-09-17T22:43:12.048 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.048 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.048 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.048 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157247 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4128768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.048 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:43:12.048 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.860102647999994, 2024-09-17T22:43:12.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043016916000000002, 2024-09-17T22:43:12.048 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.049 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.049 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.049 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.049 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.049 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.049 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157247 2024-09-17T22:43:12.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.049 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:43:12.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.146971+0000", 2024-09-17T22:43:12.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147071+0000", 2024-09-17T22:43:12.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100769 2024-09-17T22:43:12.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147081+0000", 2024-09-17T22:43:12.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.9229999999999997e-06 2024-09-17T22:43:12.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147090+0000", 2024-09-17T22:43:12.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.935e-06 2024-09-17T22:43:12.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147097+0000", 2024-09-17T22:43:12.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5699999999999998e-06 2024-09-17T22:43:12.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189787+0000", 2024-09-17T22:43:12.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042690525 2024-09-17T22:43:12.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189884+0000", 2024-09-17T22:43:12.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6858e-05 2024-09-17T22:43:12.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.189988+0000", 2024-09-17T22:43:12.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000103336 2024-09-17T22:43:12.052 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.052 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.052 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.053 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157250 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4134912~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.053 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:43:12.053 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.859959187000001, 2024-09-17T22:43:12.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043099446, 2024-09-17T22:43:12.053 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.053 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.053 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.053 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.053 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.053 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.054 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157250 2024-09-17T22:43:12.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.054 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:43:12.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147114+0000", 2024-09-17T22:43:12.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147235+0000", 2024-09-17T22:43:12.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000120975 2024-09-17T22:43:12.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147238+0000", 2024-09-17T22:43:12.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5289999999999998e-06 2024-09-17T22:43:12.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147244+0000", 2024-09-17T22:43:12.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6050000000000003e-06 2024-09-17T22:43:12.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147249+0000", 2024-09-17T22:43:12.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4939999999999997e-06 2024-09-17T22:43:12.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.056 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.056 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190020+0000", 2024-09-17T22:43:12.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042771300999999998 2024-09-17T22:43:12.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190102+0000", 2024-09-17T22:43:12.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1600000000000005e-05 2024-09-17T22:43:12.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190214+0000", 2024-09-17T22:43:12.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00011194200000000001 2024-09-17T22:43:12.057 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.057 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.057 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.057 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157252 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4141056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.057 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:43:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.859816518000002, 2024-09-17T22:43:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043165508999999998, 2024-09-17T22:43:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157252 2024-09-17T22:43:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:43:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147257+0000", 2024-09-17T22:43:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147402+0000", 2024-09-17T22:43:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014475600000000001 2024-09-17T22:43:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147411+0000", 2024-09-17T22:43:12.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4350000000000003e-06 2024-09-17T22:43:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147420+0000", 2024-09-17T22:43:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4790000000000002e-06 2024-09-17T22:43:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147427+0000", 2024-09-17T22:43:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5400000000000001e-06 2024-09-17T22:43:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190254+0000", 2024-09-17T22:43:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042826731 2024-09-17T22:43:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190326+0000", 2024-09-17T22:43:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1798999999999995e-05 2024-09-17T22:43:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190422+0000", 2024-09-17T22:43:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6768999999999995e-05 2024-09-17T22:43:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.062 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157253 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4147200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.062 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:43:12.062 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.859630151999994, 2024-09-17T22:43:12.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043157752000000001, 2024-09-17T22:43:12.062 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.062 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.062 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.062 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.062 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157253 2024-09-17T22:43:12.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.062 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:43:12.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147443+0000", 2024-09-17T22:43:12.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147590+0000", 2024-09-17T22:43:12.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00014709 2024-09-17T22:43:12.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147593+0000", 2024-09-17T22:43:12.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.5840000000000002e-06 2024-09-17T22:43:12.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147598+0000", 2024-09-17T22:43:12.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2469999999999999e-06 2024-09-17T22:43:12.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147602+0000", 2024-09-17T22:43:12.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0049999999999998e-06 2024-09-17T22:43:12.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190475+0000", 2024-09-17T22:43:12.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872767999999999 2024-09-17T22:43:12.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190528+0000", 2024-09-17T22:43:12.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2889999999999997e-05 2024-09-17T22:43:12.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190601+0000", 2024-09-17T22:43:12.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3168000000000001e-05 2024-09-17T22:43:12.066 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.066 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.066 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.066 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157255 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4153344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.066 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:43:12.066 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.859464170999999, 2024-09-17T22:43:12.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043132251000000003, 2024-09-17T22:43:12.066 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.066 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.066 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.066 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.067 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.067 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157255 2024-09-17T22:43:12.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.067 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:43:12.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147609+0000", 2024-09-17T22:43:12.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147820+0000", 2024-09-17T22:43:12.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00021123600000000001 2024-09-17T22:43:12.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147824+0000", 2024-09-17T22:43:12.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6780000000000002e-06 2024-09-17T22:43:12.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147833+0000", 2024-09-17T22:43:12.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4250000000000004e-06 2024-09-17T22:43:12.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147842+0000", 2024-09-17T22:43:12.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0679999999999997e-06 2024-09-17T22:43:12.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190632+0000", 2024-09-17T22:43:12.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042790863999999998 2024-09-17T22:43:12.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190674+0000", 2024-09-17T22:43:12.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1804000000000002e-05 2024-09-17T22:43:12.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.070 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.070 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190741+0000", 2024-09-17T22:43:12.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7175999999999995e-05 2024-09-17T22:43:12.070 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.070 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.070 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.070 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157257 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4159488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.070 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:43:12.070 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.859223681000003, 2024-09-17T22:43:12.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.04307035, 2024-09-17T22:43:12.071 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.071 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.071 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.071 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.071 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.071 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.071 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157257 2024-09-17T22:43:12.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.071 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:43:12.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.147850+0000", 2024-09-17T22:43:12.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148844+0000", 2024-09-17T22:43:12.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00099455599999999996 2024-09-17T22:43:12.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148848+0000", 2024-09-17T22:43:12.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0180000000000003e-06 2024-09-17T22:43:12.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148858+0000", 2024-09-17T22:43:12.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.825e-06 2024-09-17T22:43:12.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148865+0000", 2024-09-17T22:43:12.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8290000000000002e-06 2024-09-17T22:43:12.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190789+0000", 2024-09-17T22:43:12.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.041924630999999997 2024-09-17T22:43:12.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190860+0000", 2024-09-17T22:43:12.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0557999999999998e-05 2024-09-17T22:43:12.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190920+0000", 2024-09-17T22:43:12.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9932999999999997e-05 2024-09-17T22:43:12.074 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.074 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.074 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.075 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157259 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4165632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.075 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:43:12.075 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.858197661999995, 2024-09-17T22:43:12.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042261411999999998, 2024-09-17T22:43:12.075 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.075 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.075 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.075 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.075 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.075 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.075 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157259 2024-09-17T22:43:12.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.075 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:43:12.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:43:12.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148876+0000", 2024-09-17T22:43:12.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3500000000000002e-07 2024-09-17T22:43:12.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148878+0000", 2024-09-17T22:43:12.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1320000000000001e-06 2024-09-17T22:43:12.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148885+0000", 2024-09-17T22:43:12.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7290000000000003e-06 2024-09-17T22:43:12.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148893+0000", 2024-09-17T22:43:12.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8110000000000003e-06 2024-09-17T22:43:12.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.190967+0000", 2024-09-17T22:43:12.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042073948999999999 2024-09-17T22:43:12.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191069+0000", 2024-09-17T22:43:12.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000102075 2024-09-17T22:43:12.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191137+0000", 2024-09-17T22:43:12.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8281000000000006e-05 2024-09-17T22:43:12.078 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.078 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.079 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.079 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157261 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4171776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.079 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:43:12.079 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.858148055000001, 2024-09-17T22:43:12.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042437255, 2024-09-17T22:43:12.079 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.079 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.079 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.079 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.079 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.079 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.079 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157261 2024-09-17T22:43:12.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.080 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:43:12.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148925+0000", 2024-09-17T22:43:12.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148926+0000", 2024-09-17T22:43:12.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5999999999999999e-07 2024-09-17T22:43:12.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148928+0000", 2024-09-17T22:43:12.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.4329999999999998e-06 2024-09-17T22:43:12.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148936+0000", 2024-09-17T22:43:12.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6969999999999993e-06 2024-09-17T22:43:12.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148942+0000", 2024-09-17T22:43:12.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9479999999999996e-06 2024-09-17T22:43:12.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191185+0000", 2024-09-17T22:43:12.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042243147000000002 2024-09-17T22:43:12.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191266+0000", 2024-09-17T22:43:12.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1186000000000005e-05 2024-09-17T22:43:12.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191362+0000", 2024-09-17T22:43:12.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6384000000000004e-05 2024-09-17T22:43:12.083 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.083 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.083 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.083 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157263 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4177920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.083 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:43:12.083 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.858123656000004, 2024-09-17T22:43:12.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042636395000000001, 2024-09-17T22:43:12.083 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.083 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.083 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.083 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.083 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.083 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.084 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157263 2024-09-17T22:43:12.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.084 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:43:12.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.084 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:43:12.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.084 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.084 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148950+0000", 2024-09-17T22:43:12.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.37e-07 2024-09-17T22:43:12.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148952+0000", 2024-09-17T22:43:12.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.88e-06 2024-09-17T22:43:12.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148959+0000", 2024-09-17T22:43:12.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5649999999999998e-06 2024-09-17T22:43:12.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148963+0000", 2024-09-17T22:43:12.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8570000000000003e-06 2024-09-17T22:43:12.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191408+0000", 2024-09-17T22:43:12.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042444707999999998 2024-09-17T22:43:12.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191489+0000", 2024-09-17T22:43:12.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0612999999999995e-05 2024-09-17T22:43:12.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191586+0000", 2024-09-17T22:43:12.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7235e-05 2024-09-17T22:43:12.087 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.087 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.087 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.087 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157265 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4184064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.087 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:43:12.087 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.858102106999993, 2024-09-17T22:43:12.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042791830000000003, 2024-09-17T22:43:12.087 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.087 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.087 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.087 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.087 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.088 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.088 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157265 2024-09-17T22:43:12.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.088 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:43:12.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148971+0000", 2024-09-17T22:43:12.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148972+0000", 2024-09-17T22:43:12.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:43:12.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148973+0000", 2024-09-17T22:43:12.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.894e-06 2024-09-17T22:43:12.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148985+0000", 2024-09-17T22:43:12.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1995e-05 2024-09-17T22:43:12.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148990+0000", 2024-09-17T22:43:12.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8269999999999997e-06 2024-09-17T22:43:12.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191630+0000", 2024-09-17T22:43:12.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042639639 2024-09-17T22:43:12.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191686+0000", 2024-09-17T22:43:12.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6066000000000002e-05 2024-09-17T22:43:12.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191763+0000", 2024-09-17T22:43:12.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7035999999999999e-05 2024-09-17T22:43:12.091 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.091 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.091 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.091 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:157267 3.1f 3:fc904920:::rbd_data.115f14f2d1d5.000000000000000b:e [read 4190208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:12.091 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:43:12.091 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 81.858076294, 2024-09-17T22:43:12.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.043028813999999999, 2024-09-17T22:43:12.091 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.091 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.091 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.091 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.091 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.092 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.092 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 157267 2024-09-17T22:43:12.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.092 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:43:12.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:43:12.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148997+0000", 2024-09-17T22:43:12.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7599999999999998e-07 2024-09-17T22:43:12.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.148999+0000", 2024-09-17T22:43:12.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.432e-06 2024-09-17T22:43:12.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.093 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.093 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149002+0000", 2024-09-17T22:43:12.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7850000000000002e-06 2024-09-17T22:43:12.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.149005+0000", 2024-09-17T22:43:12.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:43:12.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191878+0000", 2024-09-17T22:43:12.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.042872312000000003 2024-09-17T22:43:12.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.191961+0000", 2024-09-17T22:43:12.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3418000000000007e-05 2024-09-17T22:43:12.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:41:50.192026+0000", 2024-09-17T22:43:12.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4689000000000004e-05 2024-09-17T22:43:12.095 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.095 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.095 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.095 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.095 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.095 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:12.095 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:43:12.096 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:43:12.137 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:12.135+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:12.207 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:12.205+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:12.245 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:12.245 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:12.245 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:43:12.245 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:12.254 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:43:12.285 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:43:12.285 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:43:12.285 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T22:43:12.286 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T22:43:12.329 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:12.327+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:12.398 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:12.399 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:12.399 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:43:12.399 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:43:12.399 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:12.408 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:43:12.409 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:12.406+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:12.510 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:12.507+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:12.512 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:12.512 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:43:12.512 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:43:12.512 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:43:12.512 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.513 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188693 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5881856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:12.513 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:43:12.513 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.682224902999998, 2024-09-17T22:43:12.513 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028193223, 2024-09-17T22:43:12.513 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.513 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.513 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.513 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.513 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.513 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.513 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188693 2024-09-17T22:43:12.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.513 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:43:12.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:43:12.514 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.514 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.514 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.514 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:43:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:43:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827090+0000", 2024-09-17T22:43:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2170000000000001e-06 2024-09-17T22:43:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827098+0000", 2024-09-17T22:43:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1510000000000004e-06 2024-09-17T22:43:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827103+0000", 2024-09-17T22:43:12.515 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0839999999999999e-06 2024-09-17T22:43:12.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855158+0000", 2024-09-17T22:43:12.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028055217 2024-09-17T22:43:12.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.516 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855201+0000", 2024-09-17T22:43:12.516 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3272000000000002e-05 2024-09-17T22:43:12.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.516 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.516 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855280+0000", 2024-09-17T22:43:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8845999999999997e-05 2024-09-17T22:43:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188694 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5971968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:43:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.682201016, 2024-09-17T22:43:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028312278, 2024-09-17T22:43:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.517 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188694 2024-09-17T22:43:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:43:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.518 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:43:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:43:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:43:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827114+0000", 2024-09-17T22:43:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3029999999999998e-06 2024-09-17T22:43:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.519 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827121+0000", 2024-09-17T22:43:12.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3699999999999997e-06 2024-09-17T22:43:12.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827126+0000", 2024-09-17T22:43:12.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7350000000000004e-06 2024-09-17T22:43:12.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.520 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.520 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.520 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855307+0000", 2024-09-17T22:43:12.520 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028180851999999999 2024-09-17T22:43:12.520 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855346+0000", 2024-09-17T22:43:12.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9274999999999999e-05 2024-09-17T22:43:12.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.521 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.521 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.521 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855423+0000", 2024-09-17T22:43:12.521 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7294999999999998e-05 2024-09-17T22:43:12.521 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.521 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.521 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.522 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.522 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188695 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:12.522 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:43:12.522 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.682180238000001, 2024-09-17T22:43:12.522 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028435876999999998, 2024-09-17T22:43:12.522 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.522 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.522 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.522 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.522 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188695 2024-09-17T22:43:12.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.523 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:43:12.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.523 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:43:12.523 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.523 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:43:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:43:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827138+0000", 2024-09-17T22:43:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2580000000000001e-06 2024-09-17T22:43:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827145+0000", 2024-09-17T22:43:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0129999999999996e-06 2024-09-17T22:43:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.524 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827150+0000", 2024-09-17T22:43:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3749999999999996e-06 2024-09-17T22:43:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855447+0000", 2024-09-17T22:43:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028297224999999999 2024-09-17T22:43:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855487+0000", 2024-09-17T22:43:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0454000000000003e-05 2024-09-17T22:43:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.525 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855567+0000", 2024-09-17T22:43:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0149999999999997e-05 2024-09-17T22:43:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188696 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:43:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.682156454000001, 2024-09-17T22:43:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028543570000000001, 2024-09-17T22:43:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.526 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188696 2024-09-17T22:43:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:43:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.527 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:43:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827156+0000", 2024-09-17T22:43:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T22:43:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827158+0000", 2024-09-17T22:43:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.875e-06 2024-09-17T22:43:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.528 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827164+0000", 2024-09-17T22:43:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7950000000000002e-06 2024-09-17T22:43:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827169+0000", 2024-09-17T22:43:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2880000000000003e-06 2024-09-17T22:43:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.529 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855582+0000", 2024-09-17T22:43:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028412929 2024-09-17T22:43:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855624+0000", 2024-09-17T22:43:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2484000000000002e-05 2024-09-17T22:43:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855699+0000", 2024-09-17T22:43:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4869000000000004e-05 2024-09-17T22:43:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.530 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.531 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.531 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188697 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6242304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:12.531 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:43:12.531 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 63.682124924999997, 2024-09-17T22:43:12.531 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028658598, 2024-09-17T22:43:12.531 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.531 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.531 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.531 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.531 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.531 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.531 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188697 2024-09-17T22:43:12.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.532 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:43:12.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.532 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.532 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.532 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:43:12.532 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:43:12.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8800000000000003e-07 2024-09-17T22:43:12.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.533 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827190+0000", 2024-09-17T22:43:12.533 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.351e-06 2024-09-17T22:43:12.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.533 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.533 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827198+0000", 2024-09-17T22:43:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.909e-06 2024-09-17T22:43:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827203+0000", 2024-09-17T22:43:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4020000000000001e-06 2024-09-17T22:43:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855720+0000", 2024-09-17T22:43:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028516909999999999 2024-09-17T22:43:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.534 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855762+0000", 2024-09-17T22:43:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2184999999999997e-05 2024-09-17T22:43:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855845+0000", 2024-09-17T22:43:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3353e-05 2024-09-17T22:43:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.535 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202883 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2342912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:43:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.784488543999998, 2024-09-17T22:43:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028098636, 2024-09-17T22:43:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202883 2024-09-17T22:43:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.536 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:43:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:43:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.537 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724824+0000", 2024-09-17T22:43:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:43:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724826+0000", 2024-09-17T22:43:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.086e-06 2024-09-17T22:43:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724832+0000", 2024-09-17T22:43:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9100000000000002e-06 2024-09-17T22:43:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.538 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724837+0000", 2024-09-17T22:43:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9439999999999996e-06 2024-09-17T22:43:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752790+0000", 2024-09-17T22:43:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027953657999999999 2024-09-17T22:43:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752848+0000", 2024-09-17T22:43:12.539 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7677999999999999e-05 2024-09-17T22:43:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752922+0000", 2024-09-17T22:43:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3957999999999999e-05 2024-09-17T22:43:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202884 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2473984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:43:12.540 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.784460318999997, 2024-09-17T22:43:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028206208999999999, 2024-09-17T22:43:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202884 2024-09-17T22:43:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.541 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:43:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:43:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724852+0000", 2024-09-17T22:43:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2599999999999998e-07 2024-09-17T22:43:12.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724854+0000", 2024-09-17T22:43:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.26e-06 2024-09-17T22:43:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724871+0000", 2024-09-17T22:43:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6589999999999999e-05 2024-09-17T22:43:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724877+0000", 2024-09-17T22:43:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.054e-06 2024-09-17T22:43:12.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752951+0000", 2024-09-17T22:43:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028073747 2024-09-17T22:43:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752993+0000", 2024-09-17T22:43:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2082000000000001e-05 2024-09-17T22:43:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753058+0000", 2024-09-17T22:43:12.544 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5049999999999996e-05 2024-09-17T22:43:12.545 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.545 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.545 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.545 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.545 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202885 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:12.545 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:43:12.545 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.784427016000002, 2024-09-17T22:43:12.545 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028333028, 2024-09-17T22:43:12.545 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.545 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.545 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.545 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202885 2024-09-17T22:43:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:43:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:43:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:43:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3300000000000002e-07 2024-09-17T22:43:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724892+0000", 2024-09-17T22:43:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2779999999999999e-06 2024-09-17T22:43:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724899+0000", 2024-09-17T22:43:12.547 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9110000000000004e-06 2024-09-17T22:43:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724904+0000", 2024-09-17T22:43:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2360000000000004e-06 2024-09-17T22:43:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753094+0000", 2024-09-17T22:43:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02819033 2024-09-17T22:43:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.548 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753142+0000", 2024-09-17T22:43:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.829e-05 2024-09-17T22:43:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753218+0000", 2024-09-17T22:43:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5450000000000004e-05 2024-09-17T22:43:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.549 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202886 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:43:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.784401817999999, 2024-09-17T22:43:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028442512, 2024-09-17T22:43:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202886 2024-09-17T22:43:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.550 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:43:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:43:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:43:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:43:12.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724912+0000", 2024-09-17T22:43:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.967e-06 2024-09-17T22:43:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724918+0000", 2024-09-17T22:43:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8069999999999996e-06 2024-09-17T22:43:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.552 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724923+0000", 2024-09-17T22:43:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.386e-06 2024-09-17T22:43:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753244+0000", 2024-09-17T22:43:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028321567999999998 2024-09-17T22:43:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753289+0000", 2024-09-17T22:43:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4620000000000003e-05 2024-09-17T22:43:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.553 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753353+0000", 2024-09-17T22:43:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3795999999999997e-05 2024-09-17T22:43:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202887 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:43:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.784383904999999, 2024-09-17T22:43:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028565838999999999, 2024-09-17T22:43:12.554 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202887 2024-09-17T22:43:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:43:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:43:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:43:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:43:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.556 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724930+0000", 2024-09-17T22:43:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8980000000000001e-06 2024-09-17T22:43:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724936+0000", 2024-09-17T22:43:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7849999999999997e-06 2024-09-17T22:43:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724942+0000", 2024-09-17T22:43:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9649999999999996e-06 2024-09-17T22:43:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753380+0000", 2024-09-17T22:43:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028438049 2024-09-17T22:43:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753425+0000", 2024-09-17T22:43:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5163000000000001e-05 2024-09-17T22:43:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753494+0000", 2024-09-17T22:43:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8588000000000001e-05 2024-09-17T22:43:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.558 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202888 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:43:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.784364406999998, 2024-09-17T22:43:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028703314000000001, 2024-09-17T22:43:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202888 2024-09-17T22:43:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.559 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:43:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:43:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724948+0000", 2024-09-17T22:43:12.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:43:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724950+0000", 2024-09-17T22:43:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8029999999999999e-06 2024-09-17T22:43:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724955+0000", 2024-09-17T22:43:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8760000000000002e-06 2024-09-17T22:43:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.561 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724960+0000", 2024-09-17T22:43:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5979999999999999e-06 2024-09-17T22:43:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753527+0000", 2024-09-17T22:43:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028566864000000001 2024-09-17T22:43:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753568+0000", 2024-09-17T22:43:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0775000000000002e-05 2024-09-17T22:43:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753651+0000", 2024-09-17T22:43:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3039000000000002e-05 2024-09-17T22:43:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202889 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:43:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.784346835000001, 2024-09-17T22:43:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028820964000000001, 2024-09-17T22:43:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.563 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202889 2024-09-17T22:43:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:43:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:43:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:43:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2599999999999998e-07 2024-09-17T22:43:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724967+0000", 2024-09-17T22:43:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T22:43:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724973+0000", 2024-09-17T22:43:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6060000000000002e-06 2024-09-17T22:43:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724978+0000", 2024-09-17T22:43:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8230000000000002e-06 2024-09-17T22:43:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753687+0000", 2024-09-17T22:43:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028709495000000002 2024-09-17T22:43:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753727+0000", 2024-09-17T22:43:12.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9953999999999997e-05 2024-09-17T22:43:12.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753786+0000", 2024-09-17T22:43:12.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8941999999999999e-05 2024-09-17T22:43:12.567 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.567 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.567 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.567 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202890 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:12.567 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:43:12.567 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.784329266999997, 2024-09-17T22:43:12.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028933838, 2024-09-17T22:43:12.568 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.568 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.568 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.568 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.568 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.568 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.568 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202890 2024-09-17T22:43:12.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.568 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:43:12.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:43:12.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:43:12.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5999999999999999e-07 2024-09-17T22:43:12.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724985+0000", 2024-09-17T22:43:12.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8169999999999999e-06 2024-09-17T22:43:12.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724991+0000", 2024-09-17T22:43:12.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:43:12.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724995+0000", 2024-09-17T22:43:12.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4530000000000004e-06 2024-09-17T22:43:12.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753811+0000", 2024-09-17T22:43:12.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028815584000000002 2024-09-17T22:43:12.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753854+0000", 2024-09-17T22:43:12.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3356999999999998e-05 2024-09-17T22:43:12.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.571 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.571 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.571 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753916+0000", 2024-09-17T22:43:12.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2488000000000002e-05 2024-09-17T22:43:12.571 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.571 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.571 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.572 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202891 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:12.572 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:43:12.572 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.784311809999998, 2024-09-17T22:43:12.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029067554999999998, 2024-09-17T22:43:12.572 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.572 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.572 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.572 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.572 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.572 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.572 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202891 2024-09-17T22:43:12.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.573 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:43:12.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:43:12.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:43:12.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:43:12.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725002+0000", 2024-09-17T22:43:12.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:43:12.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725008+0000", 2024-09-17T22:43:12.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6629999999999998e-06 2024-09-17T22:43:12.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725016+0000", 2024-09-17T22:43:12.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8269999999999998e-06 2024-09-17T22:43:12.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753949+0000", 2024-09-17T22:43:12.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028932625999999999 2024-09-17T22:43:12.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753994+0000", 2024-09-17T22:43:12.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5717000000000003e-05 2024-09-17T22:43:12.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754068+0000", 2024-09-17T22:43:12.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3312000000000006e-05 2024-09-17T22:43:12.576 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.576 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.576 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.576 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202892 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:12.576 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:43:12.576 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.784269103, 2024-09-17T22:43:12.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029159939999999999, 2024-09-17T22:43:12.576 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.577 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.577 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.577 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.577 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.577 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.577 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202892 2024-09-17T22:43:12.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.577 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:43:12.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:43:12.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:43:12.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7999999999999995e-07 2024-09-17T22:43:12.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725045+0000", 2024-09-17T22:43:12.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-17T22:43:12.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725052+0000", 2024-09-17T22:43:12.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3629999999999999e-06 2024-09-17T22:43:12.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725056+0000", 2024-09-17T22:43:12.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3250000000000001e-06 2024-09-17T22:43:12.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754094+0000", 2024-09-17T22:43:12.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029038270000000001 2024-09-17T22:43:12.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754134+0000", 2024-09-17T22:43:12.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9722999999999999e-05 2024-09-17T22:43:12.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.580 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.580 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754203+0000", 2024-09-17T22:43:12.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8524999999999999e-05 2024-09-17T22:43:12.580 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.580 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.580 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.581 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202893 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:12.581 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:43:12.581 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.784249525999996, 2024-09-17T22:43:12.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029341651, 2024-09-17T22:43:12.581 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.581 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.581 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.581 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.581 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.581 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.581 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202893 2024-09-17T22:43:12.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.581 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:43:12.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:43:12.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725063+0000", 2024-09-17T22:43:12.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6699999999999999e-07 2024-09-17T22:43:12.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725065+0000", 2024-09-17T22:43:12.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2120000000000002e-06 2024-09-17T22:43:12.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725071+0000", 2024-09-17T22:43:12.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5169999999999996e-06 2024-09-17T22:43:12.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725076+0000", 2024-09-17T22:43:12.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.78e-06 2024-09-17T22:43:12.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754233+0000", 2024-09-17T22:43:12.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029156277000000001 2024-09-17T22:43:12.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754298+0000", 2024-09-17T22:43:12.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5320000000000005e-05 2024-09-17T22:43:12.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.585 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.585 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754404+0000", 2024-09-17T22:43:12.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106078 2024-09-17T22:43:12.585 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.585 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.585 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.585 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202894 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:12.585 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:43:12.585 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.784229646999997, 2024-09-17T22:43:12.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029456572, 2024-09-17T22:43:12.585 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.586 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.586 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.586 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.586 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.586 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.586 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202894 2024-09-17T22:43:12.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.586 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:43:12.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:43:12.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725083+0000", 2024-09-17T22:43:12.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5900000000000002e-07 2024-09-17T22:43:12.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725085+0000", 2024-09-17T22:43:12.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.979e-06 2024-09-17T22:43:12.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725091+0000", 2024-09-17T22:43:12.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1480000000000003e-06 2024-09-17T22:43:12.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725095+0000", 2024-09-17T22:43:12.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1339999999999997e-06 2024-09-17T22:43:12.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754429+0000", 2024-09-17T22:43:12.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029333758000000001 2024-09-17T22:43:12.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754471+0000", 2024-09-17T22:43:12.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2274999999999998e-05 2024-09-17T22:43:12.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754539+0000", 2024-09-17T22:43:12.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7818999999999996e-05 2024-09-17T22:43:12.589 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.589 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.589 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.590 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202895 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:12.590 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:43:12.590 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.784211251000002, 2024-09-17T22:43:12.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029635578999999999, 2024-09-17T22:43:12.590 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.590 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.590 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.590 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.590 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.590 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.590 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202895 2024-09-17T22:43:12.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.591 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:43:12.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:43:12.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:43:12.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.08e-07 2024-09-17T22:43:12.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725107+0000", 2024-09-17T22:43:12.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9020000000000004e-06 2024-09-17T22:43:12.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725113+0000", 2024-09-17T22:43:12.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7400000000000001e-06 2024-09-17T22:43:12.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725117+0000", 2024-09-17T22:43:12.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:43:12.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754567+0000", 2024-09-17T22:43:12.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029449411000000002 2024-09-17T22:43:12.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754638+0000", 2024-09-17T22:43:12.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1432000000000003e-05 2024-09-17T22:43:12.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.594 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.594 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754736+0000", 2024-09-17T22:43:12.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8066000000000006e-05 2024-09-17T22:43:12.595 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.595 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.595 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.595 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202896 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:12.595 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:43:12.595 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.784189519999998, 2024-09-17T22:43:12.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029760534000000002, 2024-09-17T22:43:12.596 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.596 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.596 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.596 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.596 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.596 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.596 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202896 2024-09-17T22:43:12.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.596 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:43:12.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:43:12.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725123+0000", 2024-09-17T22:43:12.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:43:12.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725125+0000", 2024-09-17T22:43:12.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.863e-06 2024-09-17T22:43:12.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725130+0000", 2024-09-17T22:43:12.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7250000000000002e-06 2024-09-17T22:43:12.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725135+0000", 2024-09-17T22:43:12.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:43:12.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754763+0000", 2024-09-17T22:43:12.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029628202999999999 2024-09-17T22:43:12.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754805+0000", 2024-09-17T22:43:12.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2370000000000003e-05 2024-09-17T22:43:12.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754883+0000", 2024-09-17T22:43:12.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7565999999999996e-05 2024-09-17T22:43:12.600 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.600 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.600 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.600 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202897 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:12.600 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:43:12.600 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 54.784172159999997, 2024-09-17T22:43:12.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029947972999999999, 2024-09-17T22:43:12.601 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:12.601 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:12.601 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:12.601 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:12.601 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:12.601 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:12.601 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202897 2024-09-17T22:43:12.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.601 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:12.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:12.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:43:12.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:12.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:43:12.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:12.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:12.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:43:12.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:43:12.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:12.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725146+0000", 2024-09-17T22:43:12.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6030000000000004e-06 2024-09-17T22:43:12.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.603 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:12.603 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725152+0000", 2024-09-17T22:43:12.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9970000000000004e-06 2024-09-17T22:43:12.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:12.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725156+0000", 2024-09-17T22:43:12.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-06 2024-09-17T22:43:12.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:12.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754909+0000", 2024-09-17T22:43:12.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029752958999999999 2024-09-17T22:43:12.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:12.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754987+0000", 2024-09-17T22:43:12.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8208999999999996e-05 2024-09-17T22:43:12.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:12.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:12.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:12.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.755088+0000", 2024-09-17T22:43:12.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100348 2024-09-17T22:43:12.605 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.605 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.605 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.605 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:12.605 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:12.605 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:12.607 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:43:12.608 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:43:12.608 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:12.608 INFO:teuthology.orchestra.run.smithi115.stdout: "epoch": 124, 2024-09-17T22:43:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: "fsid": "551c5ed8-e790-485a-bc95-a9578a29275c", 2024-09-17T22:43:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: "created": "2024-09-17T22:40:50.189230+0000", 2024-09-17T22:43:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: "modified": "2024-09-17T22:43:12.085414+0000", 2024-09-17T22:43:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: "last_up_change": "2024-09-17T22:41:53.099515+0000", 2024-09-17T22:43:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: "last_in_change": "2024-09-17T22:40:52.260526+0000", 2024-09-17T22:43:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T22:43:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_num": 5806080, 2024-09-17T22:43:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_set": [ 2024-09-17T22:43:12.609 INFO:teuthology.orchestra.run.smithi115.stdout: "nodeep-scrub", 2024-09-17T22:43:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "noscrub", 2024-09-17T22:43:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "pglog_hardlimit", 2024-09-17T22:43:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "purged_snapdirs", 2024-09-17T22:43:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "recovery_deletes", 2024-09-17T22:43:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "sortbitwise" 2024-09-17T22:43:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:43:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_version": 5, 2024-09-17T22:43:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T22:43:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T22:43:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T22:43:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_snapshot": "", 2024-09-17T22:43:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_max": 3, 2024-09-17T22:43:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "max_osd": 8, 2024-09-17T22:43:12.610 INFO:teuthology.orchestra.run.smithi115.stdout: "require_min_compat_client": "luminous", 2024-09-17T22:43:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: "min_compat_client": "luminous", 2024-09-17T22:43:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: "require_osd_release": "squid", 2024-09-17T22:43:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: "allow_crimson": false, 2024-09-17T22:43:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: "pools": [ 2024-09-17T22:43:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 1, 2024-09-17T22:43:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_name": ".mgr", 2024-09-17T22:43:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: "create_time": "2024-09-17T22:41:02.338356+0000", 2024-09-17T22:43:12.611 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 1, 2024-09-17T22:43:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_names": "hashpspool", 2024-09-17T22:43:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "type": 1, 2024-09-17T22:43:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 2, 2024-09-17T22:43:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "min_size": 1, 2024-09-17T22:43:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_rule": 0, 2024-09-17T22:43:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:43:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:43:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:43:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:43:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "is_stretch_pool": false, 2024-09-17T22:43:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "object_hash": 2, 2024-09-17T22:43:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:43:12.612 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num": 1, 2024-09-17T22:43:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num": 1, 2024-09-17T22:43:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num_target": 1, 2024-09-17T22:43:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_target": 1, 2024-09-17T22:43:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_pending": 1, 2024-09-17T22:43:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "last_pg_merge_meta": { 2024-09-17T22:43:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "source_pgid": "0.0", 2024-09-17T22:43:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "ready_epoch": 0, 2024-09-17T22:43:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_started": 0, 2024-09-17T22:43:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_clean": 0, 2024-09-17T22:43:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "source_version": "0'0", 2024-09-17T22:43:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "target_version": "0'0" 2024-09-17T22:43:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "last_change": "17", 2024-09-17T22:43:12.613 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend": "0", 2024-09-17T22:43:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:43:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:43:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "auid": 0, 2024-09-17T22:43:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:43:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_seq": 0, 2024-09-17T22:43:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_epoch": 0, 2024-09-17T22:43:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_snaps": [], 2024-09-17T22:43:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps": "[]", 2024-09-17T22:43:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_bytes": 0, 2024-09-17T22:43:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_objects": 0, 2024-09-17T22:43:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "tiers": [], 2024-09-17T22:43:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "tier_of": -1, 2024-09-17T22:43:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "read_tier": -1, 2024-09-17T22:43:12.614 INFO:teuthology.orchestra.run.smithi115.stdout: "write_tier": -1, 2024-09-17T22:43:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_mode": "none", 2024-09-17T22:43:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_bytes": 0, 2024-09-17T22:43:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_objects": 0, 2024-09-17T22:43:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:43:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:43:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:43:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_flush_age": 0, 2024-09-17T22:43:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_evict_age": 0, 2024-09-17T22:43:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profile": "", 2024-09-17T22:43:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_params": { 2024-09-17T22:43:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "none" 2024-09-17T22:43:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.615 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_period": 0, 2024-09-17T22:43:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_count": 0, 2024-09-17T22:43:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: "use_gmt_hitset": true, 2024-09-17T22:43:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:43:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:43:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:43:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:43:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: "grade_table": [], 2024-09-17T22:43:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: "stripe_width": 0, 2024-09-17T22:43:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: "expected_num_objects": 0, 2024-09-17T22:43:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: "fast_read": false, 2024-09-17T22:43:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: "options": { 2024-09-17T22:43:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_max": 32, 2024-09-17T22:43:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_min": 1 2024-09-17T22:43:12.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.617 INFO:teuthology.orchestra.run.smithi115.stdout: "application_metadata": { 2024-09-17T22:43:12.617 INFO:teuthology.orchestra.run.smithi115.stdout: "mgr": {} 2024-09-17T22:43:12.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.617 INFO:teuthology.orchestra.run.smithi115.stdout: "read_balance": { 2024-09-17T22:43:12.617 INFO:teuthology.orchestra.run.smithi115.stdout: "score_type": "Fair distribution", 2024-09-17T22:43:12.617 INFO:teuthology.orchestra.run.smithi115.stdout: "score_acting": 6.9000000953674316, 2024-09-17T22:43:12.617 INFO:teuthology.orchestra.run.smithi115.stdout: "score_stable": 6.9000000953674316, 2024-09-17T22:43:12.617 INFO:teuthology.orchestra.run.smithi115.stdout: "optimal_score": 0.28999999165534973, 2024-09-17T22:43:12.617 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_acting": 2, 2024-09-17T22:43:12.617 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_stable": 2, 2024-09-17T22:43:12.617 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity_weighted": 0.81284332275390625, 2024-09-17T22:43:12.617 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-17T22:43:12.617 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity_weighted": 0.81000000238418579 2024-09-17T22:43:12.618 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.618 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 2, 2024-09-17T22:43:12.618 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_name": "rbd", 2024-09-17T22:43:12.618 INFO:teuthology.orchestra.run.smithi115.stdout: "create_time": "2024-09-17T22:41:08.097861+0000", 2024-09-17T22:43:12.618 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 8193, 2024-09-17T22:43:12.618 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T22:43:12.618 INFO:teuthology.orchestra.run.smithi115.stdout: "type": 1, 2024-09-17T22:43:12.618 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 2, 2024-09-17T22:43:12.618 INFO:teuthology.orchestra.run.smithi115.stdout: "min_size": 1, 2024-09-17T22:43:12.618 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_rule": 0, 2024-09-17T22:43:12.618 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:43:12.618 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:43:12.619 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:43:12.619 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:43:12.619 INFO:teuthology.orchestra.run.smithi115.stdout: "is_stretch_pool": false, 2024-09-17T22:43:12.619 INFO:teuthology.orchestra.run.smithi115.stdout: "object_hash": 2, 2024-09-17T22:43:12.619 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:43:12.619 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num": 8, 2024-09-17T22:43:12.619 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num": 8, 2024-09-17T22:43:12.619 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num_target": 8, 2024-09-17T22:43:12.619 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_target": 8, 2024-09-17T22:43:12.619 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_pending": 8, 2024-09-17T22:43:12.619 INFO:teuthology.orchestra.run.smithi115.stdout: "last_pg_merge_meta": { 2024-09-17T22:43:12.619 INFO:teuthology.orchestra.run.smithi115.stdout: "source_pgid": "0.0", 2024-09-17T22:43:12.619 INFO:teuthology.orchestra.run.smithi115.stdout: "ready_epoch": 0, 2024-09-17T22:43:12.619 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_started": 0, 2024-09-17T22:43:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_clean": 0, 2024-09-17T22:43:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: "source_version": "0'0", 2024-09-17T22:43:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: "target_version": "0'0" 2024-09-17T22:43:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: "last_change": "21", 2024-09-17T22:43:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend": "0", 2024-09-17T22:43:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:43:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:43:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: "auid": 0, 2024-09-17T22:43:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:43:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_seq": 2, 2024-09-17T22:43:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_epoch": 21, 2024-09-17T22:43:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_snaps": [], 2024-09-17T22:43:12.620 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps": "[]", 2024-09-17T22:43:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_bytes": 0, 2024-09-17T22:43:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_objects": 0, 2024-09-17T22:43:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "tiers": [], 2024-09-17T22:43:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "tier_of": -1, 2024-09-17T22:43:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "read_tier": -1, 2024-09-17T22:43:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "write_tier": -1, 2024-09-17T22:43:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_mode": "none", 2024-09-17T22:43:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_bytes": 0, 2024-09-17T22:43:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_objects": 0, 2024-09-17T22:43:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:43:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:43:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:43:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_flush_age": 0, 2024-09-17T22:43:12.621 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_evict_age": 0, 2024-09-17T22:43:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profile": "", 2024-09-17T22:43:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_params": { 2024-09-17T22:43:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "none" 2024-09-17T22:43:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_period": 0, 2024-09-17T22:43:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_count": 0, 2024-09-17T22:43:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "use_gmt_hitset": true, 2024-09-17T22:43:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:43:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:43:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:43:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:43:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "grade_table": [], 2024-09-17T22:43:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "stripe_width": 0, 2024-09-17T22:43:12.622 INFO:teuthology.orchestra.run.smithi115.stdout: "expected_num_objects": 0, 2024-09-17T22:43:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "fast_read": false, 2024-09-17T22:43:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "options": {}, 2024-09-17T22:43:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "application_metadata": { 2024-09-17T22:43:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd": {} 2024-09-17T22:43:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "read_balance": { 2024-09-17T22:43:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "score_type": "Fair distribution", 2024-09-17T22:43:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "score_acting": 2.619999885559082, 2024-09-17T22:43:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "score_stable": 2.619999885559082, 2024-09-17T22:43:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "optimal_score": 0.86000001430511475, 2024-09-17T22:43:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_acting": 2.25, 2024-09-17T22:43:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_stable": 2.25, 2024-09-17T22:43:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity_weighted": 0.81284332275390625, 2024-09-17T22:43:12.623 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-17T22:43:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity_weighted": 0.81000000238418579 2024-09-17T22:43:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 3, 2024-09-17T22:43:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_name": "pool_client.0", 2024-09-17T22:43:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "create_time": "2024-09-17T22:41:17.501542+0000", 2024-09-17T22:43:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "flags": 8193, 2024-09-17T22:43:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T22:43:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "type": 1, 2024-09-17T22:43:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 2, 2024-09-17T22:43:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "min_size": 1, 2024-09-17T22:43:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_rule": 0, 2024-09-17T22:43:12.624 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_count": 0, 2024-09-17T22:43:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_target": 0, 2024-09-17T22:43:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T22:43:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T22:43:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "is_stretch_pool": false, 2024-09-17T22:43:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "object_hash": 2, 2024-09-17T22:43:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_autoscale_mode": "off", 2024-09-17T22:43:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num": 32, 2024-09-17T22:43:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num": 32, 2024-09-17T22:43:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_placement_num_target": 32, 2024-09-17T22:43:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_target": 32, 2024-09-17T22:43:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_num_pending": 32, 2024-09-17T22:43:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "last_pg_merge_meta": { 2024-09-17T22:43:12.625 INFO:teuthology.orchestra.run.smithi115.stdout: "source_pgid": "0.0", 2024-09-17T22:43:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: "ready_epoch": 0, 2024-09-17T22:43:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_started": 0, 2024-09-17T22:43:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: "last_epoch_clean": 0, 2024-09-17T22:43:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: "source_version": "0'0", 2024-09-17T22:43:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: "target_version": "0'0" 2024-09-17T22:43:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: "last_change": "124", 2024-09-17T22:43:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend": "0", 2024-09-17T22:43:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T22:43:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T22:43:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: "auid": 0, 2024-09-17T22:43:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_mode": "selfmanaged", 2024-09-17T22:43:12.626 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_seq": 40, 2024-09-17T22:43:12.627 INFO:teuthology.orchestra.run.smithi115.stdout: "snap_epoch": 124, 2024-09-17T22:43:12.627 INFO:teuthology.orchestra.run.smithi115.stdout: "pool_snaps": [], 2024-09-17T22:43:12.627 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps": "[]", 2024-09-17T22:43:12.627 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_bytes": 0, 2024-09-17T22:43:12.627 INFO:teuthology.orchestra.run.smithi115.stdout: "quota_max_objects": 0, 2024-09-17T22:43:12.627 INFO:teuthology.orchestra.run.smithi115.stdout: "tiers": [], 2024-09-17T22:43:12.627 INFO:teuthology.orchestra.run.smithi115.stdout: "tier_of": -1, 2024-09-17T22:43:12.627 INFO:teuthology.orchestra.run.smithi115.stdout: "read_tier": -1, 2024-09-17T22:43:12.627 INFO:teuthology.orchestra.run.smithi115.stdout: "write_tier": -1, 2024-09-17T22:43:12.627 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_mode": "none", 2024-09-17T22:43:12.627 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_bytes": 0, 2024-09-17T22:43:12.627 INFO:teuthology.orchestra.run.smithi115.stdout: "target_max_objects": 0, 2024-09-17T22:43:12.627 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T22:43:12.627 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T22:43:12.628 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T22:43:12.628 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_flush_age": 0, 2024-09-17T22:43:12.628 INFO:teuthology.orchestra.run.smithi115.stdout: "cache_min_evict_age": 0, 2024-09-17T22:43:12.628 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profile": "", 2024-09-17T22:43:12.628 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_params": { 2024-09-17T22:43:12.628 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "none" 2024-09-17T22:43:12.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.628 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_period": 0, 2024-09-17T22:43:12.628 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_count": 0, 2024-09-17T22:43:12.628 INFO:teuthology.orchestra.run.smithi115.stdout: "use_gmt_hitset": true, 2024-09-17T22:43:12.628 INFO:teuthology.orchestra.run.smithi115.stdout: "min_read_recency_for_promote": 0, 2024-09-17T22:43:12.628 INFO:teuthology.orchestra.run.smithi115.stdout: "min_write_recency_for_promote": 0, 2024-09-17T22:43:12.628 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T22:43:12.629 INFO:teuthology.orchestra.run.smithi115.stdout: "hit_set_search_last_n": 0, 2024-09-17T22:43:12.629 INFO:teuthology.orchestra.run.smithi115.stdout: "grade_table": [], 2024-09-17T22:43:12.629 INFO:teuthology.orchestra.run.smithi115.stdout: "stripe_width": 0, 2024-09-17T22:43:12.629 INFO:teuthology.orchestra.run.smithi115.stdout: "expected_num_objects": 0, 2024-09-17T22:43:12.629 INFO:teuthology.orchestra.run.smithi115.stdout: "fast_read": false, 2024-09-17T22:43:12.629 INFO:teuthology.orchestra.run.smithi115.stdout: "options": {}, 2024-09-17T22:43:12.629 INFO:teuthology.orchestra.run.smithi115.stdout: "application_metadata": { 2024-09-17T22:43:12.629 INFO:teuthology.orchestra.run.smithi115.stdout: "rbd": {} 2024-09-17T22:43:12.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.629 INFO:teuthology.orchestra.run.smithi115.stdout: "read_balance": { 2024-09-17T22:43:12.629 INFO:teuthology.orchestra.run.smithi115.stdout: "score_type": "Fair distribution", 2024-09-17T22:43:12.629 INFO:teuthology.orchestra.run.smithi115.stdout: "score_acting": 1.5299999713897705, 2024-09-17T22:43:12.629 INFO:teuthology.orchestra.run.smithi115.stdout: "score_stable": 1.5299999713897705, 2024-09-17T22:43:12.629 INFO:teuthology.orchestra.run.smithi115.stdout: "optimal_score": 1, 2024-09-17T22:43:12.630 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-17T22:43:12.630 INFO:teuthology.orchestra.run.smithi115.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-17T22:43:12.630 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity_weighted": 0.81284332275390625, 2024-09-17T22:43:12.630 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-17T22:43:12.630 INFO:teuthology.orchestra.run.smithi115.stdout: "average_primary_affinity_weighted": 0.81000000238418579 2024-09-17T22:43:12.630 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.630 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.630 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:43:12.630 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:43:12.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.630 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 0, 2024-09-17T22:43:12.630 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "3bd5c11f-e7c2-4968-839a-2799c00d93a6", 2024-09-17T22:43:12.630 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 0, 2024-09-17T22:43:12.630 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:43:12.631 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:43:12.631 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 0, 2024-09-17T22:43:12.631 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:43:12.631 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:43:12.631 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:43:12.631 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 23, 2024-09-17T22:43:12.631 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 51, 2024-09-17T22:43:12.631 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:43:12.631 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:43:12.631 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.631 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.631 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6824", 2024-09-17T22:43:12.631 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:43:12.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.632 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.632 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6825", 2024-09-17T22:43:12.632 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:43:12.632 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.632 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.632 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:43:12.632 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.632 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.632 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6826", 2024-09-17T22:43:12.633 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:43:12.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.633 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.633 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6827", 2024-09-17T22:43:12.633 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:43:12.633 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.633 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.633 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:43:12.633 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.633 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.633 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6830", 2024-09-17T22:43:12.634 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:43:12.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.634 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.634 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6831", 2024-09-17T22:43:12.634 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:43:12.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.634 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.634 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:43:12.634 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.634 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.635 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6828", 2024-09-17T22:43:12.635 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:43:12.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.635 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.635 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6829", 2024-09-17T22:43:12.635 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 2782271783 2024-09-17T22:43:12.635 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.635 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.635 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6825/2782271783", 2024-09-17T22:43:12.635 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6827/2782271783", 2024-09-17T22:43:12.635 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6831/2782271783", 2024-09-17T22:43:12.636 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6829/2782271783", 2024-09-17T22:43:12.636 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:43:12.636 INFO:teuthology.orchestra.run.smithi115.stdout: "exists" 2024-09-17T22:43:12.636 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.636 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 1, 2024-09-17T22:43:12.636 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "02918203-cf2e-49fc-b994-743b5d81b4a1", 2024-09-17T22:43:12.636 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:43:12.636 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:43:12.636 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:43:12.636 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:43:12.636 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:43:12.637 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:43:12.637 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:43:12.637 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 109, 2024-09-17T22:43:12.637 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:43:12.637 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:43:12.637 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:43:12.637 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.637 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.637 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6816", 2024-09-17T22:43:12.637 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:43:12.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.637 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.638 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6817", 2024-09-17T22:43:12.638 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:43:12.638 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.638 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.638 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:43:12.638 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.638 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.638 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6818", 2024-09-17T22:43:12.638 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:43:12.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.638 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.639 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6819", 2024-09-17T22:43:12.639 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:43:12.639 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.639 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.639 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:43:12.639 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.639 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.639 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6822", 2024-09-17T22:43:12.639 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:43:12.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.639 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.640 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6823", 2024-09-17T22:43:12.640 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:43:12.640 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.640 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.640 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:43:12.640 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.640 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.640 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6820", 2024-09-17T22:43:12.640 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:43:12.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.640 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.641 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6821", 2024-09-17T22:43:12.641 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1941780390 2024-09-17T22:43:12.641 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.641 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.641 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6817/1941780390", 2024-09-17T22:43:12.641 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6819/1941780390", 2024-09-17T22:43:12.641 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6823/1941780390", 2024-09-17T22:43:12.641 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6821/1941780390", 2024-09-17T22:43:12.641 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:43:12.641 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:43:12.641 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:43:12.641 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.642 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 2, 2024-09-17T22:43:12.642 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "22ee34ef-522e-4121-85b7-38bbafeca5de", 2024-09-17T22:43:12.642 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:43:12.642 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:43:12.642 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:43:12.642 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:43:12.642 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:43:12.642 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:43:12.642 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 13, 2024-09-17T22:43:12.642 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 99, 2024-09-17T22:43:12.642 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:43:12.642 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:43:12.642 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:43:12.643 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.643 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.643 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6808", 2024-09-17T22:43:12.643 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:43:12.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.643 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.643 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6809", 2024-09-17T22:43:12.643 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:43:12.643 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.643 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.644 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:43:12.644 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.644 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.644 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6810", 2024-09-17T22:43:12.644 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:43:12.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.644 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.644 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6811", 2024-09-17T22:43:12.644 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:43:12.644 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.644 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.645 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:43:12.645 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.645 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.645 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6814", 2024-09-17T22:43:12.645 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:43:12.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.645 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.645 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6815", 2024-09-17T22:43:12.645 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:43:12.645 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.645 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.646 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:43:12.646 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.646 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.646 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6812", 2024-09-17T22:43:12.646 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:43:12.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.646 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.646 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6813", 2024-09-17T22:43:12.646 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 689014698 2024-09-17T22:43:12.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.646 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.647 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6809/689014698", 2024-09-17T22:43:12.647 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6811/689014698", 2024-09-17T22:43:12.647 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6815/689014698", 2024-09-17T22:43:12.647 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6813/689014698", 2024-09-17T22:43:12.647 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:43:12.647 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:43:12.647 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:43:12.647 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.647 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 3, 2024-09-17T22:43:12.647 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "5d464921-5696-43c5-b89d-430a54ddefb5", 2024-09-17T22:43:12.647 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:43:12.648 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:43:12.648 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:43:12.648 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 0.50274658203125, 2024-09-17T22:43:12.648 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:43:12.648 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:43:12.648 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 13, 2024-09-17T22:43:12.648 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 54, 2024-09-17T22:43:12.648 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:43:12.648 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:43:12.648 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:43:12.648 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.648 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.649 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6800", 2024-09-17T22:43:12.649 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:43:12.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.649 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.649 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6801", 2024-09-17T22:43:12.649 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:43:12.649 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.649 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.649 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:43:12.649 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.649 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.650 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6802", 2024-09-17T22:43:12.650 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:43:12.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.650 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.650 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6803", 2024-09-17T22:43:12.650 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:43:12.650 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.650 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.650 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:43:12.650 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.650 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.651 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6806", 2024-09-17T22:43:12.651 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:43:12.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.651 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.651 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6807", 2024-09-17T22:43:12.651 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:43:12.651 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.651 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.651 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:43:12.651 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.652 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.652 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6804", 2024-09-17T22:43:12.652 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:43:12.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.652 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.652 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.115:6805", 2024-09-17T22:43:12.652 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 453192890 2024-09-17T22:43:12.652 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.652 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.652 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.115:6801/453192890", 2024-09-17T22:43:12.652 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.115:6803/453192890", 2024-09-17T22:43:12.653 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.115:6807/453192890", 2024-09-17T22:43:12.653 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.115:6805/453192890", 2024-09-17T22:43:12.653 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:43:12.653 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:43:12.653 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:43:12.653 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.653 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 4, 2024-09-17T22:43:12.653 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "4cb6973a-63ae-4f48-af94-37610aee3c06", 2024-09-17T22:43:12.653 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:43:12.653 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:43:12.653 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:43:12.653 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:43:12.654 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:43:12.654 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:43:12.654 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:43:12.654 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 51, 2024-09-17T22:43:12.654 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:43:12.654 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:43:12.654 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:43:12.654 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.654 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.654 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6800", 2024-09-17T22:43:12.654 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:43:12.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.655 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.655 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6801", 2024-09-17T22:43:12.655 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:43:12.655 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.655 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.655 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:43:12.655 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.655 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.655 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6802", 2024-09-17T22:43:12.655 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:43:12.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.656 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.656 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6803", 2024-09-17T22:43:12.656 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:43:12.656 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.656 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.656 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:43:12.656 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.656 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.656 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6806", 2024-09-17T22:43:12.656 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:43:12.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.657 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.657 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6807", 2024-09-17T22:43:12.657 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:43:12.657 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.657 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.657 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:43:12.657 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.657 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.657 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6804", 2024-09-17T22:43:12.657 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:43:12.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.658 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.658 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6805", 2024-09-17T22:43:12.658 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 380388860 2024-09-17T22:43:12.658 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.658 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.658 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6801/380388860", 2024-09-17T22:43:12.658 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6803/380388860", 2024-09-17T22:43:12.658 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6807/380388860", 2024-09-17T22:43:12.658 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6805/380388860", 2024-09-17T22:43:12.658 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:43:12.658 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:43:12.659 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:43:12.659 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.659 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 5, 2024-09-17T22:43:12.659 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "ef9fe11b-b719-48f4-9c6c-e1b1b355acdc", 2024-09-17T22:43:12.659 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:43:12.659 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:43:12.659 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:43:12.659 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:43:12.659 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:43:12.659 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:43:12.659 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:43:12.659 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 23, 2024-09-17T22:43:12.660 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:43:12.660 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:43:12.660 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:43:12.660 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.660 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.660 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6808", 2024-09-17T22:43:12.660 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:43:12.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.660 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.660 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6809", 2024-09-17T22:43:12.660 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:43:12.661 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.661 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.661 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:43:12.661 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.661 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.661 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6810", 2024-09-17T22:43:12.661 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:43:12.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.661 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.661 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6811", 2024-09-17T22:43:12.661 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:43:12.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.662 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.662 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:43:12.662 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.662 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.662 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6814", 2024-09-17T22:43:12.662 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:43:12.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.662 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.662 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6815", 2024-09-17T22:43:12.662 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:43:12.663 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.663 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.663 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:43:12.663 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.663 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.663 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6812", 2024-09-17T22:43:12.663 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:43:12.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.663 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.663 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6813", 2024-09-17T22:43:12.663 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 943488949 2024-09-17T22:43:12.664 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.664 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.664 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6809/943488949", 2024-09-17T22:43:12.664 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6811/943488949", 2024-09-17T22:43:12.664 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6815/943488949", 2024-09-17T22:43:12.664 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6813/943488949", 2024-09-17T22:43:12.664 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:43:12.664 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:43:12.664 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:43:12.664 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.665 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 6, 2024-09-17T22:43:12.665 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "e4045aec-22c1-401b-bfbf-64f115bc32af", 2024-09-17T22:43:12.665 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:43:12.665 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:43:12.665 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:43:12.665 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:43:12.665 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:43:12.665 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:43:12.665 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:43:12.665 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 99, 2024-09-17T22:43:12.665 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:43:12.665 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:43:12.665 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:43:12.665 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.666 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.666 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6816", 2024-09-17T22:43:12.666 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:43:12.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.666 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.666 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6817", 2024-09-17T22:43:12.666 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:43:12.666 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.666 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.666 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:43:12.667 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.667 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.667 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6818", 2024-09-17T22:43:12.667 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:43:12.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.667 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.667 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6819", 2024-09-17T22:43:12.667 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:43:12.667 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.667 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.667 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:43:12.668 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.668 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.668 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6822", 2024-09-17T22:43:12.668 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:43:12.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.668 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.668 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6823", 2024-09-17T22:43:12.668 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:43:12.668 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.668 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.669 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:43:12.669 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.669 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.669 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6820", 2024-09-17T22:43:12.669 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:43:12.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.669 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.669 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6821", 2024-09-17T22:43:12.669 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 1873501077 2024-09-17T22:43:12.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.669 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.670 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6817/1873501077", 2024-09-17T22:43:12.670 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6819/1873501077", 2024-09-17T22:43:12.670 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6823/1873501077", 2024-09-17T22:43:12.670 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6821/1873501077", 2024-09-17T22:43:12.670 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:43:12.670 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:43:12.670 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:43:12.670 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.670 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 7, 2024-09-17T22:43:12.670 INFO:teuthology.orchestra.run.smithi115.stdout: "uuid": "4a05a5d7-8281-4572-a9d0-6095ee5e168b", 2024-09-17T22:43:12.670 INFO:teuthology.orchestra.run.smithi115.stdout: "up": 1, 2024-09-17T22:43:12.671 INFO:teuthology.orchestra.run.smithi115.stdout: "in": 1, 2024-09-17T22:43:12.671 INFO:teuthology.orchestra.run.smithi115.stdout: "weight": 1, 2024-09-17T22:43:12.671 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_affinity": 1, 2024-09-17T22:43:12.671 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_begin": 0, 2024-09-17T22:43:12.671 INFO:teuthology.orchestra.run.smithi115.stdout: "last_clean_end": 0, 2024-09-17T22:43:12.671 INFO:teuthology.orchestra.run.smithi115.stdout: "up_from": 14, 2024-09-17T22:43:12.671 INFO:teuthology.orchestra.run.smithi115.stdout: "up_thru": 108, 2024-09-17T22:43:12.671 INFO:teuthology.orchestra.run.smithi115.stdout: "down_at": 0, 2024-09-17T22:43:12.671 INFO:teuthology.orchestra.run.smithi115.stdout: "lost_at": 0, 2024-09-17T22:43:12.671 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addrs": { 2024-09-17T22:43:12.671 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.671 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.672 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6824", 2024-09-17T22:43:12.672 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:43:12.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.672 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.672 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6825", 2024-09-17T22:43:12.672 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:43:12.672 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.672 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.672 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addrs": { 2024-09-17T22:43:12.672 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.672 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.673 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6826", 2024-09-17T22:43:12.673 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:43:12.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.673 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.673 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6827", 2024-09-17T22:43:12.673 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:43:12.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.673 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.673 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addrs": { 2024-09-17T22:43:12.673 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.674 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.674 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6830", 2024-09-17T22:43:12.674 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:43:12.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.674 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.674 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6831", 2024-09-17T22:43:12.674 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:43:12.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.674 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.674 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addrs": { 2024-09-17T22:43:12.674 INFO:teuthology.orchestra.run.smithi115.stdout: "addrvec": [ 2024-09-17T22:43:12.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.675 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v2", 2024-09-17T22:43:12.675 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6828", 2024-09-17T22:43:12.675 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:43:12.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.675 INFO:teuthology.orchestra.run.smithi115.stdout: "type": "v1", 2024-09-17T22:43:12.675 INFO:teuthology.orchestra.run.smithi115.stdout: "addr": "172.21.15.129:6829", 2024-09-17T22:43:12.675 INFO:teuthology.orchestra.run.smithi115.stdout: "nonce": 3609357666 2024-09-17T22:43:12.675 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.675 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.675 INFO:teuthology.orchestra.run.smithi115.stdout: "public_addr": "172.21.15.129:6825/3609357666", 2024-09-17T22:43:12.675 INFO:teuthology.orchestra.run.smithi115.stdout: "cluster_addr": "172.21.15.129:6827/3609357666", 2024-09-17T22:43:12.675 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_back_addr": "172.21.15.129:6831/3609357666", 2024-09-17T22:43:12.676 INFO:teuthology.orchestra.run.smithi115.stdout: "heartbeat_front_addr": "172.21.15.129:6829/3609357666", 2024-09-17T22:43:12.676 INFO:teuthology.orchestra.run.smithi115.stdout: "state": [ 2024-09-17T22:43:12.676 INFO:teuthology.orchestra.run.smithi115.stdout: "exists", 2024-09-17T22:43:12.676 INFO:teuthology.orchestra.run.smithi115.stdout: "up" 2024-09-17T22:43:12.676 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.676 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.676 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:43:12.676 INFO:teuthology.orchestra.run.smithi115.stdout: "osd_xinfo": [ 2024-09-17T22:43:12.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.676 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 0, 2024-09-17T22:43:12.676 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:43:12.676 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:43:12.676 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:43:12.677 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:43:12.677 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:43:12.677 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:59.351758+0000", 2024-09-17T22:43:12.677 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 51 2024-09-17T22:43:12.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.677 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 1, 2024-09-17T22:43:12.677 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:43:12.677 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:43:12.677 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:43:12.677 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:43:12.677 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:43:12.677 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:59.058915+0000", 2024-09-17T22:43:12.677 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:43:12.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.678 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 2, 2024-09-17T22:43:12.678 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:43:12.678 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:43:12.678 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:43:12.678 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:43:12.678 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:43:12.678 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.922383+0000", 2024-09-17T22:43:12.678 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:43:12.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.678 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 3, 2024-09-17T22:43:12.678 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:43:12.679 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:43:12.679 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:43:12.679 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:43:12.679 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:43:12.679 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.802379+0000", 2024-09-17T22:43:12.679 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:43:12.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.679 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 4, 2024-09-17T22:43:12.679 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:43:12.679 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:43:12.679 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:43:12.679 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:43:12.679 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:43:12.680 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.881681+0000", 2024-09-17T22:43:12.680 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:43:12.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.680 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 5, 2024-09-17T22:43:12.680 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:43:12.680 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:43:12.680 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:43:12.680 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:43:12.680 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:43:12.680 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.616649+0000", 2024-09-17T22:43:12.680 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:43:12.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.681 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 6, 2024-09-17T22:43:12.681 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:43:12.681 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:43:12.681 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:43:12.681 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:43:12.681 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:43:12.681 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:58.589555+0000", 2024-09-17T22:43:12.681 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:43:12.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.681 INFO:teuthology.orchestra.run.smithi115.stdout: "osd": 7, 2024-09-17T22:43:12.681 INFO:teuthology.orchestra.run.smithi115.stdout: "down_stamp": "0.000000", 2024-09-17T22:43:12.681 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_probability": 0, 2024-09-17T22:43:12.682 INFO:teuthology.orchestra.run.smithi115.stdout: "laggy_interval": 0, 2024-09-17T22:43:12.682 INFO:teuthology.orchestra.run.smithi115.stdout: "features": 4540701547738038271, 2024-09-17T22:43:12.682 INFO:teuthology.orchestra.run.smithi115.stdout: "old_weight": 0, 2024-09-17T22:43:12.682 INFO:teuthology.orchestra.run.smithi115.stdout: "last_purged_snaps_scrub": "2024-09-17T22:40:59.047783+0000", 2024-09-17T22:43:12.682 INFO:teuthology.orchestra.run.smithi115.stdout: "dead_epoch": 0 2024-09-17T22:43:12.682 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.682 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:43:12.682 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_upmap": [ 2024-09-17T22:43:12.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.682 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.1c", 2024-09-17T22:43:12.682 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:43:12.682 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-09-17T22:43:12.682 INFO:teuthology.orchestra.run.smithi115.stdout: 3 2024-09-17T22:43:12.682 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.683 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.1d", 2024-09-17T22:43:12.683 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:43:12.683 INFO:teuthology.orchestra.run.smithi115.stdout: 1, 2024-09-17T22:43:12.683 INFO:teuthology.orchestra.run.smithi115.stdout: 6 2024-09-17T22:43:12.683 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.683 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:43:12.683 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_upmap_items": [ 2024-09-17T22:43:12.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.683 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.12", 2024-09-17T22:43:12.683 INFO:teuthology.orchestra.run.smithi115.stdout: "mappings": [ 2024-09-17T22:43:12.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.684 INFO:teuthology.orchestra.run.smithi115.stdout: "from": 3, 2024-09-17T22:43:12.684 INFO:teuthology.orchestra.run.smithi115.stdout: "to": 0 2024-09-17T22:43:12.684 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.684 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.684 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.16", 2024-09-17T22:43:12.684 INFO:teuthology.orchestra.run.smithi115.stdout: "mappings": [ 2024-09-17T22:43:12.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.684 INFO:teuthology.orchestra.run.smithi115.stdout: "from": 2, 2024-09-17T22:43:12.684 INFO:teuthology.orchestra.run.smithi115.stdout: "to": 0 2024-09-17T22:43:12.684 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.684 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.684 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.685 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:43:12.685 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_upmap_primaries": [], 2024-09-17T22:43:12.685 INFO:teuthology.orchestra.run.smithi115.stdout: "pg_temp": [ 2024-09-17T22:43:12.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.685 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.12", 2024-09-17T22:43:12.685 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:43:12.685 INFO:teuthology.orchestra.run.smithi115.stdout: 6, 2024-09-17T22:43:12.685 INFO:teuthology.orchestra.run.smithi115.stdout: 3 2024-09-17T22:43:12.685 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.685 INFO:teuthology.orchestra.run.smithi115.stdout: "pgid": "3.16", 2024-09-17T22:43:12.685 INFO:teuthology.orchestra.run.smithi115.stdout: "osds": [ 2024-09-17T22:43:12.686 INFO:teuthology.orchestra.run.smithi115.stdout: 3, 2024-09-17T22:43:12.686 INFO:teuthology.orchestra.run.smithi115.stdout: 2 2024-09-17T22:43:12.686 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.686 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:43:12.686 INFO:teuthology.orchestra.run.smithi115.stdout: "primary_temp": [], 2024-09-17T22:43:12.686 INFO:teuthology.orchestra.run.smithi115.stdout: "blocklist": {}, 2024-09-17T22:43:12.686 INFO:teuthology.orchestra.run.smithi115.stdout: "range_blocklist": {}, 2024-09-17T22:43:12.686 INFO:teuthology.orchestra.run.smithi115.stdout: "erasure_code_profiles": { 2024-09-17T22:43:12.686 INFO:teuthology.orchestra.run.smithi115.stdout: "default": { 2024-09-17T22:43:12.686 INFO:teuthology.orchestra.run.smithi115.stdout: "crush-failure-domain": "osd", 2024-09-17T22:43:12.686 INFO:teuthology.orchestra.run.smithi115.stdout: "k": "2", 2024-09-17T22:43:12.686 INFO:teuthology.orchestra.run.smithi115.stdout: "m": "1", 2024-09-17T22:43:12.686 INFO:teuthology.orchestra.run.smithi115.stdout: "plugin": "jerasure", 2024-09-17T22:43:12.686 INFO:teuthology.orchestra.run.smithi115.stdout: "technique": "reed_sol_van" 2024-09-17T22:43:12.687 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:12.687 INFO:teuthology.orchestra.run.smithi115.stdout: "removed_snaps_queue": [ 2024-09-17T22:43:12.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.687 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 3, 2024-09-17T22:43:12.687 INFO:teuthology.orchestra.run.smithi115.stdout: "snaps": [ 2024-09-17T22:43:12.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.687 INFO:teuthology.orchestra.run.smithi115.stdout: "begin": 10, 2024-09-17T22:43:12.687 INFO:teuthology.orchestra.run.smithi115.stdout: "length": 31 2024-09-17T22:43:12.687 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.687 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.687 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.687 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:43:12.687 INFO:teuthology.orchestra.run.smithi115.stdout: "new_removed_snaps": [ 2024-09-17T22:43:12.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.688 INFO:teuthology.orchestra.run.smithi115.stdout: "pool": 3, 2024-09-17T22:43:12.688 INFO:teuthology.orchestra.run.smithi115.stdout: "snaps": [ 2024-09-17T22:43:12.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:12.688 INFO:teuthology.orchestra.run.smithi115.stdout: "begin": 10, 2024-09-17T22:43:12.688 INFO:teuthology.orchestra.run.smithi115.stdout: "length": 1 2024-09-17T22:43:12.688 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.688 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:12.688 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.688 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-09-17T22:43:12.688 INFO:teuthology.orchestra.run.smithi115.stdout: "new_purged_snaps": [], 2024-09-17T22:43:12.688 INFO:teuthology.orchestra.run.smithi115.stdout: "crush_node_flags": {}, 2024-09-17T22:43:12.688 INFO:teuthology.orchestra.run.smithi115.stdout: "device_class_flags": {}, 2024-09-17T22:43:12.688 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode": { 2024-09-17T22:43:12.689 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode_enabled": false, 2024-09-17T22:43:12.689 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_bucket_count": 0, 2024-09-17T22:43:12.689 INFO:teuthology.orchestra.run.smithi115.stdout: "degraded_stretch_mode": 0, 2024-09-17T22:43:12.689 INFO:teuthology.orchestra.run.smithi115.stdout: "recovering_stretch_mode": 0, 2024-09-17T22:43:12.689 INFO:teuthology.orchestra.run.smithi115.stdout: "stretch_mode_bucket": 0 2024-09-17T22:43:12.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:12.689 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:12.690 INFO:tasks.thrashosds.thrasher:j is {'epoch': 124, 'fsid': '551c5ed8-e790-485a-bc95-a9578a29275c', 'created': '2024-09-17T22:40:50.189230+0000', 'modified': '2024-09-17T22:43:12.085414+0000', 'last_up_change': '2024-09-17T22:41:53.099515+0000', 'last_in_change': '2024-09-17T22:40:52.260526+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T22:41:02.338356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8128433227539062, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.8100000023841858}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T22:41:08.097861+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.619999885559082, 'score_stable': 2.619999885559082, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8128433227539062, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.8100000023841858}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T22:41:17.501542+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 40, 'snap_epoch': 124, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5299999713897705, 'score_stable': 1.5299999713897705, 'optimal_score': 1, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 0.8128433227539062, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.8100000023841858}}], 'osds': [{'osd': 0, 'uuid': '3bd5c11f-e7c2-4968-839a-2799c00d93a6', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 51, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6824', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6825', 'nonce': 2782271783}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 2782271783}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 2782271783}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 2782271783}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 2782271783}]}, 'public_addr': '172.21.15.115:6825/2782271783', 'cluster_addr': '172.21.15.115:6827/2782271783', 'heartbeat_back_addr': '172.21.15.115:6831/2782271783', 'heartbeat_front_addr': '172.21.15.115:6829/2782271783', 'state': ['exists']}, {'osd': 1, 'uuid': '02918203-cf2e-49fc-b994-743b5d81b4a1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 109, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6816', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 1941780390}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6819', 'nonce': 1941780390}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 1941780390}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6820', 'nonce': 1941780390}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 1941780390}]}, 'public_addr': '172.21.15.115:6817/1941780390', 'cluster_addr': '172.21.15.115:6819/1941780390', 'heartbeat_back_addr': '172.21.15.115:6823/1941780390', 'heartbeat_front_addr': '172.21.15.115:6821/1941780390', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '22ee34ef-522e-4121-85b7-38bbafeca5de', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 689014698}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6811', 'nonce': 689014698}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6815', 'nonce': 689014698}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6812', 'nonce': 689014698}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 689014698}]}, 'public_addr': '172.21.15.115:6809/689014698', 'cluster_addr': '172.21.15.115:6811/689014698', 'heartbeat_back_addr': '172.21.15.115:6815/689014698', 'heartbeat_front_addr': '172.21.15.115:6813/689014698', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5d464921-5696-43c5-b89d-430a54ddefb5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.50274658203125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 453192890}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 453192890}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 453192890}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 453192890}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 453192890}]}, 'public_addr': '172.21.15.115:6801/453192890', 'cluster_addr': '172.21.15.115:6803/453192890', 'heartbeat_back_addr': '172.21.15.115:6807/453192890', 'heartbeat_front_addr': '172.21.15.115:6805/453192890', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '4cb6973a-63ae-4f48-af94-37610aee3c06', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6800', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6801', 'nonce': 380388860}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6802', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6803', 'nonce': 380388860}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6806', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6807', 'nonce': 380388860}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6804', 'nonce': 380388860}, {'type': 'v1', 'addr': '172.21.15.129:6805', 'nonce': 380388860}]}, 'public_addr': '172.21.15.129:6801/380388860', 'cluster_addr': '172.21.15.129:6803/380388860', 'heartbeat_back_addr': '172.21.15.129:6807/380388860', 'heartbeat_front_addr': '172.21.15.129:6805/380388860', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ef9fe11b-b719-48f4-9c6c-e1b1b355acdc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6808', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6809', 'nonce': 943488949}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6810', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6811', 'nonce': 943488949}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6814', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6815', 'nonce': 943488949}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6812', 'nonce': 943488949}, {'type': 'v1', 'addr': '172.21.15.129:6813', 'nonce': 943488949}]}, 'public_addr': '172.21.15.129:6809/943488949', 'cluster_addr': '172.21.15.129:6811/943488949', 'heartbeat_back_addr': '172.21.15.129:6815/943488949', 'heartbeat_front_addr': '172.21.15.129:6813/943488949', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e4045aec-22c1-401b-bfbf-64f115bc32af', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6816', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6817', 'nonce': 1873501077}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6818', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6819', 'nonce': 1873501077}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6822', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6823', 'nonce': 1873501077}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6820', 'nonce': 1873501077}, {'type': 'v1', 'addr': '172.21.15.129:6821', 'nonce': 1873501077}]}, 'public_addr': '172.21.15.129:6817/1873501077', 'cluster_addr': '172.21.15.129:6819/1873501077', 'heartbeat_back_addr': '172.21.15.129:6823/1873501077', 'heartbeat_front_addr': '172.21.15.129:6821/1873501077', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '4a05a5d7-8281-4572-a9d0-6095ee5e168b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 108, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6824', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6825', 'nonce': 3609357666}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6826', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6827', 'nonce': 3609357666}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6830', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6831', 'nonce': 3609357666}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6828', 'nonce': 3609357666}, {'type': 'v1', 'addr': '172.21.15.129:6829', 'nonce': 3609357666}]}, 'public_addr': '172.21.15.129:6825/3609357666', 'cluster_addr': '172.21.15.129:6827/3609357666', 'heartbeat_back_addr': '172.21.15.129:6831/3609357666', 'heartbeat_front_addr': '172.21.15.129:6829/3609357666', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:59.351758+0000', 'dead_epoch': 51}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:59.058915+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.922383+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.802379+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.881681+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.616649+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:58.589555+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T22:40:59.047783+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1c', 'osds': [1, 3]}, {'pgid': '3.1d', 'osds': [1, 6]}], 'pg_upmap_items': [{'pgid': '3.12', 'mappings': [{'from': 3, 'to': 0}]}, {'pgid': '3.16', 'mappings': [{'from': 2, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.12', 'osds': [6, 3]}, {'pgid': '3.16', 'osds': [3, 2]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 10, 'length': 31}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 10, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T22:43:12.690 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:43:12.692 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:12.689+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:12.793 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:12.790+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:12.796 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:12.796 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:12.796 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:43:12.796 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:12.805 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:43:12.894 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:12.891+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:12.907 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:12.908 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:12.908 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:43:12.908 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:43:12.908 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:12.916 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:43:12.929 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:43:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:43:12.968 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:43:12.980 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2024-09-17T22:43:12.329506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":396433072,"num_objects":1063,"num_object_clones":221,"num_object_copies":2126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":352,"num_objects_misplaced":53,"num_objects_unfound":0,"num_objects_dirty":1063,"num_whiteouts":710,"num_read":266209,"num_read_kb":800598,"num_write":17873,"num_write_kb":296016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":14661402,"num_keys_recovered":86,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15848,"ondisk_log_size":15848,"up":71,"acting":73,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":339480,"kb_used_data":23300,"kb_used_omap":12,"kb_used_meta":315955,"kb_avail":754635240,"statfs":{"total":773094113280,"available":772746485760,"internally_reserved":0,"allocated":23859200,"data_stored":101513449,"data_compressed":2616687,"data_compressed_allocated":9027584,"data_compressed_original":94060237,"omap_allocated":12750,"internal_metadata":323538482},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-168661814,"num_objects":-117,"num_object_clones":-34,"num_object_copies":-234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-13,"num_objects_misplaced":-10,"num_objects_unfound":0,"num_objects_dirty":-117,"num_whiteouts":-34,"num_read":1196,"num_read_kb":1010,"num_write":192,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000734"},"pg_stats":[{"pgid":"3.1f","version":"123'829","reported_seq":7103,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-17T22:43:11.161268+0000","last_change":"2024-09-17T22:43:11.161539+0000","last_active":"2024-09-17T22:43:11.161268+0000","last_peered":"2024-09-17T22:43:11.161268+0000","last_clean":"2024-09-17T22:43:11.161268+0000","last_became_active":"2024-09-17T22:41:19.443658+0000","last_became_peered":"2024-09-17T22:41:19.443658+0000","last_unstale":"2024-09-17T22:43:11.161268+0000","last_undegraded":"2024-09-17T22:43:11.161268+0000","last_fullsized":"2024-09-17T22:43:11.161268+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:53:02.843735+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0033354840000000001,"stat_sum":{"num_bytes":1521594,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":16,"num_read":8645,"num_read_kb":14759,"num_write":1082,"num_write_kb":4542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"b","length":"1e"}]},{"pgid":"3.1e","version":"123'777","reported_seq":7310,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-17T22:43:11.160739+0000","last_change":"2024-09-17T22:43:11.160927+0000","last_active":"2024-09-17T22:43:11.160739+0000","last_peered":"2024-09-17T22:43:11.160739+0000","last_clean":"2024-09-17T22:43:11.160739+0000","last_became_active":"2024-09-17T22:41:19.449357+0000","last_became_peered":"2024-09-17T22:41:19.449357+0000","last_unstale":"2024-09-17T22:43:11.160739+0000","last_undegraded":"2024-09-17T22:43:11.160739+0000","last_fullsized":"2024-09-17T22:43:11.160739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:17:44.366965+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0028341180000000001,"stat_sum":{"num_bytes":9206771,"num_objects":29,"num_object_clones":3,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":23,"num_read":8865,"num_read_kb":26537,"num_write":1123,"num_write_kb":10958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"b","length":"1e"}]},{"pgid":"3.1d","version":"119'422","reported_seq":2236,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T22:43:07.083654+0000","last_change":"2024-09-17T22:43:07.085647+0000","last_active":"2024-09-17T22:43:07.083654+0000","last_peered":"2024-09-17T22:43:07.083654+0000","last_clean":"2024-09-17T22:43:07.083654+0000","last_became_active":"2024-09-17T22:42:58.002822+0000","last_became_peered":"2024-09-17T22:42:58.002822+0000","last_unstale":"2024-09-17T22:43:07.083654+0000","last_undegraded":"2024-09-17T22:43:07.083654+0000","last_fullsized":"2024-09-17T22:43:07.083654+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:17:00.357598+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0061459890000000001,"stat_sum":{"num_bytes":26060805,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":14,"num_read":17641,"num_read_kb":48444,"num_write":370,"num_write_kb":8987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":7308630,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"f","length":"1a"}]},{"pgid":"3.1c","version":"119'468","reported_seq":2623,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T22:43:07.106738+0000","last_change":"2024-09-17T22:43:07.085957+0000","last_active":"2024-09-17T22:43:07.106738+0000","last_peered":"2024-09-17T22:43:07.106738+0000","last_clean":"2024-09-17T22:43:07.106738+0000","last_became_active":"2024-09-17T22:42:34.310411+0000","last_became_peered":"2024-09-17T22:42:34.310411+0000","last_unstale":"2024-09-17T22:43:07.106738+0000","last_undegraded":"2024-09-17T22:43:07.106738+0000","last_fullsized":"2024-09-17T22:43:07.106738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:27:44.865052+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0064877670000000002,"stat_sum":{"num_bytes":1420340,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":14,"num_read":2544,"num_read_kb":14144,"num_write":573,"num_write_kb":12172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":7352772,"num_keys_recovered":74,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"f","length":"1a"}]},{"pgid":"3.1b","version":"123'214","reported_seq":7733,"reported_epoch":123,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:43:11.157262+0000","last_change":"2024-09-17T22:41:54.118027+0000","last_active":"2024-09-17T22:43:11.157262+0000","last_peered":"2024-09-17T22:43:11.157262+0000","last_clean":"2024-09-17T22:41:52.368521+0000","last_became_active":"2024-09-17T22:41:54.118027+0000","last_became_peered":"2024-09-17T22:41:54.118027+0000","last_unstale":"2024-09-17T22:43:11.157262+0000","last_undegraded":"2024-09-17T22:41:54.116518+0000","last_fullsized":"2024-09-17T22:41:54.116411+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":30,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:36:56.935435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055813,"stat_sum":{"num_bytes":11094970,"num_objects":42,"num_object_clones":14,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":26,"num_read":7315,"num_read_kb":22822,"num_write":184,"num_write_kb":6215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":42}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"119'327","reported_seq":4924,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T22:43:07.169035+0000","last_change":"2024-09-17T22:43:07.087267+0000","last_active":"2024-09-17T22:43:07.169035+0000","last_peered":"2024-09-17T22:43:07.169035+0000","last_clean":"2024-09-17T22:43:07.169035+0000","last_became_active":"2024-09-17T22:41:19.446541+0000","last_became_peered":"2024-09-17T22:41:19.446541+0000","last_unstale":"2024-09-17T22:43:07.169035+0000","last_undegraded":"2024-09-17T22:43:07.169035+0000","last_fullsized":"2024-09-17T22:43:07.169035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:57:42.698236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073790669999999996,"stat_sum":{"num_bytes":4877750,"num_objects":40,"num_object_clones":3,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":30,"num_read":4837,"num_read_kb":19456,"num_write":307,"num_write_kb":4173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"f","length":"1a"}]},{"pgid":"3.19","version":"123'378","reported_seq":6648,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-17T22:43:11.091666+0000","last_change":"2024-09-17T22:43:11.091920+0000","last_active":"2024-09-17T22:43:11.091666+0000","last_peered":"2024-09-17T22:43:11.091666+0000","last_clean":"2024-09-17T22:43:11.091666+0000","last_became_active":"2024-09-17T22:41:19.449940+0000","last_became_peered":"2024-09-17T22:41:19.449940+0000","last_unstale":"2024-09-17T22:43:11.091666+0000","last_undegraded":"2024-09-17T22:43:11.091666+0000","last_fullsized":"2024-09-17T22:43:11.091666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:57:53.168430+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0038479899999999999,"stat_sum":{"num_bytes":14059650,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":21,"num_read":6664,"num_read_kb":22141,"num_write":342,"num_write_kb":9035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"1e"}]},{"pgid":"3.18","version":"123'375","reported_seq":3110,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-17T22:43:11.163910+0000","last_change":"2024-09-17T22:43:11.164155+0000","last_active":"2024-09-17T22:43:11.163910+0000","last_peered":"2024-09-17T22:43:11.163910+0000","last_clean":"2024-09-17T22:43:11.163910+0000","last_became_active":"2024-09-17T22:41:19.446287+0000","last_became_peered":"2024-09-17T22:41:19.446287+0000","last_unstale":"2024-09-17T22:43:11.163910+0000","last_undegraded":"2024-09-17T22:43:11.163910+0000","last_fullsized":"2024-09-17T22:43:11.163910+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:37:32.523221+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0052167309999999996,"stat_sum":{"num_bytes":5522111,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":18,"num_read":3441,"num_read_kb":20083,"num_write":319,"num_write_kb":10930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"b","length":"1e"}]},{"pgid":"3.17","version":"121'498","reported_seq":2635,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-17T22:43:11.160968+0000","last_change":"2024-09-17T22:43:11.161333+0000","last_active":"2024-09-17T22:43:11.160968+0000","last_peered":"2024-09-17T22:43:11.160968+0000","last_clean":"2024-09-17T22:43:11.160968+0000","last_became_active":"2024-09-17T22:41:19.445923+0000","last_became_peered":"2024-09-17T22:41:19.445923+0000","last_unstale":"2024-09-17T22:43:11.160968+0000","last_undegraded":"2024-09-17T22:43:11.160968+0000","last_fullsized":"2024-09-17T22:43:11.160968+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:48:14.198590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028336559999999999,"stat_sum":{"num_bytes":5583325,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":21,"num_read":3445,"num_read_kb":15105,"num_write":510,"num_write_kb":8714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"b","length":"1e"}]},{"pgid":"3.16","version":"121'521","reported_seq":22196,"reported_epoch":122,"state":"active+clean+remapped","last_fresh":"2024-09-17T22:43:10.102555+0000","last_change":"2024-09-17T22:43:10.103116+0000","last_active":"2024-09-17T22:43:10.102555+0000","last_peered":"2024-09-17T22:43:10.102555+0000","last_clean":"2024-09-17T22:43:10.102555+0000","last_became_active":"2024-09-17T22:41:57.132018+0000","last_became_peered":"2024-09-17T22:41:57.132018+0000","last_unstale":"2024-09-17T22:43:10.102555+0000","last_undegraded":"2024-09-17T22:43:10.102555+0000","last_fullsized":"2024-09-17T22:43:10.102555+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:39:12.904318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001115397,"stat_sum":{"num_bytes":12463051,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":35,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":23,"num_read":22661,"num_read_kb":63861,"num_write":585,"num_write_kb":12646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":35}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"1d"}]},{"pgid":"3.15","version":"119'412","reported_seq":7301,"reported_epoch":123,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:43:11.097267+0000","last_change":"2024-09-17T22:41:54.119655+0000","last_active":"2024-09-17T22:43:11.097267+0000","last_peered":"2024-09-17T22:43:11.097267+0000","last_clean":"2024-09-17T22:41:52.245886+0000","last_became_active":"2024-09-17T22:41:54.119655+0000","last_became_peered":"2024-09-17T22:41:54.119655+0000","last_unstale":"2024-09-17T22:43:11.097267+0000","last_undegraded":"2024-09-17T22:41:54.119161+0000","last_fullsized":"2024-09-17T22:41:54.119071+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":30,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:24:58.547950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041055799999999999,"stat_sum":{"num_bytes":61057766,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":28,"num_read":8763,"num_read_kb":20352,"num_write":340,"num_write_kb":4822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":42}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"121'578","reported_seq":7734,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T22:43:10.103014+0000","last_change":"2024-09-17T22:43:10.103530+0000","last_active":"2024-09-17T22:43:10.103014+0000","last_peered":"2024-09-17T22:43:10.103014+0000","last_clean":"2024-09-17T22:43:10.103014+0000","last_became_active":"2024-09-17T22:41:19.451631+0000","last_became_peered":"2024-09-17T22:41:19.451631+0000","last_unstale":"2024-09-17T22:43:10.103014+0000","last_undegraded":"2024-09-17T22:43:10.103014+0000","last_fullsized":"2024-09-17T22:43:10.103014+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:38:39.128135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001422022,"stat_sum":{"num_bytes":1561851,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":14,"num_read":8318,"num_read_kb":19441,"num_write":733,"num_write_kb":19536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"1d"}]},{"pgid":"2.7","version":"0'0","reported_seq":223,"reported_epoch":122,"state":"active+undersized","last_fresh":"2024-09-17T22:43:10.101894+0000","last_change":"2024-09-17T22:41:54.118987+0000","last_active":"2024-09-17T22:43:10.101894+0000","last_peered":"2024-09-17T22:43:10.101894+0000","last_clean":"2024-09-17T22:41:52.098057+0000","last_became_active":"2024-09-17T22:41:54.118987+0000","last_became_peered":"2024-09-17T22:41:54.118987+0000","last_unstale":"2024-09-17T22:43:10.101894+0000","last_undegraded":"2024-09-17T22:43:10.101894+0000","last_fullsized":"2024-09-17T22:41:54.114557+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.941682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"123'546","reported_seq":5200,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-17T22:43:11.098910+0000","last_change":"2024-09-17T22:43:11.098910+0000","last_active":"2024-09-17T22:43:11.098910+0000","last_peered":"2024-09-17T22:43:11.098910+0000","last_clean":"2024-09-17T22:43:11.098910+0000","last_became_active":"2024-09-17T22:41:19.448966+0000","last_became_peered":"2024-09-17T22:41:19.448966+0000","last_unstale":"2024-09-17T22:43:11.098910+0000","last_undegraded":"2024-09-17T22:43:11.098910+0000","last_fullsized":"2024-09-17T22:43:11.098910+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:41:48.413000+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.010618435000000001,"stat_sum":{"num_bytes":3300625,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":15,"num_read":6227,"num_read_kb":29896,"num_write":522,"num_write_kb":6238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"1e"}]},{"pgid":"2.6","version":"0'0","reported_seq":224,"reported_epoch":123,"state":"active+undersized","last_fresh":"2024-09-17T22:43:11.088244+0000","last_change":"2024-09-17T22:41:54.118378+0000","last_active":"2024-09-17T22:43:11.088244+0000","last_peered":"2024-09-17T22:43:11.088244+0000","last_clean":"2024-09-17T22:41:52.097696+0000","last_became_active":"2024-09-17T22:41:54.118378+0000","last_became_peered":"2024-09-17T22:41:54.118378+0000","last_unstale":"2024-09-17T22:43:11.088244+0000","last_undegraded":"2024-09-17T22:43:11.088244+0000","last_fullsized":"2024-09-17T22:41:54.116433+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:48:40.138581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"123'367","reported_seq":1640,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-17T22:43:11.092341+0000","last_change":"2024-09-17T22:43:11.092341+0000","last_active":"2024-09-17T22:43:11.092341+0000","last_peered":"2024-09-17T22:43:11.092341+0000","last_clean":"2024-09-17T22:43:11.092341+0000","last_became_active":"2024-09-17T22:41:19.451637+0000","last_became_peered":"2024-09-17T22:41:19.451637+0000","last_unstale":"2024-09-17T22:43:11.092341+0000","last_undegraded":"2024-09-17T22:43:11.092341+0000","last_fullsized":"2024-09-17T22:43:11.092341+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:14:52.940676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040516069999999996,"stat_sum":{"num_bytes":12566418,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":24,"num_read":1443,"num_read_kb":9951,"num_write":393,"num_write_kb":7019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"1e"}]},{"pgid":"2.5","version":"0'0","reported_seq":221,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T22:43:10.101830+0000","last_change":"2024-09-17T22:41:11.397753+0000","last_active":"2024-09-17T22:43:10.101830+0000","last_peered":"2024-09-17T22:43:10.101830+0000","last_clean":"2024-09-17T22:43:10.101830+0000","last_became_active":"2024-09-17T22:41:09.387651+0000","last_became_peered":"2024-09-17T22:41:09.387651+0000","last_unstale":"2024-09-17T22:43:10.101830+0000","last_undegraded":"2024-09-17T22:43:10.101830+0000","last_fullsized":"2024-09-17T22:43:10.101830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:14:30.915047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"116'467","reported_seq":1319,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-17T22:43:11.096700+0000","last_change":"2024-09-17T22:43:11.097428+0000","last_active":"2024-09-17T22:43:11.096700+0000","last_peered":"2024-09-17T22:43:11.096700+0000","last_clean":"2024-09-17T22:43:11.096700+0000","last_became_active":"2024-09-17T22:41:19.444893+0000","last_became_peered":"2024-09-17T22:41:19.444893+0000","last_unstale":"2024-09-17T22:43:11.096700+0000","last_undegraded":"2024-09-17T22:43:11.096700+0000","last_fullsized":"2024-09-17T22:43:11.096700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:26:09.341145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089176200000000005,"stat_sum":{"num_bytes":5475472,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":19,"num_read":1718,"num_read_kb":9540,"num_write":630,"num_write_kb":7900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"b","length":"1e"}]},{"pgid":"2.4","version":"0'0","reported_seq":222,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-17T22:43:11.158290+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:43:11.158290+0000","last_peered":"2024-09-17T22:43:11.158290+0000","last_clean":"2024-09-17T22:43:11.158290+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:43:11.158290+0000","last_undegraded":"2024-09-17T22:43:11.158290+0000","last_fullsized":"2024-09-17T22:43:11.158290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"122'358","reported_seq":15461,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T22:43:10.180497+0000","last_change":"2024-09-17T22:43:10.080255+0000","last_active":"2024-09-17T22:43:10.180497+0000","last_peered":"2024-09-17T22:43:10.180497+0000","last_clean":"2024-09-17T22:43:10.180497+0000","last_became_active":"2024-09-17T22:41:19.451723+0000","last_became_peered":"2024-09-17T22:41:19.451723+0000","last_unstale":"2024-09-17T22:43:10.180497+0000","last_undegraded":"2024-09-17T22:43:10.180497+0000","last_fullsized":"2024-09-17T22:43:10.180497+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:57:23.805345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063470099999999999,"stat_sum":{"num_bytes":18591974,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":13,"num_read":15314,"num_read_kb":33614,"num_write":400,"num_write_kb":8744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"c","length":"1d"}]},{"pgid":"3.3","version":"122'755","reported_seq":11487,"reported_epoch":122,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:43:10.169412+0000","last_change":"2024-09-17T22:41:54.119266+0000","last_active":"2024-09-17T22:43:10.169412+0000","last_peered":"2024-09-17T22:43:10.169412+0000","last_clean":"2024-09-17T22:41:52.262287+0000","last_became_active":"2024-09-17T22:41:54.119266+0000","last_became_peered":"2024-09-17T22:41:54.119266+0000","last_unstale":"2024-09-17T22:43:10.169412+0000","last_undegraded":"2024-09-17T22:41:54.116326+0000","last_fullsized":"2024-09-17T22:41:54.116242+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":29,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:33:53.613631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063898899999999996,"stat_sum":{"num_bytes":33098273,"num_objects":72,"num_object_clones":21,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":72,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":46,"num_read":14895,"num_read_kb":33312,"num_write":784,"num_write_kb":8667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":72}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":76,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-17T22:43:11.096487+0000","last_change":"2024-09-17T22:42:48.420232+0000","last_active":"2024-09-17T22:43:11.096487+0000","last_peered":"2024-09-17T22:43:11.096487+0000","last_clean":"2024-09-17T22:43:11.096487+0000","last_became_active":"2024-09-17T22:42:48.419946+0000","last_became_peered":"2024-09-17T22:42:48.419946+0000","last_unstale":"2024-09-17T22:43:11.096487+0000","last_undegraded":"2024-09-17T22:43:11.096487+0000","last_fullsized":"2024-09-17T22:43:11.096487+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:11:29.985673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"122'360","reported_seq":4056,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-17T22:43:11.161382+0000","last_change":"2024-09-17T22:43:11.161904+0000","last_active":"2024-09-17T22:43:11.161382+0000","last_peered":"2024-09-17T22:43:11.161382+0000","last_clean":"2024-09-17T22:43:11.161382+0000","last_became_active":"2024-09-17T22:41:19.444314+0000","last_became_peered":"2024-09-17T22:41:19.444314+0000","last_unstale":"2024-09-17T22:43:11.161382+0000","last_undegraded":"2024-09-17T22:43:11.161382+0000","last_fullsized":"2024-09-17T22:43:11.161382+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:52:35.530707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030721350000000001,"stat_sum":{"num_bytes":14740320,"num_objects":31,"num_object_clones":4,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":21,"num_read":3972,"num_read_kb":22329,"num_write":339,"num_write_kb":9348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"b","length":"1e"}]},{"pgid":"2.1","version":"0'0","reported_seq":223,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-17T22:43:11.094742+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:43:11.094742+0000","last_peered":"2024-09-17T22:43:11.094742+0000","last_clean":"2024-09-17T22:43:11.094742+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:43:11.094742+0000","last_undegraded":"2024-09-17T22:43:11.094742+0000","last_fullsized":"2024-09-17T22:43:11.094742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"123'470","reported_seq":7341,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-17T22:43:11.105776+0000","last_change":"2024-09-17T22:43:11.106268+0000","last_active":"2024-09-17T22:43:11.105776+0000","last_peered":"2024-09-17T22:43:11.105776+0000","last_clean":"2024-09-17T22:43:11.105776+0000","last_became_active":"2024-09-17T22:41:19.449791+0000","last_became_peered":"2024-09-17T22:41:19.449791+0000","last_unstale":"2024-09-17T22:43:11.105776+0000","last_undegraded":"2024-09-17T22:43:11.105776+0000","last_fullsized":"2024-09-17T22:43:11.105776+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:18.612821+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0097886139999999993,"stat_sum":{"num_bytes":5317155,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":14,"num_read":8090,"num_read_kb":9246,"num_write":514,"num_write_kb":4605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"b","length":"1e"}]},{"pgid":"2.0","version":"0'0","reported_seq":222,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T22:43:10.102270+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:43:10.102270+0000","last_peered":"2024-09-17T22:43:10.102270+0000","last_clean":"2024-09-17T22:43:10.102270+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:43:10.102270+0000","last_undegraded":"2024-09-17T22:43:10.102270+0000","last_fullsized":"2024-09-17T22:43:10.102270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"123'737","reported_seq":8618,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-17T22:43:11.094886+0000","last_change":"2024-09-17T22:43:11.094886+0000","last_active":"2024-09-17T22:43:11.094886+0000","last_peered":"2024-09-17T22:43:11.094886+0000","last_clean":"2024-09-17T22:43:11.094886+0000","last_became_active":"2024-09-17T22:41:19.449279+0000","last_became_peered":"2024-09-17T22:41:19.449279+0000","last_unstale":"2024-09-17T22:43:11.094886+0000","last_undegraded":"2024-09-17T22:43:11.094886+0000","last_fullsized":"2024-09-17T22:43:11.094886+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:26.301355+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0067761239999999997,"stat_sum":{"num_bytes":5530973,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":22,"num_read":11288,"num_read_kb":22946,"num_write":953,"num_write_kb":2860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"1e"}]},{"pgid":"2.3","version":"19'1","reported_seq":61,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T22:43:10.079541+0000","last_change":"2024-09-17T22:42:48.420535+0000","last_active":"2024-09-17T22:43:10.079541+0000","last_peered":"2024-09-17T22:43:10.079541+0000","last_clean":"2024-09-17T22:43:10.079541+0000","last_became_active":"2024-09-17T22:42:48.420210+0000","last_became_peered":"2024-09-17T22:42:48.420210+0000","last_unstale":"2024-09-17T22:43:10.079541+0000","last_undegraded":"2024-09-17T22:43:10.079541+0000","last_fullsized":"2024-09-17T22:43:10.079541+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:36.249795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":367,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T22:43:07.079791+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:43:07.079791+0000","last_peered":"2024-09-17T22:43:07.079791+0000","last_clean":"2024-09-17T22:43:07.079791+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:43:07.079791+0000","last_undegraded":"2024-09-17T22:43:07.079791+0000","last_fullsized":"2024-09-17T22:43:07.079791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"120'604","reported_seq":11837,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-17T22:43:11.088358+0000","last_change":"2024-09-17T22:43:11.088786+0000","last_active":"2024-09-17T22:43:11.088358+0000","last_peered":"2024-09-17T22:43:11.088358+0000","last_clean":"2024-09-17T22:43:11.088358+0000","last_became_active":"2024-09-17T22:41:19.448956+0000","last_became_peered":"2024-09-17T22:41:19.448956+0000","last_unstale":"2024-09-17T22:43:11.088358+0000","last_undegraded":"2024-09-17T22:43:11.088358+0000","last_fullsized":"2024-09-17T22:43:11.088358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:57:00.565384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070585199999999998,"stat_sum":{"num_bytes":2437136,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":19,"num_read":13249,"num_read_kb":46910,"num_write":652,"num_write_kb":25175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"1e"}]},{"pgid":"3.9","version":"121'754","reported_seq":14618,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T22:43:10.103176+0000","last_change":"2024-09-17T22:43:10.103746+0000","last_active":"2024-09-17T22:43:10.103176+0000","last_peered":"2024-09-17T22:43:10.103176+0000","last_clean":"2024-09-17T22:43:10.103176+0000","last_became_active":"2024-09-17T22:41:19.449003+0000","last_became_peered":"2024-09-17T22:41:19.449003+0000","last_unstale":"2024-09-17T22:43:10.103176+0000","last_undegraded":"2024-09-17T22:43:10.103176+0000","last_fullsized":"2024-09-17T22:43:10.103176+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:52:30.257730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016161350000000001,"stat_sum":{"num_bytes":9902556,"num_objects":30,"num_object_clones":9,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":17,"num_read":16204,"num_read_kb":43229,"num_write":1028,"num_write_kb":9843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"1d"}]},{"pgid":"3.a","version":"119'430","reported_seq":8340,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T22:43:07.091601+0000","last_change":"2024-09-17T22:43:07.091601+0000","last_active":"2024-09-17T22:43:07.091601+0000","last_peered":"2024-09-17T22:43:07.091601+0000","last_clean":"2024-09-17T22:43:07.091601+0000","last_became_active":"2024-09-17T22:42:48.419427+0000","last_became_peered":"2024-09-17T22:42:48.419427+0000","last_unstale":"2024-09-17T22:43:07.091601+0000","last_undegraded":"2024-09-17T22:43:07.091601+0000","last_fullsized":"2024-09-17T22:43:07.091601+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:31.574560+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011791902,"stat_sum":{"num_bytes":19462291,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":23,"num_read":8165,"num_read_kb":41060,"num_write":450,"num_write_kb":9769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"f","length":"1a"}]},{"pgid":"3.b","version":"123'412","reported_seq":2623,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-17T22:43:11.182449+0000","last_change":"2024-09-17T22:43:11.098450+0000","last_active":"2024-09-17T22:43:11.182449+0000","last_peered":"2024-09-17T22:43:11.182449+0000","last_clean":"2024-09-17T22:43:11.182449+0000","last_became_active":"2024-09-17T22:41:19.447215+0000","last_became_peered":"2024-09-17T22:41:19.447215+0000","last_unstale":"2024-09-17T22:43:11.182449+0000","last_undegraded":"2024-09-17T22:43:11.182449+0000","last_fullsized":"2024-09-17T22:43:11.182449+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:04:19.970519+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0036130670000000002,"stat_sum":{"num_bytes":5355981,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":18,"num_read":3409,"num_read_kb":14480,"num_write":401,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"b","length":"1e"}]},{"pgid":"3.c","version":"120'306","reported_seq":21259,"reported_epoch":123,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:43:11.094947+0000","last_change":"2024-09-17T22:41:54.119555+0000","last_active":"2024-09-17T22:43:11.094947+0000","last_peered":"2024-09-17T22:43:11.094947+0000","last_clean":"2024-09-17T22:41:52.249181+0000","last_became_active":"2024-09-17T22:41:54.119555+0000","last_became_peered":"2024-09-17T22:41:54.119555+0000","last_unstale":"2024-09-17T22:43:11.094947+0000","last_undegraded":"2024-09-17T22:41:54.118897+0000","last_fullsized":"2024-09-17T22:41:54.118785+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":30,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:42:13.143028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000395712,"stat_sum":{"num_bytes":43758350,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":46,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":31,"num_read":21316,"num_read_kb":63555,"num_write":320,"num_write_kb":14934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":46}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"119'473","reported_seq":1924,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T22:43:07.086518+0000","last_change":"2024-09-17T22:43:07.086518+0000","last_active":"2024-09-17T22:43:07.086518+0000","last_peered":"2024-09-17T22:43:07.086518+0000","last_clean":"2024-09-17T22:43:07.086518+0000","last_became_active":"2024-09-17T22:42:48.419360+0000","last_became_peered":"2024-09-17T22:42:48.419360+0000","last_unstale":"2024-09-17T22:43:07.086518+0000","last_undegraded":"2024-09-17T22:43:07.086518+0000","last_fullsized":"2024-09-17T22:43:07.086518+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:08:00.400907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067925729999999997,"stat_sum":{"num_bytes":1384296,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":19,"num_read":2214,"num_read_kb":10203,"num_write":571,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"f","length":"1a"}]},{"pgid":"3.e","version":"122'324","reported_seq":8406,"reported_epoch":122,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:43:10.182138+0000","last_change":"2024-09-17T22:41:54.119016+0000","last_active":"2024-09-17T22:43:10.182138+0000","last_peered":"2024-09-17T22:43:10.182138+0000","last_clean":"2024-09-17T22:41:52.398679+0000","last_became_active":"2024-09-17T22:41:54.119016+0000","last_became_peered":"2024-09-17T22:41:54.119016+0000","last_unstale":"2024-09-17T22:43:10.182138+0000","last_undegraded":"2024-09-17T22:41:54.116346+0000","last_fullsized":"2024-09-17T22:41:54.116297+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":29,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:32:03.547496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038538199999999998,"stat_sum":{"num_bytes":8776281,"num_objects":48,"num_object_clones":15,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":31,"num_read":8704,"num_read_kb":23017,"num_write":279,"num_write_kb":11332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":48}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"117'318","reported_seq":1518,"reported_epoch":119,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:43:07.079488+0000","last_change":"2024-09-17T22:41:54.113116+0000","last_active":"2024-09-17T22:43:07.079488+0000","last_peered":"2024-09-17T22:43:07.079488+0000","last_clean":"2024-09-17T22:41:52.346319+0000","last_became_active":"2024-09-17T22:41:54.113116+0000","last_became_peered":"2024-09-17T22:41:54.113116+0000","last_unstale":"2024-09-17T22:43:07.079488+0000","last_undegraded":"2024-09-17T22:41:54.112692+0000","last_fullsized":"2024-09-17T22:41:54.112614+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":26,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:04:13.156418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000602379,"stat_sum":{"num_bytes":21963964,"num_objects":56,"num_object_clones":15,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":56,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":36,"num_read":1627,"num_read_kb":5498,"num_write":375,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":56}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"118'296","reported_seq":1844,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-17T22:43:07.085761+0000","last_change":"2024-09-17T22:43:07.086296+0000","last_active":"2024-09-17T22:43:07.085761+0000","last_peered":"2024-09-17T22:43:07.085761+0000","last_clean":"2024-09-17T22:43:07.085761+0000","last_became_active":"2024-09-17T22:41:19.447419+0000","last_became_peered":"2024-09-17T22:41:19.447419+0000","last_unstale":"2024-09-17T22:43:07.085761+0000","last_undegraded":"2024-09-17T22:43:07.085761+0000","last_fullsized":"2024-09-17T22:43:07.085761+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:47:05.150842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006599237,"stat_sum":{"num_bytes":2869096,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":21,"num_read":2774,"num_read_kb":8197,"num_write":250,"num_write_kb":3149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"f","length":"1a"}]},{"pgid":"3.11","version":"121'591","reported_seq":3548,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T22:43:10.102926+0000","last_change":"2024-09-17T22:43:10.102926+0000","last_active":"2024-09-17T22:43:10.102926+0000","last_peered":"2024-09-17T22:43:10.102926+0000","last_clean":"2024-09-17T22:43:10.102926+0000","last_became_active":"2024-09-17T22:41:19.454151+0000","last_became_peered":"2024-09-17T22:41:19.454151+0000","last_unstale":"2024-09-17T22:43:10.102926+0000","last_undegraded":"2024-09-17T22:43:10.102926+0000","last_fullsized":"2024-09-17T22:43:10.102926+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:22:31.413783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042035,"stat_sum":{"num_bytes":8766273,"num_objects":40,"num_object_clones":12,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":27,"num_read":5264,"num_read_kb":13545,"num_write":657,"num_write_kb":3731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"1d"}]},{"pgid":"3.12","version":"123'904","reported_seq":11426,"reported_epoch":123,"state":"active+clean+remapped","last_fresh":"2024-09-17T22:43:11.098938+0000","last_change":"2024-09-17T22:43:11.098938+0000","last_active":"2024-09-17T22:43:11.098938+0000","last_peered":"2024-09-17T22:43:11.098938+0000","last_clean":"2024-09-17T22:43:11.098938+0000","last_became_active":"2024-09-17T22:41:57.131666+0000","last_became_peered":"2024-09-17T22:41:57.131666+0000","last_unstale":"2024-09-17T22:43:11.098938+0000","last_undegraded":"2024-09-17T22:43:11.098938+0000","last_fullsized":"2024-09-17T22:43:11.098938+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":904,"log_dups_size":0,"ondisk_log_size":904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:03.146683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021440930000000001,"stat_sum":{"num_bytes":4374576,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":18,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":13,"num_read":14640,"num_read_kb":44073,"num_write":1138,"num_write_kb":22161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6,3],"avail_no_missing":["6","3"],"object_location_counts":[{"shards":"3,6","objects":18}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"b","length":"1e"}]},{"pgid":"3.13","version":"122'457","reported_seq":2245,"reported_epoch":122,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:43:10.102184+0000","last_change":"2024-09-17T22:41:54.118972+0000","last_active":"2024-09-17T22:43:10.102184+0000","last_peered":"2024-09-17T22:43:10.102184+0000","last_clean":"2024-09-17T22:41:52.267782+0000","last_became_active":"2024-09-17T22:41:54.118972+0000","last_became_peered":"2024-09-17T22:41:54.118972+0000","last_unstale":"2024-09-17T22:43:10.102184+0000","last_undegraded":"2024-09-17T22:41:54.114427+0000","last_fullsized":"2024-09-17T22:41:54.114366+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":29,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:12.098876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062146200000000003,"stat_sum":{"num_bytes":13740691,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":46,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":33,"num_read":2441,"num_read_kb":8752,"num_write":588,"num_write_kb":8374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":46}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":395842685,"num_objects":1059,"num_object_clones":221,"num_object_copies":2118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":352,"num_objects_misplaced":53,"num_objects_unfound":0,"num_objects_dirty":1059,"num_whiteouts":710,"num_read":266093,"num_read_kb":800498,"num_write":17712,"num_write_kb":293598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":14661402,"num_keys_recovered":86,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12804096,"data_stored":92744724,"data_compressed":2555459,"data_compressed_allocated":8761344,"data_compressed_original":90455757,"omap_allocated":0,"internal_metadata":0},"log_size":15728,"ondisk_log_size":15728,"up":55,"acting":57,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":14,"acting":14,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30164,"kb_used_data":3132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341676,"statfs":{"total":96636764160,"available":96605876224,"internally_reserved":0,"allocated":3207168,"data_stored":15349007,"data_compressed":370614,"data_compressed_allocated":1310720,"data_compressed_original":14401536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542156,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34156,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337684,"statfs":{"total":96636764160,"available":96601788416,"internally_reserved":0,"allocated":3043328,"data_stored":23926169,"data_compressed":437861,"data_compressed_allocated":1896448,"data_compressed_original":23315877,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542172,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41716,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330124,"statfs":{"total":96636764160,"available":96594046976,"internally_reserved":0,"allocated":2256896,"data_stored":5217183,"data_compressed":161714,"data_compressed_allocated":491520,"data_compressed_original":4347428,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30276,"kb_used_data":3244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341564,"statfs":{"total":96636764160,"available":96605761536,"internally_reserved":0,"allocated":3321856,"data_stored":14260611,"data_compressed":476314,"data_compressed_allocated":1331200,"data_compressed_original":13273892,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542172,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47764,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324076,"statfs":{"total":96636764160,"available":96587853824,"internally_reserved":0,"allocated":2744320,"data_stored":10815990,"data_compressed":182154,"data_compressed_allocated":884736,"data_compressed_original":9897108,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542172,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46340,"kb_used_data":2660,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325500,"statfs":{"total":96636764160,"available":96589312000,"internally_reserved":0,"allocated":2723840,"data_stored":8346086,"data_compressed":274966,"data_compressed_allocated":823296,"data_compressed_original":7401472,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56384,"kb_used_data":2988,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315456,"statfs":{"total":96636764160,"available":96579026944,"internally_reserved":0,"allocated":3059712,"data_stored":8658015,"data_compressed":296605,"data_compressed_allocated":933888,"data_compressed_original":7578444,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52680,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319160,"statfs":{"total":96636764160,"available":96582819840,"internally_reserved":0,"allocated":3502080,"data_stored":14940388,"data_compressed":416459,"data_compressed_allocated":1355776,"data_compressed_original":13844480,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2514944,"data_stored":23924031,"data_compressed":438892,"data_compressed_allocated":1921024,"data_compressed_original":23549349,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1515520,"data_stored":11395855,"data_compressed":323607,"data_compressed_allocated":1069056,"data_compressed_original":11153408,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2056192,"data_stored":14248629,"data_compressed":416459,"data_compressed_allocated":1355776,"data_compressed_original":13844480,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":7946049,"data_compressed":286331,"data_compressed_allocated":921600,"data_compressed_original":7562060,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1257472,"data_stored":7640985,"data_compressed":274966,"data_compressed_allocated":823296,"data_compressed_original":7401472,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1228800,"data_stored":9526338,"data_compressed":177176,"data_compressed_allocated":847872,"data_compressed_original":9323668,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1843200,"data_stored":13550755,"data_compressed":476314,"data_compressed_allocated":1331200,"data_compressed_original":13273892,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":790528,"data_stored":4512082,"data_compressed":161714,"data_compressed_allocated":491520,"data_compressed_original":4347428,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:43:12.982 INFO:tasks.thrashosds.thrasher:Setting 2.5 to [4, 7, 6, 1] 2024-09-17T22:43:12.982 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.5', '4', '7', '6', '1'] 2024-09-17T22:43:12.982 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.5 4 7 6 1 2024-09-17T22:43:13.021 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:13.021 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:43:13.021 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:43:13.021 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:43:13.021 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.021 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202977 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:13.022 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:43:13.022 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.292210935999996, 2024-09-17T22:43:13.022 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023068750999999998, 2024-09-17T22:43:13.022 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.022 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.022 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:13.022 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:13.022 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:13.022 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:13.022 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202977 2024-09-17T22:43:13.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.022 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.022 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.022 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.022 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:43:13.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:43:13.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.023 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725980+0000", 2024-09-17T22:43:13.023 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:43:13.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.023 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.023 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725982+0000", 2024-09-17T22:43:13.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0669999999999999e-06 2024-09-17T22:43:13.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725988+0000", 2024-09-17T22:43:13.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6610000000000003e-06 2024-09-17T22:43:13.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.024 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.024 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.024 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725993+0000", 2024-09-17T22:43:13.024 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:43:13.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748888+0000", 2024-09-17T22:43:13.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022894930000000001 2024-09-17T22:43:13.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748951+0000", 2024-09-17T22:43:13.025 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3638000000000003e-05 2024-09-17T22:43:13.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.025 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.025 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.025 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749048+0000", 2024-09-17T22:43:13.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6343999999999998e-05 2024-09-17T22:43:13.026 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.026 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.026 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.026 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202978 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2211840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:13.026 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:43:13.026 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.292190423999998, 2024-09-17T22:43:13.026 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023238050999999999, 2024-09-17T22:43:13.026 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.026 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.026 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:13.026 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:13.026 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:13.027 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:13.027 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202978 2024-09-17T22:43:13.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.027 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:43:13.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.027 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.027 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.027 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:43:13.027 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:43:13.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.58e-07 2024-09-17T22:43:13.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.028 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.028 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726002+0000", 2024-09-17T22:43:13.028 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.193e-06 2024-09-17T22:43:13.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.028 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726009+0000", 2024-09-17T22:43:13.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8000000000000001e-06 2024-09-17T22:43:13.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726014+0000", 2024-09-17T22:43:13.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9840000000000001e-06 2024-09-17T22:43:13.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.029 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.029 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.029 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749090+0000", 2024-09-17T22:43:13.029 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023075917000000001 2024-09-17T22:43:13.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749163+0000", 2024-09-17T22:43:13.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3219999999999994e-05 2024-09-17T22:43:13.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.030 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.030 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749238+0000", 2024-09-17T22:43:13.030 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4479000000000003e-05 2024-09-17T22:43:13.030 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.030 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.030 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.030 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.031 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202979 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2342912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:13.031 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:43:13.031 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.292168973000003, 2024-09-17T22:43:13.031 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023393536999999999, 2024-09-17T22:43:13.031 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.031 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.031 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:13.031 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:13.031 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:13.031 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:13.031 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202979 2024-09-17T22:43:13.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.031 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.031 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:43:13.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:43:13.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.032 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.032 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726022+0000", 2024-09-17T22:43:13.032 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:43:13.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726024+0000", 2024-09-17T22:43:13.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1969999999999999e-06 2024-09-17T22:43:13.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.033 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726030+0000", 2024-09-17T22:43:13.033 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.691e-06 2024-09-17T22:43:13.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.033 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.033 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726035+0000", 2024-09-17T22:43:13.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8029999999999996e-06 2024-09-17T22:43:13.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749280+0000", 2024-09-17T22:43:13.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023244668 2024-09-17T22:43:13.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.034 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.034 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.034 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749344+0000", 2024-09-17T22:43:13.034 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4418000000000006e-05 2024-09-17T22:43:13.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.035 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.035 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749415+0000", 2024-09-17T22:43:13.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0272999999999996e-05 2024-09-17T22:43:13.035 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.035 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.035 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.035 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.035 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202980 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2473984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:13.035 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:43:13.035 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.292148204, 2024-09-17T22:43:13.035 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023562221000000001, 2024-09-17T22:43:13.036 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.036 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.036 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:13.036 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:13.036 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:13.036 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:13.036 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202980 2024-09-17T22:43:13.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.036 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.036 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.036 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.036 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:43:13.036 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.036 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:43:13.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:43:13.037 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:43:13.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.037 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.037 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.037 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726044+0000", 2024-09-17T22:43:13.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9489999999999999e-06 2024-09-17T22:43:13.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.038 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.038 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.038 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726066+0000", 2024-09-17T22:43:13.038 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1508000000000001e-05 2024-09-17T22:43:13.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.040 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.040 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726072+0000", 2024-09-17T22:43:13.040 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3450000000000002e-06 2024-09-17T22:43:13.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749455+0000", 2024-09-17T22:43:13.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023382824999999999 2024-09-17T22:43:13.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749527+0000", 2024-09-17T22:43:13.041 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1728000000000002e-05 2024-09-17T22:43:13.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.041 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.041 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.041 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749604+0000", 2024-09-17T22:43:13.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7428999999999993e-05 2024-09-17T22:43:13.042 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.042 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.042 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.042 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.042 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202981 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:13.042 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:43:13.042 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.292109928999999, 2024-09-17T22:43:13.042 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023677800999999998, 2024-09-17T22:43:13.042 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.042 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.042 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:13.042 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:13.043 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:13.043 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:13.043 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202981 2024-09-17T22:43:13.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.043 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.043 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:43:13.043 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.043 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.043 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:43:13.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726081+0000", 2024-09-17T22:43:13.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5499999999999998e-07 2024-09-17T22:43:13.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.044 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.044 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.044 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726083+0000", 2024-09-17T22:43:13.044 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1780000000000002e-06 2024-09-17T22:43:13.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726090+0000", 2024-09-17T22:43:13.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8510000000000001e-06 2024-09-17T22:43:13.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726101+0000", 2024-09-17T22:43:13.045 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.133e-05 2024-09-17T22:43:13.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.045 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.045 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749639+0000", 2024-09-17T22:43:13.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023537679999999998 2024-09-17T22:43:13.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749688+0000", 2024-09-17T22:43:13.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9548000000000001e-05 2024-09-17T22:43:13.046 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.046 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.046 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.046 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749758+0000", 2024-09-17T22:43:13.046 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9758999999999994e-05 2024-09-17T22:43:13.046 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.046 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.047 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.047 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.047 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202982 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:13.047 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:43:13.047 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.292082346000001, 2024-09-17T22:43:13.047 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023841469000000001, 2024-09-17T22:43:13.047 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.047 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.047 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:13.047 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:13.047 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:13.047 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:13.048 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202982 2024-09-17T22:43:13.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.048 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:43:13.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.048 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.048 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:43:13.048 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.048 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:43:13.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:43:13.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726110+0000", 2024-09-17T22:43:13.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.902e-06 2024-09-17T22:43:13.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.049 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.049 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.049 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726117+0000", 2024-09-17T22:43:13.049 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6189999999999996e-06 2024-09-17T22:43:13.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726122+0000", 2024-09-17T22:43:13.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.198e-06 2024-09-17T22:43:13.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749795+0000", 2024-09-17T22:43:13.050 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023673397999999998 2024-09-17T22:43:13.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.050 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.050 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749877+0000", 2024-09-17T22:43:13.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1910999999999996e-05 2024-09-17T22:43:13.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.051 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.051 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749949+0000", 2024-09-17T22:43:13.051 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2001000000000004e-05 2024-09-17T22:43:13.051 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.051 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.051 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.051 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.051 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202983 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:13.051 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:43:13.051 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.292061287000003, 2024-09-17T22:43:13.052 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024014081, 2024-09-17T22:43:13.052 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.052 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.052 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:13.052 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:13.052 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:13.052 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:13.052 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202983 2024-09-17T22:43:13.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.052 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.052 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.052 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.052 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:43:13.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:43:13.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.053 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.053 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:43:13.053 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:43:13.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.053 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726131+0000", 2024-09-17T22:43:13.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9429999999999999e-06 2024-09-17T22:43:13.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726138+0000", 2024-09-17T22:43:13.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T22:43:13.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.054 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.054 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726143+0000", 2024-09-17T22:43:13.054 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0629999999999996e-06 2024-09-17T22:43:13.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749991+0000", 2024-09-17T22:43:13.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023847671000000001 2024-09-17T22:43:13.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750065+0000", 2024-09-17T22:43:13.055 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4569999999999999e-05 2024-09-17T22:43:13.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.055 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.055 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.055 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750143+0000", 2024-09-17T22:43:13.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7649000000000001e-05 2024-09-17T22:43:13.056 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.056 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.056 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.056 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.056 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.056 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202984 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:13.056 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:43:13.056 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.292040585999999, 2024-09-17T22:43:13.056 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024177605000000001, 2024-09-17T22:43:13.056 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.056 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.056 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:13.057 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:13.057 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:13.057 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:13.057 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202984 2024-09-17T22:43:13.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.057 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.057 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:43:13.057 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.057 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.057 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:43:13.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726150+0000", 2024-09-17T22:43:13.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T22:43:13.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.058 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.058 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.058 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726152+0000", 2024-09-17T22:43:13.058 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:43:13.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726162+0000", 2024-09-17T22:43:13.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7839999999999998e-06 2024-09-17T22:43:13.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726167+0000", 2024-09-17T22:43:13.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5690000000000004e-06 2024-09-17T22:43:13.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.059 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.059 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.059 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750184+0000", 2024-09-17T22:43:13.059 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024016909999999999 2024-09-17T22:43:13.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750255+0000", 2024-09-17T22:43:13.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1137999999999996e-05 2024-09-17T22:43:13.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.060 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.060 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.060 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750327+0000", 2024-09-17T22:43:13.060 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1903000000000006e-05 2024-09-17T22:43:13.060 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.060 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.060 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.061 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.061 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202985 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:13.061 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:43:13.061 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.292015198999998, 2024-09-17T22:43:13.061 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024280500999999999, 2024-09-17T22:43:13.061 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.061 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.061 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:13.061 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:13.061 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:13.061 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:13.061 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202985 2024-09-17T22:43:13.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.062 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:43:13.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:43:13.062 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.062 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.062 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.062 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:43:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8100000000000003e-07 2024-09-17T22:43:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726182+0000", 2024-09-17T22:43:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:43:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726190+0000", 2024-09-17T22:43:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:43:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.063 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726195+0000", 2024-09-17T22:43:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2809999999999999e-06 2024-09-17T22:43:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750366+0000", 2024-09-17T22:43:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024171635 2024-09-17T22:43:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750410+0000", 2024-09-17T22:43:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3823999999999999e-05 2024-09-17T22:43:13.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750455+0000", 2024-09-17T22:43:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5039e-05 2024-09-17T22:43:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202986 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:43:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.291987618999997, 2024-09-17T22:43:13.065 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024453077, 2024-09-17T22:43:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202986 2024-09-17T22:43:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:43:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:43:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726203+0000", 2024-09-17T22:43:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T22:43:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.067 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726205+0000", 2024-09-17T22:43:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0250000000000001e-06 2024-09-17T22:43:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726211+0000", 2024-09-17T22:43:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1340000000000001e-06 2024-09-17T22:43:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726222+0000", 2024-09-17T22:43:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1105e-05 2024-09-17T22:43:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.068 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750520+0000", 2024-09-17T22:43:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024297462999999998 2024-09-17T22:43:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750585+0000", 2024-09-17T22:43:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5940999999999998e-05 2024-09-17T22:43:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750655+0000", 2024-09-17T22:43:13.069 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0012000000000007e-05 2024-09-17T22:43:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202987 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:43:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.291961239000003, 2024-09-17T22:43:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024622569, 2024-09-17T22:43:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:13.070 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202987 2024-09-17T22:43:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:43:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:43:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:43:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:43:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726238+0000", 2024-09-17T22:43:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7430000000000008e-06 2024-09-17T22:43:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726245+0000", 2024-09-17T22:43:13.072 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6640000000000001e-06 2024-09-17T22:43:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726249+0000", 2024-09-17T22:43:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9119999999999997e-06 2024-09-17T22:43:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750694+0000", 2024-09-17T22:43:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024444105000000001 2024-09-17T22:43:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.073 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750767+0000", 2024-09-17T22:43:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3577000000000004e-05 2024-09-17T22:43:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750851+0000", 2024-09-17T22:43:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4189000000000003e-05 2024-09-17T22:43:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202988 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:13.074 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:43:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.291934376999997, 2024-09-17T22:43:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024808883, 2024-09-17T22:43:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202988 2024-09-17T22:43:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.075 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:43:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:43:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:43:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5499999999999999e-07 2024-09-17T22:43:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726258+0000", 2024-09-17T22:43:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8020000000000001e-06 2024-09-17T22:43:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726272+0000", 2024-09-17T22:43:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3828e-05 2024-09-17T22:43:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726277+0000", 2024-09-17T22:43:13.077 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2379999999999997e-06 2024-09-17T22:43:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750895+0000", 2024-09-17T22:43:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024618543 2024-09-17T22:43:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750978+0000", 2024-09-17T22:43:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3030999999999998e-05 2024-09-17T22:43:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751065+0000", 2024-09-17T22:43:13.078 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6086000000000002e-05 2024-09-17T22:43:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202989 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:43:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.291905503999999, 2024-09-17T22:43:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024976788, 2024-09-17T22:43:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:13.079 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202989 2024-09-17T22:43:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:43:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:43:13.080 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726285+0000", 2024-09-17T22:43:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T22:43:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726287+0000", 2024-09-17T22:43:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9860000000000001e-06 2024-09-17T22:43:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.081 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726294+0000", 2024-09-17T22:43:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2849999999999997e-06 2024-09-17T22:43:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726299+0000", 2024-09-17T22:43:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0150000000000003e-06 2024-09-17T22:43:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751109+0000", 2024-09-17T22:43:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024810175 2024-09-17T22:43:13.082 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751184+0000", 2024-09-17T22:43:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4640000000000004e-05 2024-09-17T22:43:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751261+0000", 2024-09-17T22:43:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7317000000000005e-05 2024-09-17T22:43:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.083 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202990 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:43:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.291885289, 2024-09-17T22:43:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025156767999999999, 2024-09-17T22:43:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202990 2024-09-17T22:43:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.084 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:43:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:43:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:43:13.085 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:43:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726307+0000", 2024-09-17T22:43:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7090000000000001e-06 2024-09-17T22:43:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726312+0000", 2024-09-17T22:43:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4909999999999996e-06 2024-09-17T22:43:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.086 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726317+0000", 2024-09-17T22:43:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3800000000000004e-06 2024-09-17T22:43:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751308+0000", 2024-09-17T22:43:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024991143 2024-09-17T22:43:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751385+0000", 2024-09-17T22:43:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6793999999999997e-05 2024-09-17T22:43:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.087 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751461+0000", 2024-09-17T22:43:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6891999999999995e-05 2024-09-17T22:43:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202991 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:43:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.291867255, 2024-09-17T22:43:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025289477000000001, 2024-09-17T22:43:13.088 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202991 2024-09-17T22:43:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:43:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:43:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:43:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.27e-07 2024-09-17T22:43:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726325+0000", 2024-09-17T22:43:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1459999999999999e-06 2024-09-17T22:43:13.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726333+0000", 2024-09-17T22:43:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1289999999999996e-06 2024-09-17T22:43:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726339+0000", 2024-09-17T22:43:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2530000000000004e-06 2024-09-17T22:43:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.091 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751499+0000", 2024-09-17T22:43:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025160109 2024-09-17T22:43:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751543+0000", 2024-09-17T22:43:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4317000000000003e-05 2024-09-17T22:43:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751612+0000", 2024-09-17T22:43:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9096000000000004e-05 2024-09-17T22:43:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.092 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202992 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:43:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.291845191, 2024-09-17T22:43:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025420585999999998, 2024-09-17T22:43:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:13.093 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202992 2024-09-17T22:43:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:43:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:43:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.094 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:43:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5199999999999998e-07 2024-09-17T22:43:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726347+0000", 2024-09-17T22:43:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9939999999999999e-06 2024-09-17T22:43:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726358+0000", 2024-09-17T22:43:13.095 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0421999999999999e-05 2024-09-17T22:43:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726363+0000", 2024-09-17T22:43:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:43:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751644+0000", 2024-09-17T22:43:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025280727999999999 2024-09-17T22:43:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.096 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751712+0000", 2024-09-17T22:43:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8207999999999995e-05 2024-09-17T22:43:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751765+0000", 2024-09-17T22:43:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3533999999999999e-05 2024-09-17T22:43:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.097 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202993 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:43:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 55.291821200000001, 2024-09-17T22:43:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025516259999999999, 2024-09-17T22:43:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202993 2024-09-17T22:43:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.098 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:43:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:43:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:43:13.099 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:43:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726371+0000", 2024-09-17T22:43:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T22:43:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726380+0000", 2024-09-17T22:43:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9109999999999999e-06 2024-09-17T22:43:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.100 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726385+0000", 2024-09-17T22:43:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7380000000000002e-06 2024-09-17T22:43:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751785+0000", 2024-09-17T22:43:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025400526 2024-09-17T22:43:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751824+0000", 2024-09-17T22:43:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8989999999999998e-05 2024-09-17T22:43:13.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751885+0000", 2024-09-17T22:43:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0937000000000003e-05 2024-09-17T22:43:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:215737 3.b 3:d5ee24db:::rbd_data.115f2f67ca35.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-17T22:43:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:43:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.717052504999998, 2024-09-17T22:43:13.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025642133000000001, 2024-09-17T22:43:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:43:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 215737 2024-09-17T22:43:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:43:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:43:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301138+0000", 2024-09-17T22:43:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8100000000000002e-07 2024-09-17T22:43:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301143+0000", 2024-09-17T22:43:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6999999999999999e-06 2024-09-17T22:43:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301152+0000", 2024-09-17T22:43:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4979999999999995e-06 2024-09-17T22:43:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301159+0000", 2024-09-17T22:43:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9430000000000003e-06 2024-09-17T22:43:13.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301201+0000", 2024-09-17T22:43:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1219e-05 2024-09-17T22:43:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301247+0000", 2024-09-17T22:43:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5924999999999998e-05 2024-09-17T22:43:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:43:13.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301870+0000", 2024-09-17T22:43:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00062316100000000001 2024-09-17T22:43:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:43:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326658+0000", 2024-09-17T22:43:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024788719000000001 2024-09-17T22:43:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:43:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326694+0000", 2024-09-17T22:43:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5858999999999997e-05 2024-09-17T22:43:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:43:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326727+0000", 2024-09-17T22:43:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3176999999999998e-05 2024-09-17T22:43:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326780+0000", 2024-09-17T22:43:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2151000000000003e-05 2024-09-17T22:43:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:215738 3.b 3:d5937cde:::rbd_data.115f2f67ca35.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-17T22:43:13.108 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:43:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.717015973000002, 2024-09-17T22:43:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025650267000000001, 2024-09-17T22:43:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:43:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 215738 2024-09-17T22:43:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:43:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:43:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301175+0000", 2024-09-17T22:43:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2600000000000002e-07 2024-09-17T22:43:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301178+0000", 2024-09-17T22:43:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2049999999999998e-06 2024-09-17T22:43:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301185+0000", 2024-09-17T22:43:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8900000000000001e-06 2024-09-17T22:43:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301190+0000", 2024-09-17T22:43:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6910000000000002e-06 2024-09-17T22:43:13.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.304605+0000", 2024-09-17T22:43:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003414594 2024-09-17T22:43:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.304658+0000", 2024-09-17T22:43:13.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3334000000000001e-05 2024-09-17T22:43:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:43:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.308265+0000", 2024-09-17T22:43:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036068680000000001 2024-09-17T22:43:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:43:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326688+0000", 2024-09-17T22:43:13.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018422851000000001 2024-09-17T22:43:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:43:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326785+0000", 2024-09-17T22:43:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7050000000000001e-05 2024-09-17T22:43:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:43:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326806+0000", 2024-09-17T22:43:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1299999999999999e-05 2024-09-17T22:43:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326824+0000", 2024-09-17T22:43:13.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7958e-05 2024-09-17T22:43:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop_reply(client.4447.0:215737 3.b e87/23 ondisk, result = 0)", 2024-09-17T22:43:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:43:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 37.714560761999998, 2024-09-17T22:43:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023035287000000002, 2024-09-17T22:43:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:13.115 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:43:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:43:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303630+0000", 2024-09-17T22:43:13.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2e-07 2024-09-17T22:43:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303633+0000", 2024-09-17T22:43:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7470000000000001e-06 2024-09-17T22:43:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303641+0000", 2024-09-17T22:43:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7979999999999995e-06 2024-09-17T22:43:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:13.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303646+0000", 2024-09-17T22:43:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9910000000000002e-06 2024-09-17T22:43:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326647+0000", 2024-09-17T22:43:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023000936999999999 2024-09-17T22:43:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326654+0000", 2024-09-17T22:43:13.118 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2529999999999998e-06 2024-09-17T22:43:13.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:13.119 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:13.119 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:13.119 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326665+0000", 2024-09-17T22:43:13.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0040999999999999e-05 2024-09-17T22:43:13.119 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.119 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.119 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.119 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:13.119 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:13.119 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:13.120 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:13.019+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:13.121 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:43:13.121 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:43:13.122 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:43:13.122 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:13.118+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:13.122 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:43:13.123 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:43:13.123 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:43:13.123 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:43:13.124 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:43:13.124 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is unset 2024-09-17T22:43:13.124 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:43:13.125 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:43:13.126 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:43:13.126 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:43:13.144 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:43:13.144 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:43:13.163 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:43:13.163 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:43:13.167 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:43:13.167 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:43:13.222 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:13.219+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:13.257 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:13.257 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:13.266 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:43:13.322 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:13.320+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:13.370 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:13.370 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:13.379 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:43:13.463 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:13.461+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:13.523 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:13.523 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:13.524 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:13.521+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:13.532 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:43:13.624 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:13.621+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:13.675 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:13.675 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:13.684 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:43:13.727 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:13.725+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:13.786 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:13.786 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:13.795 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:43:13.825 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:13.823+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:13.926 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:13.924+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:13.937 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:13.937 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:13.945 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:43:14.027 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:43:14.024+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:43:14.048 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:14.048 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:14.057 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:43:14.103 INFO:teuthology.orchestra.run.smithi115.stderr:set 2.5 pg_upmap_items mapping to [4->7,6->1] 2024-09-17T22:43:14.128 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:14.125+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:14.173 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:14.173 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:14.183 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:43:14.228 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:14.226+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:14.289 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:14.289 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:14.297 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:43:14.330 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:14.327+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:14.405 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:14.405 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:14.415 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:43:14.460 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:14.458+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:14.519 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:14.519 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:14.528 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:43:14.531 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:14.528+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:14.632 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:14.630+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:14.670 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:14.671 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:14.679 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:43:14.763 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:14.761+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:14.823 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:14.823 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:14.832 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:43:14.872 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:14.870+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:14.935 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:14.933+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:14.975 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:14.976 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:14.985 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:43:15.036 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:15.034+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:15.127 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:43:15.131 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:15.131 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:15.140 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:43:15.168 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T22:43:15.171 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:15.168+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:15.237 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:15.235+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:15.247 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:15.247 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:15.256 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:43:15.283 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:43:15.300 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:43:15.300 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:43:15.317 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:43:15.317 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:43:15.333 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:43:15.334 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:43:15.341 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:15.339+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:15.350 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:43:15.350 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:43:15.366 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:43:15.366 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:43:15.383 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:43:15.383 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:43:15.400 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:43:15.400 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:43:15.402 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:15.402 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:15.402 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:43:15.402 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:43:15.402 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:15.403 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:43:15.404 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:43:15.411 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:43:15.438 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:15.436+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:15.539 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:15.537+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:15.556 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:15.556 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:43:15.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:43:15.556 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:43:15.556 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.556 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188693 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5881856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:15.556 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:43:15.556 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.725658910000007, 2024-09-17T22:43:15.556 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028193223, 2024-09-17T22:43:15.557 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.557 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.557 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.557 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.557 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.557 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.557 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188693 2024-09-17T22:43:15.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.557 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.557 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.557 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.557 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:43:15.557 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:43:15.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827087+0000", 2024-09-17T22:43:15.558 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3599999999999999e-07 2024-09-17T22:43:15.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.558 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.558 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.558 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827090+0000", 2024-09-17T22:43:15.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2170000000000001e-06 2024-09-17T22:43:15.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827098+0000", 2024-09-17T22:43:15.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1510000000000004e-06 2024-09-17T22:43:15.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.559 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.559 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827103+0000", 2024-09-17T22:43:15.559 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0839999999999999e-06 2024-09-17T22:43:15.559 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.559 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855158+0000", 2024-09-17T22:43:15.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028055217 2024-09-17T22:43:15.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855201+0000", 2024-09-17T22:43:15.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3272000000000002e-05 2024-09-17T22:43:15.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.560 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.560 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855280+0000", 2024-09-17T22:43:15.560 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8845999999999997e-05 2024-09-17T22:43:15.561 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.561 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.561 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.561 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.561 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188694 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 5971968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:15.561 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:43:15.561 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.725635022999995, 2024-09-17T22:43:15.561 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028312278, 2024-09-17T22:43:15.561 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.561 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.562 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.562 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.562 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.562 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.562 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188694 2024-09-17T22:43:15.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.562 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.562 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.562 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:43:15.562 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.562 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:43:15.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827111+0000", 2024-09-17T22:43:15.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4799999999999999e-07 2024-09-17T22:43:15.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.563 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.563 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.563 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827114+0000", 2024-09-17T22:43:15.563 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.3029999999999998e-06 2024-09-17T22:43:15.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827121+0000", 2024-09-17T22:43:15.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3699999999999997e-06 2024-09-17T22:43:15.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.564 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.564 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827126+0000", 2024-09-17T22:43:15.564 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7350000000000004e-06 2024-09-17T22:43:15.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.564 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855307+0000", 2024-09-17T22:43:15.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028180851999999999 2024-09-17T22:43:15.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855346+0000", 2024-09-17T22:43:15.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9274999999999999e-05 2024-09-17T22:43:15.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.565 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.565 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855423+0000", 2024-09-17T22:43:15.565 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7294999999999998e-05 2024-09-17T22:43:15.566 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.566 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.566 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.566 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.566 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188695 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6062080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:15.566 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:43:15.566 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.725614245000003, 2024-09-17T22:43:15.566 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028435876999999998, 2024-09-17T22:43:15.566 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.566 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.566 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.566 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.567 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.567 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.567 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188695 2024-09-17T22:43:15.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.567 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:43:15.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.567 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.567 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.567 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:43:15.567 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827132+0000", 2024-09-17T22:43:15.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:43:15.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.568 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827138+0000", 2024-09-17T22:43:15.568 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2580000000000001e-06 2024-09-17T22:43:15.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.568 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.568 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827145+0000", 2024-09-17T22:43:15.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0129999999999996e-06 2024-09-17T22:43:15.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827150+0000", 2024-09-17T22:43:15.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3749999999999996e-06 2024-09-17T22:43:15.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.569 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.569 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.569 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855447+0000", 2024-09-17T22:43:15.569 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028297224999999999 2024-09-17T22:43:15.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855487+0000", 2024-09-17T22:43:15.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0454000000000003e-05 2024-09-17T22:43:15.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.570 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.570 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855567+0000", 2024-09-17T22:43:15.570 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.0149999999999997e-05 2024-09-17T22:43:15.570 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.570 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.570 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.570 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.570 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.571 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188696 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:15.571 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:43:15.571 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.725590460999996, 2024-09-17T22:43:15.571 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028543570000000001, 2024-09-17T22:43:15.571 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.571 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.571 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.571 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.571 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.571 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.571 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188696 2024-09-17T22:43:15.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.572 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:43:15.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827155+0000", 2024-09-17T22:43:15.572 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.572 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.572 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.572 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827156+0000", 2024-09-17T22:43:15.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3000000000000002e-07 2024-09-17T22:43:15.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827158+0000", 2024-09-17T22:43:15.573 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.875e-06 2024-09-17T22:43:15.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.573 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.573 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827164+0000", 2024-09-17T22:43:15.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7950000000000002e-06 2024-09-17T22:43:15.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827169+0000", 2024-09-17T22:43:15.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2880000000000003e-06 2024-09-17T22:43:15.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.574 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.574 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855582+0000", 2024-09-17T22:43:15.574 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028412929 2024-09-17T22:43:15.574 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.574 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855624+0000", 2024-09-17T22:43:15.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2484000000000002e-05 2024-09-17T22:43:15.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.575 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.575 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855699+0000", 2024-09-17T22:43:15.575 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4869000000000004e-05 2024-09-17T22:43:15.575 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.575 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.575 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.575 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.575 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:188697 3.12 3:495697e9:::rbd_data.115f65a1a5f0.0000000000000001:1a [read 6242304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:15.575 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:43:15.576 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 66.725558931999998, 2024-09-17T22:43:15.576 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028658598, 2024-09-17T22:43:15.576 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.576 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.576 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.576 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.576 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.576 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.576 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 188697 2024-09-17T22:43:15.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.576 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.576 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.576 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.576 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:43:15.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:43:15.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.577 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.577 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827187+0000", 2024-09-17T22:43:15.577 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8800000000000003e-07 2024-09-17T22:43:15.577 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.577 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827190+0000", 2024-09-17T22:43:15.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.351e-06 2024-09-17T22:43:15.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827198+0000", 2024-09-17T22:43:15.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.909e-06 2024-09-17T22:43:15.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.578 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.578 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.578 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.827203+0000", 2024-09-17T22:43:15.578 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4020000000000001e-06 2024-09-17T22:43:15.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855720+0000", 2024-09-17T22:43:15.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028516909999999999 2024-09-17T22:43:15.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855762+0000", 2024-09-17T22:43:15.579 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2184999999999997e-05 2024-09-17T22:43:15.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.579 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.579 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:08.855845+0000", 2024-09-17T22:43:15.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3353e-05 2024-09-17T22:43:15.580 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.580 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.580 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.580 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.580 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.580 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202883 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2342912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:15.580 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:43:15.580 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 57.827922551, 2024-09-17T22:43:15.580 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028098636, 2024-09-17T22:43:15.580 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.580 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.580 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.580 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.580 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.581 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.581 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202883 2024-09-17T22:43:15.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.581 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:43:15.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.581 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.581 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.581 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724823+0000", 2024-09-17T22:43:15.581 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724824+0000", 2024-09-17T22:43:15.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0200000000000003e-07 2024-09-17T22:43:15.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.582 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724826+0000", 2024-09-17T22:43:15.582 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.086e-06 2024-09-17T22:43:15.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.582 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.582 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724832+0000", 2024-09-17T22:43:15.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9100000000000002e-06 2024-09-17T22:43:15.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724837+0000", 2024-09-17T22:43:15.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9439999999999996e-06 2024-09-17T22:43:15.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.583 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752790+0000", 2024-09-17T22:43:15.583 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.027953657999999999 2024-09-17T22:43:15.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.583 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.583 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752848+0000", 2024-09-17T22:43:15.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7677999999999999e-05 2024-09-17T22:43:15.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.584 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.584 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752922+0000", 2024-09-17T22:43:15.584 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3957999999999999e-05 2024-09-17T22:43:15.584 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.584 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.584 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.584 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202884 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2473984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:15.585 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:43:15.585 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 57.827894325999999, 2024-09-17T22:43:15.585 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028206208999999999, 2024-09-17T22:43:15.585 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.585 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.585 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.585 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.585 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.585 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.585 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202884 2024-09-17T22:43:15.585 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.585 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.585 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:43:15.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724851+0000", 2024-09-17T22:43:15.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.586 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.586 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.586 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724852+0000", 2024-09-17T22:43:15.586 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2599999999999998e-07 2024-09-17T22:43:15.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724854+0000", 2024-09-17T22:43:15.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.26e-06 2024-09-17T22:43:15.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724871+0000", 2024-09-17T22:43:15.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.6589999999999999e-05 2024-09-17T22:43:15.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.587 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.587 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.587 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724877+0000", 2024-09-17T22:43:15.587 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.054e-06 2024-09-17T22:43:15.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752951+0000", 2024-09-17T22:43:15.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028073747 2024-09-17T22:43:15.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.588 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.588 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.752993+0000", 2024-09-17T22:43:15.588 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2082000000000001e-05 2024-09-17T22:43:15.588 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.588 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.589 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.589 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753058+0000", 2024-09-17T22:43:15.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5049999999999996e-05 2024-09-17T22:43:15.589 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.589 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.589 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.589 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.589 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202885 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:15.589 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:43:15.589 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 57.827861022999997, 2024-09-17T22:43:15.589 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028333028, 2024-09-17T22:43:15.589 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.589 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.590 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.590 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.590 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.590 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.590 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202885 2024-09-17T22:43:15.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.590 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.590 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.590 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:43:15.590 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.590 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:43:15.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724885+0000", 2024-09-17T22:43:15.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3300000000000002e-07 2024-09-17T22:43:15.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.591 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.591 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.591 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724892+0000", 2024-09-17T22:43:15.591 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2779999999999999e-06 2024-09-17T22:43:15.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724899+0000", 2024-09-17T22:43:15.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9110000000000004e-06 2024-09-17T22:43:15.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724904+0000", 2024-09-17T22:43:15.592 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2360000000000004e-06 2024-09-17T22:43:15.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.592 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.592 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753094+0000", 2024-09-17T22:43:15.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.02819033 2024-09-17T22:43:15.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753142+0000", 2024-09-17T22:43:15.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.829e-05 2024-09-17T22:43:15.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.593 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.593 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.593 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753218+0000", 2024-09-17T22:43:15.593 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.5450000000000004e-05 2024-09-17T22:43:15.593 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.593 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.593 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.594 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202886 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:15.594 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:43:15.594 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 57.827835825000001, 2024-09-17T22:43:15.594 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028442512, 2024-09-17T22:43:15.594 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.594 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.594 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.594 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.594 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.594 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.594 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202886 2024-09-17T22:43:15.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.594 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.594 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:43:15.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:43:15.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.595 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.595 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.595 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724910+0000", 2024-09-17T22:43:15.595 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:43:15.595 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724912+0000", 2024-09-17T22:43:15.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.967e-06 2024-09-17T22:43:15.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.596 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724918+0000", 2024-09-17T22:43:15.596 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8069999999999996e-06 2024-09-17T22:43:15.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.596 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.596 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724923+0000", 2024-09-17T22:43:15.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.386e-06 2024-09-17T22:43:15.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753244+0000", 2024-09-17T22:43:15.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028321567999999998 2024-09-17T22:43:15.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.597 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.597 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753289+0000", 2024-09-17T22:43:15.597 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4620000000000003e-05 2024-09-17T22:43:15.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.597 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.598 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.598 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753353+0000", 2024-09-17T22:43:15.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3795999999999997e-05 2024-09-17T22:43:15.598 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.598 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.598 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.598 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.598 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202887 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:15.598 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:43:15.598 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 57.827817912, 2024-09-17T22:43:15.598 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028565838999999999, 2024-09-17T22:43:15.598 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.599 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.599 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.599 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.599 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.599 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.599 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202887 2024-09-17T22:43:15.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.599 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.599 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.599 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.599 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:43:15.599 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:43:15.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724928+0000", 2024-09-17T22:43:15.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9099999999999999e-07 2024-09-17T22:43:15.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.600 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.600 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.600 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724930+0000", 2024-09-17T22:43:15.600 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8980000000000001e-06 2024-09-17T22:43:15.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724936+0000", 2024-09-17T22:43:15.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7849999999999997e-06 2024-09-17T22:43:15.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.601 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724942+0000", 2024-09-17T22:43:15.601 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9649999999999996e-06 2024-09-17T22:43:15.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.601 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753380+0000", 2024-09-17T22:43:15.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028438049 2024-09-17T22:43:15.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753425+0000", 2024-09-17T22:43:15.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5163000000000001e-05 2024-09-17T22:43:15.602 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.602 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.602 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.602 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753494+0000", 2024-09-17T22:43:15.602 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8588000000000001e-05 2024-09-17T22:43:15.602 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.602 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.603 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.603 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.603 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202888 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:15.603 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:43:15.603 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 57.827798414, 2024-09-17T22:43:15.603 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028703314000000001, 2024-09-17T22:43:15.603 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.603 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.603 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.603 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.603 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.603 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.604 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202888 2024-09-17T22:43:15.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.604 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:43:15.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.604 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.604 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724947+0000", 2024-09-17T22:43:15.604 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.604 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724948+0000", 2024-09-17T22:43:15.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:43:15.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724950+0000", 2024-09-17T22:43:15.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8029999999999999e-06 2024-09-17T22:43:15.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.605 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.605 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.605 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724955+0000", 2024-09-17T22:43:15.605 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8760000000000002e-06 2024-09-17T22:43:15.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724960+0000", 2024-09-17T22:43:15.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5979999999999999e-06 2024-09-17T22:43:15.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753527+0000", 2024-09-17T22:43:15.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028566864000000001 2024-09-17T22:43:15.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.606 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.606 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.606 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753568+0000", 2024-09-17T22:43:15.606 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.0775000000000002e-05 2024-09-17T22:43:15.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.607 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.607 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753651+0000", 2024-09-17T22:43:15.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3039000000000002e-05 2024-09-17T22:43:15.607 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.607 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.607 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.607 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.607 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202889 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:15.607 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:43:15.607 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 57.827780842000003, 2024-09-17T22:43:15.607 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028820964000000001, 2024-09-17T22:43:15.608 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.608 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.608 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.608 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.608 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.608 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.608 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202889 2024-09-17T22:43:15.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.608 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.608 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.608 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.608 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:43:15.608 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:43:15.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.609 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724965+0000", 2024-09-17T22:43:15.609 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2599999999999998e-07 2024-09-17T22:43:15.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.609 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.609 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724967+0000", 2024-09-17T22:43:15.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.818e-06 2024-09-17T22:43:15.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724973+0000", 2024-09-17T22:43:15.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6060000000000002e-06 2024-09-17T22:43:15.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.610 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.610 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724978+0000", 2024-09-17T22:43:15.610 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8230000000000002e-06 2024-09-17T22:43:15.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.610 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753687+0000", 2024-09-17T22:43:15.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028709495000000002 2024-09-17T22:43:15.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753727+0000", 2024-09-17T22:43:15.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9953999999999997e-05 2024-09-17T22:43:15.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.611 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.611 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.611 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753786+0000", 2024-09-17T22:43:15.611 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.8941999999999999e-05 2024-09-17T22:43:15.612 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.612 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.612 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.612 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.612 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202890 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:15.612 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:43:15.612 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 57.827763273999999, 2024-09-17T22:43:15.612 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028933838, 2024-09-17T22:43:15.612 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.612 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.612 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.612 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.613 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.613 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.613 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202890 2024-09-17T22:43:15.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.613 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:43:15.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.613 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.613 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.613 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:43:15.613 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724983+0000", 2024-09-17T22:43:15.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5999999999999999e-07 2024-09-17T22:43:15.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.614 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724985+0000", 2024-09-17T22:43:15.614 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8169999999999999e-06 2024-09-17T22:43:15.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.614 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.614 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724991+0000", 2024-09-17T22:43:15.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7790000000000001e-06 2024-09-17T22:43:15.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.724995+0000", 2024-09-17T22:43:15.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4530000000000004e-06 2024-09-17T22:43:15.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.615 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.615 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.615 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753811+0000", 2024-09-17T22:43:15.615 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028815584000000002 2024-09-17T22:43:15.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753854+0000", 2024-09-17T22:43:15.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3356999999999998e-05 2024-09-17T22:43:15.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.616 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.616 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753916+0000", 2024-09-17T22:43:15.616 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.2488000000000002e-05 2024-09-17T22:43:15.616 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.616 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.616 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.616 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.617 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202891 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:15.617 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:43:15.617 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 57.827745817, 2024-09-17T22:43:15.617 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029067554999999998, 2024-09-17T22:43:15.617 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.617 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.617 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.617 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.617 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.617 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.617 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202891 2024-09-17T22:43:15.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.617 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:43:15.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:43:15.618 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.618 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.618 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.618 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725000+0000", 2024-09-17T22:43:15.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5400000000000002e-07 2024-09-17T22:43:15.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725002+0000", 2024-09-17T22:43:15.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0559999999999999e-06 2024-09-17T22:43:15.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.619 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.619 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725008+0000", 2024-09-17T22:43:15.619 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6629999999999998e-06 2024-09-17T22:43:15.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.619 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725016+0000", 2024-09-17T22:43:15.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8269999999999998e-06 2024-09-17T22:43:15.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753949+0000", 2024-09-17T22:43:15.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.028932625999999999 2024-09-17T22:43:15.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.620 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.620 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.620 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.753994+0000", 2024-09-17T22:43:15.620 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5717000000000003e-05 2024-09-17T22:43:15.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.621 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.621 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754068+0000", 2024-09-17T22:43:15.621 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3312000000000006e-05 2024-09-17T22:43:15.621 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.621 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.621 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.621 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.621 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202892 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:15.621 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:43:15.621 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 57.827703110000002, 2024-09-17T22:43:15.622 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029159939999999999, 2024-09-17T22:43:15.622 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.622 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.622 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.622 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.622 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.622 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.622 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202892 2024-09-17T22:43:15.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.622 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.622 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.622 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.622 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:43:15.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:43:15.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.623 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725043+0000", 2024-09-17T22:43:15.623 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7999999999999995e-07 2024-09-17T22:43:15.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.623 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.623 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725045+0000", 2024-09-17T22:43:15.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.154e-06 2024-09-17T22:43:15.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725052+0000", 2024-09-17T22:43:15.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3629999999999999e-06 2024-09-17T22:43:15.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.624 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.624 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.624 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725056+0000", 2024-09-17T22:43:15.624 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3250000000000001e-06 2024-09-17T22:43:15.624 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754094+0000", 2024-09-17T22:43:15.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029038270000000001 2024-09-17T22:43:15.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.625 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754134+0000", 2024-09-17T22:43:15.625 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9722999999999999e-05 2024-09-17T22:43:15.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.625 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.625 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.626 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754203+0000", 2024-09-17T22:43:15.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8524999999999999e-05 2024-09-17T22:43:15.626 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.626 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.626 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.626 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.626 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202893 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:15.626 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:43:15.626 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 57.827683532999998, 2024-09-17T22:43:15.626 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029341651, 2024-09-17T22:43:15.626 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.626 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.626 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.627 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.627 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202893 2024-09-17T22:43:15.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.627 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.627 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.627 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:43:15.627 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725062+0000", 2024-09-17T22:43:15.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725063+0000", 2024-09-17T22:43:15.628 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6699999999999999e-07 2024-09-17T22:43:15.628 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.628 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.628 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.628 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725065+0000", 2024-09-17T22:43:15.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.2120000000000002e-06 2024-09-17T22:43:15.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725071+0000", 2024-09-17T22:43:15.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5169999999999996e-06 2024-09-17T22:43:15.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.629 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.629 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725076+0000", 2024-09-17T22:43:15.629 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.78e-06 2024-09-17T22:43:15.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.629 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754233+0000", 2024-09-17T22:43:15.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029156277000000001 2024-09-17T22:43:15.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754298+0000", 2024-09-17T22:43:15.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5320000000000005e-05 2024-09-17T22:43:15.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.630 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.630 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.630 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754404+0000", 2024-09-17T22:43:15.630 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000106078 2024-09-17T22:43:15.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.631 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.631 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.631 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.631 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202894 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:15.631 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:43:15.631 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 57.827663653999998, 2024-09-17T22:43:15.631 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029456572, 2024-09-17T22:43:15.631 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.631 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.631 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.631 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.632 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.632 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202894 2024-09-17T22:43:15.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.632 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:43:15.632 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.632 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.632 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.632 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725082+0000", 2024-09-17T22:43:15.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725083+0000", 2024-09-17T22:43:15.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5900000000000002e-07 2024-09-17T22:43:15.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.633 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.633 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725085+0000", 2024-09-17T22:43:15.633 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.979e-06 2024-09-17T22:43:15.633 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.633 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725091+0000", 2024-09-17T22:43:15.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1480000000000003e-06 2024-09-17T22:43:15.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725095+0000", 2024-09-17T22:43:15.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1339999999999997e-06 2024-09-17T22:43:15.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.634 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.634 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.634 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754429+0000", 2024-09-17T22:43:15.634 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029333758000000001 2024-09-17T22:43:15.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754471+0000", 2024-09-17T22:43:15.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2274999999999998e-05 2024-09-17T22:43:15.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.635 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.635 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754539+0000", 2024-09-17T22:43:15.635 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7818999999999996e-05 2024-09-17T22:43:15.635 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.635 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.635 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.635 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.636 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202895 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:15.636 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:43:15.636 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 57.827645257999997, 2024-09-17T22:43:15.636 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029635578999999999, 2024-09-17T22:43:15.636 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.636 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.636 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.636 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.636 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.636 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.636 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202895 2024-09-17T22:43:15.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.636 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:43:15.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:43:15.637 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.637 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.637 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725101+0000", 2024-09-17T22:43:15.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.08e-07 2024-09-17T22:43:15.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725107+0000", 2024-09-17T22:43:15.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9020000000000004e-06 2024-09-17T22:43:15.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.638 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725113+0000", 2024-09-17T22:43:15.638 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7400000000000001e-06 2024-09-17T22:43:15.638 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.638 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.638 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725117+0000", 2024-09-17T22:43:15.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6199999999999998e-06 2024-09-17T22:43:15.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754567+0000", 2024-09-17T22:43:15.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029449411000000002 2024-09-17T22:43:15.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.639 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.639 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.639 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754638+0000", 2024-09-17T22:43:15.639 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1432000000000003e-05 2024-09-17T22:43:15.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.640 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.640 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754736+0000", 2024-09-17T22:43:15.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.8066000000000006e-05 2024-09-17T22:43:15.640 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.640 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.640 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.640 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.640 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202896 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:15.640 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:43:15.640 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 57.827623527, 2024-09-17T22:43:15.640 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029760534000000002, 2024-09-17T22:43:15.641 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.641 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.641 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.641 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.641 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.641 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.641 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202896 2024-09-17T22:43:15.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.641 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.641 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.641 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.641 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:43:15.641 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725122+0000", 2024-09-17T22:43:15.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725123+0000", 2024-09-17T22:43:15.642 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.6800000000000001e-07 2024-09-17T22:43:15.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.642 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.642 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.642 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725125+0000", 2024-09-17T22:43:15.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.863e-06 2024-09-17T22:43:15.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725130+0000", 2024-09-17T22:43:15.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.7250000000000002e-06 2024-09-17T22:43:15.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.643 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.643 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725135+0000", 2024-09-17T22:43:15.643 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4390000000000002e-06 2024-09-17T22:43:15.643 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754763+0000", 2024-09-17T22:43:15.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029628202999999999 2024-09-17T22:43:15.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754805+0000", 2024-09-17T22:43:15.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.2370000000000003e-05 2024-09-17T22:43:15.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.644 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.644 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.644 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754883+0000", 2024-09-17T22:43:15.644 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7565999999999996e-05 2024-09-17T22:43:15.645 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.645 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.645 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.645 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.645 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202897 3.12 3:4ab982bc:::rbd_data.115f7a63970a.0000000000000000:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:15.645 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:43:15.645 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 57.827606166999999, 2024-09-17T22:43:15.645 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029947972999999999, 2024-09-17T22:43:15.645 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:15.645 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:15.645 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:15.645 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:15.646 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:15.646 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:15.646 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202897 2024-09-17T22:43:15.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.646 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:15.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:15.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:43:15.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.646 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.646 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:15.646 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:43:15.646 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:15.646 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:15.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725140+0000", 2024-09-17T22:43:15.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.4700000000000002e-07 2024-09-17T22:43:15.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:15.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725146+0000", 2024-09-17T22:43:15.647 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6030000000000004e-06 2024-09-17T22:43:15.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.647 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.647 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:15.647 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725152+0000", 2024-09-17T22:43:15.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.9970000000000004e-06 2024-09-17T22:43:15.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:15.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725156+0000", 2024-09-17T22:43:15.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.51e-06 2024-09-17T22:43:15.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:15.648 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754909+0000", 2024-09-17T22:43:15.648 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.029752958999999999 2024-09-17T22:43:15.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.648 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.648 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:15.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.754987+0000", 2024-09-17T22:43:15.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8208999999999996e-05 2024-09-17T22:43:15.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:15.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:15.649 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:15.649 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.755088+0000", 2024-09-17T22:43:15.649 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.000100348 2024-09-17T22:43:15.649 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.649 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.649 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.649 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:15.649 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:15.649 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:15.651 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:43:15.651 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:43:15.648+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:43:15.751 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:15.749+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:15.796 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:15.796 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:15.796 INFO:teuthology.orchestra.run.smithi129.stdout: "num_ops": 0 2024-09-17T22:43:15.796 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:15.805 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:43:15.889 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:15.887+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:15.947 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:15.948 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [], 2024-09-17T22:43:15.948 INFO:teuthology.orchestra.run.smithi129.stdout: "complaint_time": 30, 2024-09-17T22:43:15.948 INFO:teuthology.orchestra.run.smithi129.stdout: "num_blocked_ops": 0 2024-09-17T22:43:15.948 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:15.953 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:15.951+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:15.956 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:43:16.054 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:16.052+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:16.101 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-17T22:43:16.102 INFO:teuthology.orchestra.run.smithi129.stdout: "size": 20, 2024-09-17T22:43:16.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 600, 2024-09-17T22:43:16.102 INFO:teuthology.orchestra.run.smithi129.stdout: "ops": [ 2024-09-17T22:43:16.102 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.102 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202977 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:16.102 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:43:16.102 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.372620292999997, 2024-09-17T22:43:16.102 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023068750999999998, 2024-09-17T22:43:16.102 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.102 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.102 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:16.102 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:16.102 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:16.103 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.103 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202977 2024-09-17T22:43:16.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.103 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:43:16.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.103 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.103 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725979+0000", 2024-09-17T22:43:16.103 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.103 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725980+0000", 2024-09-17T22:43:16.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0299999999999999e-07 2024-09-17T22:43:16.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725982+0000", 2024-09-17T22:43:16.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0669999999999999e-06 2024-09-17T22:43:16.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.104 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.104 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.104 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725988+0000", 2024-09-17T22:43:16.104 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6610000000000003e-06 2024-09-17T22:43:16.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.725993+0000", 2024-09-17T22:43:16.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6079999999999998e-06 2024-09-17T22:43:16.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748888+0000", 2024-09-17T22:43:16.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.022894930000000001 2024-09-17T22:43:16.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.105 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.105 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.105 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.748951+0000", 2024-09-17T22:43:16.105 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3638000000000003e-05 2024-09-17T22:43:16.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.106 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.106 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749048+0000", 2024-09-17T22:43:16.106 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.6343999999999998e-05 2024-09-17T22:43:16.106 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.106 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.106 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.106 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.106 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202978 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2211840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:16.106 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:43:16.106 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.372599780999998, 2024-09-17T22:43:16.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023238050999999999, 2024-09-17T22:43:16.107 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.107 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.107 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:16.107 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:16.107 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:16.107 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.107 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202978 2024-09-17T22:43:16.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.107 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.107 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.107 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:43:16.107 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:43:16.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726000+0000", 2024-09-17T22:43:16.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.58e-07 2024-09-17T22:43:16.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.108 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.108 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.108 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726002+0000", 2024-09-17T22:43:16.108 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.193e-06 2024-09-17T22:43:16.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726009+0000", 2024-09-17T22:43:16.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8000000000000001e-06 2024-09-17T22:43:16.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726014+0000", 2024-09-17T22:43:16.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9840000000000001e-06 2024-09-17T22:43:16.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.109 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.109 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.109 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749090+0000", 2024-09-17T22:43:16.109 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023075917000000001 2024-09-17T22:43:16.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749163+0000", 2024-09-17T22:43:16.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3219999999999994e-05 2024-09-17T22:43:16.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.110 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.110 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749238+0000", 2024-09-17T22:43:16.110 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4479000000000003e-05 2024-09-17T22:43:16.110 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.110 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.110 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.110 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.111 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202979 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2342912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:16.111 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:43:16.111 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.372578330000003, 2024-09-17T22:43:16.111 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023393536999999999, 2024-09-17T22:43:16.111 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.111 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.111 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:16.111 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:16.111 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:16.111 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.111 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202979 2024-09-17T22:43:16.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.111 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.111 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:43:16.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726021+0000", 2024-09-17T22:43:16.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.112 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.112 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726022+0000", 2024-09-17T22:43:16.112 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8699999999999995e-07 2024-09-17T22:43:16.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726024+0000", 2024-09-17T22:43:16.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1969999999999999e-06 2024-09-17T22:43:16.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726030+0000", 2024-09-17T22:43:16.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.691e-06 2024-09-17T22:43:16.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.113 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.113 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.113 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726035+0000", 2024-09-17T22:43:16.113 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8029999999999996e-06 2024-09-17T22:43:16.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749280+0000", 2024-09-17T22:43:16.114 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023244668 2024-09-17T22:43:16.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.114 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.114 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.114 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749344+0000", 2024-09-17T22:43:16.116 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.4418000000000006e-05 2024-09-17T22:43:16.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.117 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.117 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749415+0000", 2024-09-17T22:43:16.117 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0272999999999996e-05 2024-09-17T22:43:16.117 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.118 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.118 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.118 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202980 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2473984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:16.118 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:43:16.119 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.372557561000001, 2024-09-17T22:43:16.119 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023562221000000001, 2024-09-17T22:43:16.119 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.119 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.119 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:16.120 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:16.120 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:16.120 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.120 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202980 2024-09-17T22:43:16.120 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.120 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.121 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.121 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.121 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:43:16.121 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.122 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.122 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:43:16.122 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.122 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.123 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.123 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726042+0000", 2024-09-17T22:43:16.123 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3700000000000001e-07 2024-09-17T22:43:16.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.123 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.124 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.124 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726044+0000", 2024-09-17T22:43:16.124 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9489999999999999e-06 2024-09-17T22:43:16.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.125 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.125 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726066+0000", 2024-09-17T22:43:16.125 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1508000000000001e-05 2024-09-17T22:43:16.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.126 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.126 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726072+0000", 2024-09-17T22:43:16.126 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.3450000000000002e-06 2024-09-17T22:43:16.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.127 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.127 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749455+0000", 2024-09-17T22:43:16.127 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023382824999999999 2024-09-17T22:43:16.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.127 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.128 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.128 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749527+0000", 2024-09-17T22:43:16.128 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1728000000000002e-05 2024-09-17T22:43:16.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.128 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.129 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.129 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749604+0000", 2024-09-17T22:43:16.129 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7428999999999993e-05 2024-09-17T22:43:16.129 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.129 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.130 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.130 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202981 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2605056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:16.130 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:43:16.130 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.372519285999999, 2024-09-17T22:43:16.131 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023677800999999998, 2024-09-17T22:43:16.131 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.131 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.131 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:16.131 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:16.132 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:16.132 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.132 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202981 2024-09-17T22:43:16.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.132 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.132 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.133 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:43:16.133 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.133 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.133 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726080+0000", 2024-09-17T22:43:16.134 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.134 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.134 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.134 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726081+0000", 2024-09-17T22:43:16.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5499999999999998e-07 2024-09-17T22:43:16.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.135 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.135 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.135 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726083+0000", 2024-09-17T22:43:16.135 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1780000000000002e-06 2024-09-17T22:43:16.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.136 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.136 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726090+0000", 2024-09-17T22:43:16.136 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8510000000000001e-06 2024-09-17T22:43:16.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.137 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.137 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726101+0000", 2024-09-17T22:43:16.137 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.133e-05 2024-09-17T22:43:16.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.138 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749639+0000", 2024-09-17T22:43:16.138 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023537679999999998 2024-09-17T22:43:16.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.138 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.138 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749688+0000", 2024-09-17T22:43:16.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9548000000000001e-05 2024-09-17T22:43:16.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.139 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.139 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.139 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749758+0000", 2024-09-17T22:43:16.139 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9758999999999994e-05 2024-09-17T22:43:16.140 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.140 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.140 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.140 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.141 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202982 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:16.141 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:43:16.141 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.372491703000001, 2024-09-17T22:43:16.141 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023841469000000001, 2024-09-17T22:43:16.141 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.141 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.142 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:16.142 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:16.142 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:16.142 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.142 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202982 2024-09-17T22:43:16.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.143 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.143 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.143 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.143 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:43:16.143 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.144 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.144 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:43:16.144 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.145 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.145 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.145 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726108+0000", 2024-09-17T22:43:16.145 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4000000000000002e-07 2024-09-17T22:43:16.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.146 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726110+0000", 2024-09-17T22:43:16.146 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.902e-06 2024-09-17T22:43:16.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.146 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.146 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.147 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726117+0000", 2024-09-17T22:43:16.147 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6189999999999996e-06 2024-09-17T22:43:16.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.147 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.147 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726122+0000", 2024-09-17T22:43:16.148 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.198e-06 2024-09-17T22:43:16.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.148 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.148 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.148 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749795+0000", 2024-09-17T22:43:16.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023673397999999998 2024-09-17T22:43:16.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.149 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.149 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.149 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749877+0000", 2024-09-17T22:43:16.149 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1910999999999996e-05 2024-09-17T22:43:16.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.150 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.150 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.150 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749949+0000", 2024-09-17T22:43:16.150 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2001000000000004e-05 2024-09-17T22:43:16.150 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.151 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.151 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.151 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.151 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202983 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2867200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:16.152 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:43:16.152 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.372470644000003, 2024-09-17T22:43:16.152 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024014081, 2024-09-17T22:43:16.152 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.152 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.152 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:16.153 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:16.153 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:16.153 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.153 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202983 2024-09-17T22:43:16.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.154 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.154 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.154 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:43:16.154 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.155 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.155 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.155 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:43:16.155 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.156 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.156 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726129+0000", 2024-09-17T22:43:16.156 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1899999999999998e-07 2024-09-17T22:43:16.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.156 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.157 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.157 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726131+0000", 2024-09-17T22:43:16.157 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9429999999999999e-06 2024-09-17T22:43:16.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.157 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.158 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726138+0000", 2024-09-17T22:43:16.158 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.7660000000000001e-06 2024-09-17T22:43:16.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.158 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.158 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.159 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726143+0000", 2024-09-17T22:43:16.159 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0629999999999996e-06 2024-09-17T22:43:16.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.159 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.159 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.749991+0000", 2024-09-17T22:43:16.160 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023847671000000001 2024-09-17T22:43:16.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.160 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.160 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.160 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750065+0000", 2024-09-17T22:43:16.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4569999999999999e-05 2024-09-17T22:43:16.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.161 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.161 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.161 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750143+0000", 2024-09-17T22:43:16.161 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7649000000000001e-05 2024-09-17T22:43:16.162 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.162 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.162 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.162 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.163 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202984 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 2998272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:16.163 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:43:16.163 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.372449942999999, 2024-09-17T22:43:16.163 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024177605000000001, 2024-09-17T22:43:16.164 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.164 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.164 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:16.164 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:16.164 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:16.164 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.165 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202984 2024-09-17T22:43:16.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.165 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.165 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.165 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.165 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:43:16.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.166 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.166 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726149+0000", 2024-09-17T22:43:16.166 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.167 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.167 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.167 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726150+0000", 2024-09-17T22:43:16.167 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3099999999999998e-07 2024-09-17T22:43:16.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.168 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.168 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726152+0000", 2024-09-17T22:43:16.168 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8700000000000001e-06 2024-09-17T22:43:16.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.168 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726162+0000", 2024-09-17T22:43:16.169 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7839999999999998e-06 2024-09-17T22:43:16.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.169 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.169 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.169 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726167+0000", 2024-09-17T22:43:16.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.5690000000000004e-06 2024-09-17T22:43:16.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.170 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750184+0000", 2024-09-17T22:43:16.170 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024016909999999999 2024-09-17T22:43:16.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.170 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750255+0000", 2024-09-17T22:43:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1137999999999996e-05 2024-09-17T22:43:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750327+0000", 2024-09-17T22:43:16.171 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.1903000000000006e-05 2024-09-17T22:43:16.172 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.172 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.172 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.172 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202985 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3129344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:16.172 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:43:16.173 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.372424555999999, 2024-09-17T22:43:16.173 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024280500999999999, 2024-09-17T22:43:16.173 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.173 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.173 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:16.173 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:16.173 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:16.173 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.173 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202985 2024-09-17T22:43:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:43:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:43:16.174 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.175 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726175+0000", 2024-09-17T22:43:16.175 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.8100000000000003e-07 2024-09-17T22:43:16.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.175 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.175 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726182+0000", 2024-09-17T22:43:16.176 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9659999999999999e-06 2024-09-17T22:43:16.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.176 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.176 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.176 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726190+0000", 2024-09-17T22:43:16.177 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2749999999999998e-06 2024-09-17T22:43:16.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.177 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.177 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.177 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726195+0000", 2024-09-17T22:43:16.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2809999999999999e-06 2024-09-17T22:43:16.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.178 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.178 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.178 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750366+0000", 2024-09-17T22:43:16.178 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024171635 2024-09-17T22:43:16.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.179 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.179 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750410+0000", 2024-09-17T22:43:16.179 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3823999999999999e-05 2024-09-17T22:43:16.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.180 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.180 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750455+0000", 2024-09-17T22:43:16.180 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5039e-05 2024-09-17T22:43:16.180 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.180 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.181 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.181 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.181 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202986 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3260416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:16.181 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:43:16.181 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.372396975999997, 2024-09-17T22:43:16.181 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024453077, 2024-09-17T22:43:16.181 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202986 2024-09-17T22:43:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:43:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.183 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.183 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726202+0000", 2024-09-17T22:43:16.183 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.183 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.184 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726203+0000", 2024-09-17T22:43:16.184 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.9700000000000002e-07 2024-09-17T22:43:16.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.184 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.184 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726205+0000", 2024-09-17T22:43:16.185 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.0250000000000001e-06 2024-09-17T22:43:16.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.185 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.185 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.185 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726211+0000", 2024-09-17T22:43:16.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.1340000000000001e-06 2024-09-17T22:43:16.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.186 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.186 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.186 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726222+0000", 2024-09-17T22:43:16.186 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.1105e-05 2024-09-17T22:43:16.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.187 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.187 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750520+0000", 2024-09-17T22:43:16.187 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024297462999999998 2024-09-17T22:43:16.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.187 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.187 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750585+0000", 2024-09-17T22:43:16.188 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.5940999999999998e-05 2024-09-17T22:43:16.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.188 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.188 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.188 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750655+0000", 2024-09-17T22:43:16.189 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.0012000000000007e-05 2024-09-17T22:43:16.189 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.189 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.189 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.190 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202987 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:16.190 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:43:16.190 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.372370596000003, 2024-09-17T22:43:16.190 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024622569, 2024-09-17T22:43:16.190 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.190 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.190 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:16.191 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:16.191 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:16.191 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.191 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202987 2024-09-17T22:43:16.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.191 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.192 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.192 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:43:16.192 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.192 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.192 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.193 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:43:16.193 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.193 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.193 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726229+0000", 2024-09-17T22:43:16.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7899999999999999e-07 2024-09-17T22:43:16.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.194 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.194 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726238+0000", 2024-09-17T22:43:16.194 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7430000000000008e-06 2024-09-17T22:43:16.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.195 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.195 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726245+0000", 2024-09-17T22:43:16.195 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.6640000000000001e-06 2024-09-17T22:43:16.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.195 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726249+0000", 2024-09-17T22:43:16.196 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9119999999999997e-06 2024-09-17T22:43:16.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.196 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.196 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.196 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750694+0000", 2024-09-17T22:43:16.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024444105000000001 2024-09-17T22:43:16.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.197 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.197 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.197 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750767+0000", 2024-09-17T22:43:16.197 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.3577000000000004e-05 2024-09-17T22:43:16.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.198 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.198 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.198 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750851+0000", 2024-09-17T22:43:16.198 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.4189000000000003e-05 2024-09-17T22:43:16.198 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.198 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.198 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.199 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.199 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202988 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:16.199 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:43:16.199 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.372343733999998, 2024-09-17T22:43:16.199 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024808883, 2024-09-17T22:43:16.200 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.200 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.200 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:16.200 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:16.200 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:16.200 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202988 2024-09-17T22:43:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:43:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:43:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726256+0000", 2024-09-17T22:43:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5499999999999999e-07 2024-09-17T22:43:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726258+0000", 2024-09-17T22:43:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.8020000000000001e-06 2024-09-17T22:43:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.202 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726272+0000", 2024-09-17T22:43:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.3828e-05 2024-09-17T22:43:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726277+0000", 2024-09-17T22:43:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2379999999999997e-06 2024-09-17T22:43:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750895+0000", 2024-09-17T22:43:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024618543 2024-09-17T22:43:16.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.750978+0000", 2024-09-17T22:43:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.3030999999999998e-05 2024-09-17T22:43:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751065+0000", 2024-09-17T22:43:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.6086000000000002e-05 2024-09-17T22:43:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.204 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202989 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:43:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.372314861, 2024-09-17T22:43:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024976788, 2024-09-17T22:43:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202989 2024-09-17T22:43:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.205 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:43:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726284+0000", 2024-09-17T22:43:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726285+0000", 2024-09-17T22:43:16.206 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.7e-07 2024-09-17T22:43:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726287+0000", 2024-09-17T22:43:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9860000000000001e-06 2024-09-17T22:43:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726294+0000", 2024-09-17T22:43:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2849999999999997e-06 2024-09-17T22:43:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.207 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726299+0000", 2024-09-17T22:43:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.0150000000000003e-06 2024-09-17T22:43:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751109+0000", 2024-09-17T22:43:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024810175 2024-09-17T22:43:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751184+0000", 2024-09-17T22:43:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.4640000000000004e-05 2024-09-17T22:43:16.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751261+0000", 2024-09-17T22:43:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.7317000000000005e-05 2024-09-17T22:43:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202990 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3784704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:43:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.372294646, 2024-09-17T22:43:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025156767999999999, 2024-09-17T22:43:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.209 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202990 2024-09-17T22:43:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:43:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.210 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:43:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726305+0000", 2024-09-17T22:43:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5900000000000003e-07 2024-09-17T22:43:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726307+0000", 2024-09-17T22:43:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7090000000000001e-06 2024-09-17T22:43:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.211 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726312+0000", 2024-09-17T22:43:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.4909999999999996e-06 2024-09-17T22:43:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726317+0000", 2024-09-17T22:43:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.3800000000000004e-06 2024-09-17T22:43:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751308+0000", 2024-09-17T22:43:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024991143 2024-09-17T22:43:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.212 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751385+0000", 2024-09-17T22:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6793999999999997e-05 2024-09-17T22:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751461+0000", 2024-09-17T22:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.6891999999999995e-05 2024-09-17T22:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.213 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202991 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 3915776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.372276612, 2024-09-17T22:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025289477000000001, 2024-09-17T22:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202991 2024-09-17T22:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.214 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:43:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:43:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726323+0000", 2024-09-17T22:43:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.27e-07 2024-09-17T22:43:16.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726325+0000", 2024-09-17T22:43:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1459999999999999e-06 2024-09-17T22:43:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726333+0000", 2024-09-17T22:43:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.1289999999999996e-06 2024-09-17T22:43:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726339+0000", 2024-09-17T22:43:16.216 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2530000000000004e-06 2024-09-17T22:43:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751499+0000", 2024-09-17T22:43:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025160109 2024-09-17T22:43:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751543+0000", 2024-09-17T22:43:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.4317000000000003e-05 2024-09-17T22:43:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.217 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751612+0000", 2024-09-17T22:43:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9096000000000004e-05 2024-09-17T22:43:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202992 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4046848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:43:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.372254548000001, 2024-09-17T22:43:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025420585999999998, 2024-09-17T22:43:16.218 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202992 2024-09-17T22:43:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:43:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.219 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:43:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726345+0000", 2024-09-17T22:43:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5199999999999998e-07 2024-09-17T22:43:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726347+0000", 2024-09-17T22:43:16.220 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.9939999999999999e-06 2024-09-17T22:43:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726358+0000", 2024-09-17T22:43:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0421999999999999e-05 2024-09-17T22:43:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726363+0000", 2024-09-17T22:43:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3480000000000003e-06 2024-09-17T22:43:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.221 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751644+0000", 2024-09-17T22:43:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025280727999999999 2024-09-17T22:43:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751712+0000", 2024-09-17T22:43:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8207999999999995e-05 2024-09-17T22:43:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751765+0000", 2024-09-17T22:43:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3533999999999999e-05 2024-09-17T22:43:16.222 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:202993 3.b 3:d75c2f39:::rbd_data.115f7a63970a.0000000000000003:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-17T22:43:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:43:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 58.372230557000002, 2024-09-17T22:43:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025516259999999999, 2024-09-17T22:43:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:16.223 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.224 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 202993 2024-09-17T22:43:16.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.224 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.224 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.224 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:43:16.224 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.224 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:43:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726369+0000", 2024-09-17T22:43:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.1399999999999998e-07 2024-09-17T22:43:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726371+0000", 2024-09-17T22:43:16.225 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.844e-06 2024-09-17T22:43:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726380+0000", 2024-09-17T22:43:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.9109999999999999e-06 2024-09-17T22:43:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.726385+0000", 2024-09-17T22:43:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.7380000000000002e-06 2024-09-17T22:43:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.226 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751785+0000", 2024-09-17T22:43:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025400526 2024-09-17T22:43:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751824+0000", 2024-09-17T22:43:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.8989999999999998e-05 2024-09-17T22:43:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:17.751885+0000", 2024-09-17T22:43:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.0937000000000003e-05 2024-09-17T22:43:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.227 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:215737 3.b 3:d5ee24db:::rbd_data.115f2f67ca35.0000000000000002:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-17T22:43:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:43:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.797461861999999, 2024-09-17T22:43:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025642133000000001, 2024-09-17T22:43:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:43:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:16.228 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 215737 2024-09-17T22:43:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:43:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301137+0000", 2024-09-17T22:43:16.229 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301138+0000", 2024-09-17T22:43:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.8100000000000002e-07 2024-09-17T22:43:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301143+0000", 2024-09-17T22:43:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.6999999999999999e-06 2024-09-17T22:43:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301152+0000", 2024-09-17T22:43:16.230 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.4979999999999995e-06 2024-09-17T22:43:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301159+0000", 2024-09-17T22:43:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.9430000000000003e-06 2024-09-17T22:43:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301201+0000", 2024-09-17T22:43:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.1219e-05 2024-09-17T22:43:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.231 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301247+0000", 2024-09-17T22:43:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.5924999999999998e-05 2024-09-17T22:43:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:43:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301870+0000", 2024-09-17T22:43:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.00062316100000000001 2024-09-17T22:43:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:43:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326658+0000", 2024-09-17T22:43:16.232 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.024788719000000001 2024-09-17T22:43:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:43:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326694+0000", 2024-09-17T22:43:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.5858999999999997e-05 2024-09-17T22:43:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:43:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326727+0000", 2024-09-17T22:43:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.3176999999999998e-05 2024-09-17T22:43:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.233 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326780+0000", 2024-09-17T22:43:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2151000000000003e-05 2024-09-17T22:43:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_op(client.4447.0:215738 3.b 3:d5937cde:::rbd_data.115f2f67ca35.0000000000000003:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-17T22:43:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:43:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.797425330000003, 2024-09-17T22:43:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.025650267000000001, 2024-09-17T22:43:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.234 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:43:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: "client_info": { 2024-09-17T22:43:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: "client": "client.4447", 2024-09-17T22:43:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: "tid": 215738 2024-09-17T22:43:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:43:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.235 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301174+0000", 2024-09-17T22:43:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301175+0000", 2024-09-17T22:43:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2600000000000002e-07 2024-09-17T22:43:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301178+0000", 2024-09-17T22:43:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 3.2049999999999998e-06 2024-09-17T22:43:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.236 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301185+0000", 2024-09-17T22:43:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 6.8900000000000001e-06 2024-09-17T22:43:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.301190+0000", 2024-09-17T22:43:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.6910000000000002e-06 2024-09-17T22:43:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.304605+0000", 2024-09-17T22:43:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.003414594 2024-09-17T22:43:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.237 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.304658+0000", 2024-09-17T22:43:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.3334000000000001e-05 2024-09-17T22:43:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "waiting for subops from 6", 2024-09-17T22:43:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.308265+0000", 2024-09-17T22:43:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.0036068680000000001 2024-09-17T22:43:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:43:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326688+0000", 2024-09-17T22:43:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.018422851000000001 2024-09-17T22:43:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.238 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "op_commit", 2024-09-17T22:43:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326785+0000", 2024-09-17T22:43:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 9.7050000000000001e-05 2024-09-17T22:43:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "commit_sent", 2024-09-17T22:43:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326806+0000", 2024-09-17T22:43:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.1299999999999999e-05 2024-09-17T22:43:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326824+0000", 2024-09-17T22:43:16.239 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.7958e-05 2024-09-17T22:43:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: "description": "osd_repop_reply(client.4447.0:215737 3.b e87/23 ondisk, result = 0)", 2024-09-17T22:43:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: "initiated_at": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:43:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 40.794970118999998, 2024-09-17T22:43:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023035287000000002, 2024-09-17T22:43:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: "continuous": false, 2024-09-17T22:43:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: "type_data": { 2024-09-17T22:43:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: "flag_point": "started", 2024-09-17T22:43:16.240 INFO:teuthology.orchestra.run.smithi129.stdout: "events": [ 2024-09-17T22:43:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "initiated", 2024-09-17T22:43:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:43:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "header_read", 2024-09-17T22:43:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303629+0000", 2024-09-17T22:43:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0 2024-09-17T22:43:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "throttled", 2024-09-17T22:43:16.241 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303630+0000", 2024-09-17T22:43:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 5.2e-07 2024-09-17T22:43:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "all_read", 2024-09-17T22:43:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303633+0000", 2024-09-17T22:43:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 2.7470000000000001e-06 2024-09-17T22:43:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "dispatched", 2024-09-17T22:43:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303641+0000", 2024-09-17T22:43:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 8.7979999999999995e-06 2024-09-17T22:43:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.242 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.303646+0000", 2024-09-17T22:43:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 4.9910000000000002e-06 2024-09-17T22:43:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "reached_pg", 2024-09-17T22:43:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326647+0000", 2024-09-17T22:43:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 0.023000936999999999 2024-09-17T22:43:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "started", 2024-09-17T22:43:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326654+0000", 2024-09-17T22:43:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 7.2529999999999998e-06 2024-09-17T22:43:16.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-17T22:43:16.244 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-17T22:43:16.244 INFO:teuthology.orchestra.run.smithi129.stdout: "event": "done", 2024-09-17T22:43:16.244 INFO:teuthology.orchestra.run.smithi129.stdout: "time": "2024-09-17T22:42:35.326665+0000", 2024-09-17T22:43:16.244 INFO:teuthology.orchestra.run.smithi129.stdout: "duration": 1.0040999999999999e-05 2024-09-17T22:43:16.244 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.244 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.244 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.244 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-17T22:43:16.244 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-17T22:43:16.244 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-17T22:43:16.246 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is unset 2024-09-17T22:43:16.246 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:43:16.243+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:43:16.247 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T22:43:16.346 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:16.344+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:16.391 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:16.391 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:43:16.391 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:43:16.391 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:16.399 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T22:43:16.484 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:16.481+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:16.543 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:16.543 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:43:16.543 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:43:16.543 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:43:16.543 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:16.548 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:16.546+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:16.551 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T22:43:16.649 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:16.646+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:16.696 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:16.696 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:43:16.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:43:16.696 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:43:16.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.696 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4210.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-17T22:43:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:43:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 133.334092731, 2024-09-17T22:43:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.249158453, 2024-09-17T22:43:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T22:43:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4210", 2024-09-17T22:43:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.129:0/2150849834", 2024-09-17T22:43:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-09-17T22:43:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:43:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358678+0000", 2024-09-17T22:43:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358680+0000", 2024-09-17T22:43:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.592e-06 2024-09-17T22:43:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358688+0000", 2024-09-17T22:43:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1610000000000003e-06 2024-09-17T22:43:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358697+0000", 2024-09-17T22:43:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2580000000000001e-06 2024-09-17T22:43:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358706+0000", 2024-09-17T22:43:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1640000000000007e-06 2024-09-17T22:43:16.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358766+0000", 2024-09-17T22:43:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9893999999999999e-05 2024-09-17T22:43:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-09-17T22:43:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.358778+0000", 2024-09-17T22:43:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1148999999999999e-05 2024-09-17T22:43:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606347+0000", 2024-09-17T22:43:16.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.247569971 2024-09-17T22:43:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606533+0000", 2024-09-17T22:43:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018599400000000001 2024-09-17T22:43:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-09-17T22:43:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.606752+0000", 2024-09-17T22:43:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000218373 2024-09-17T22:43:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-09-17T22:43:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607455+0000", 2024-09-17T22:43:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070311799999999997 2024-09-17T22:43:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-09-17T22:43:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607735+0000", 2024-09-17T22:43:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027972300000000001 2024-09-17T22:43:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-09-17T22:43:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607793+0000", 2024-09-17T22:43:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8529999999999997e-05 2024-09-17T22:43:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:03.607837+0000", 2024-09-17T22:43:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3526000000000003e-05 2024-09-17T22:43:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186749 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 11960320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:43:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.269451408999998, 2024-09-17T22:43:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029226166000000001, 2024-09-17T22:43:16.703 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186749 2024-09-17T22:43:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:43:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:43:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423320+0000", 2024-09-17T22:43:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-09-17T22:43:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423328+0000", 2024-09-17T22:43:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6790000000000005e-06 2024-09-17T22:43:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423334+0000", 2024-09-17T22:43:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0009999999999998e-06 2024-09-17T22:43:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423338+0000", 2024-09-17T22:43:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6419999999999998e-06 2024-09-17T22:43:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452464+0000", 2024-09-17T22:43:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029125389000000002 2024-09-17T22:43:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452501+0000", 2024-09-17T22:43:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7656e-05 2024-09-17T22:43:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452546+0000", 2024-09-17T22:43:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4468999999999997e-05 2024-09-17T22:43:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186750 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12050432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:43:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.269425619000003, 2024-09-17T22:43:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029307863999999999, 2024-09-17T22:43:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.708 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186750 2024-09-17T22:43:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:43:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423345+0000", 2024-09-17T22:43:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423346+0000", 2024-09-17T22:43:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.84e-07 2024-09-17T22:43:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423348+0000", 2024-09-17T22:43:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7540000000000001e-06 2024-09-17T22:43:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423353+0000", 2024-09-17T22:43:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3720000000000004e-06 2024-09-17T22:43:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423362+0000", 2024-09-17T22:43:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9220000000000003e-06 2024-09-17T22:43:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452571+0000", 2024-09-17T22:43:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029209174000000001 2024-09-17T22:43:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452612+0000", 2024-09-17T22:43:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1363999999999999e-05 2024-09-17T22:43:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452653+0000", 2024-09-17T22:43:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0893999999999998e-05 2024-09-17T22:43:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186751 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12140544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:43:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.269403128999997, 2024-09-17T22:43:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029387488, 2024-09-17T22:43:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.712 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186751 2024-09-17T22:43:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:43:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:43:16.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423368+0000", 2024-09-17T22:43:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1399999999999997e-07 2024-09-17T22:43:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423375+0000", 2024-09-17T22:43:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8889999999999996e-06 2024-09-17T22:43:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423381+0000", 2024-09-17T22:43:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5690000000000004e-06 2024-09-17T22:43:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423386+0000", 2024-09-17T22:43:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9559999999999996e-06 2024-09-17T22:43:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452679+0000", 2024-09-17T22:43:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029292820000000001 2024-09-17T22:43:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452718+0000", 2024-09-17T22:43:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8961000000000002e-05 2024-09-17T22:43:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452755+0000", 2024-09-17T22:43:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7879000000000001e-05 2024-09-17T22:43:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.716 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186752 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12230656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:43:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.269371878000001, 2024-09-17T22:43:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029463269, 2024-09-17T22:43:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186752 2024-09-17T22:43:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:43:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423399+0000", 2024-09-17T22:43:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423400+0000", 2024-09-17T22:43:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7300000000000002e-07 2024-09-17T22:43:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423401+0000", 2024-09-17T22:43:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3599999999999999e-06 2024-09-17T22:43:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423404+0000", 2024-09-17T22:43:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5999999999999998e-06 2024-09-17T22:43:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423407+0000", 2024-09-17T22:43:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6869999999999999e-06 2024-09-17T22:43:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452782+0000", 2024-09-17T22:43:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029375107000000001 2024-09-17T22:43:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452821+0000", 2024-09-17T22:43:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9122999999999998e-05 2024-09-17T22:43:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452862+0000", 2024-09-17T22:43:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1019000000000001e-05 2024-09-17T22:43:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.720 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186753 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:43:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.269359855999994, 2024-09-17T22:43:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029553297999999999, 2024-09-17T22:43:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.721 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186753 2024-09-17T22:43:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:43:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:43:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423411+0000", 2024-09-17T22:43:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.98e-07 2024-09-17T22:43:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423412+0000", 2024-09-17T22:43:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8700000000000004e-07 2024-09-17T22:43:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423415+0000", 2024-09-17T22:43:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-09-17T22:43:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423418+0000", 2024-09-17T22:43:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.283e-06 2024-09-17T22:43:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452893+0000", 2024-09-17T22:43:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029475881999999998 2024-09-17T22:43:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452928+0000", 2024-09-17T22:43:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4109000000000002e-05 2024-09-17T22:43:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452964+0000", 2024-09-17T22:43:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6937000000000002e-05 2024-09-17T22:43:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186754 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12410880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:43:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.269350719000002, 2024-09-17T22:43:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029656021000000001, 2024-09-17T22:43:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.725 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186754 2024-09-17T22:43:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:43:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423420+0000", 2024-09-17T22:43:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423421+0000", 2024-09-17T22:43:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9999999999999999e-07 2024-09-17T22:43:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423425+0000", 2024-09-17T22:43:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6519999999999997e-06 2024-09-17T22:43:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423428+0000", 2024-09-17T22:43:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1370000000000002e-06 2024-09-17T22:43:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423431+0000", 2024-09-17T22:43:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4090000000000001e-06 2024-09-17T22:43:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.452992+0000", 2024-09-17T22:43:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029561545000000002 2024-09-17T22:43:16.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453031+0000", 2024-09-17T22:43:16.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8834000000000001e-05 2024-09-17T22:43:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453076+0000", 2024-09-17T22:43:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5244000000000003e-05 2024-09-17T22:43:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:186755 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 12500992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:43:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.269337437999994, 2024-09-17T22:43:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029745649999999998, 2024-09-17T22:43:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.731 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 186755 2024-09-17T22:43:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:43:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:43:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423434+0000", 2024-09-17T22:43:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.17e-07 2024-09-17T22:43:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423435+0000", 2024-09-17T22:43:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5499999999999996e-07 2024-09-17T22:43:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423438+0000", 2024-09-17T22:43:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8210000000000002e-06 2024-09-17T22:43:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.423443+0000", 2024-09-17T22:43:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1070000000000004e-06 2024-09-17T22:43:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453103+0000", 2024-09-17T22:43:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029660399 2024-09-17T22:43:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453141+0000", 2024-09-17T22:43:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.748e-05 2024-09-17T22:43:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.453179+0000", 2024-09-17T22:43:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8671000000000003e-05 2024-09-17T22:43:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187439 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8437760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:43:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.151900341000001, 2024-09-17T22:43:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029278048000000001, 2024-09-17T22:43:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:16.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187439 2024-09-17T22:43:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:43:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:43:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540871+0000", 2024-09-17T22:43:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6199999999999999e-07 2024-09-17T22:43:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540872+0000", 2024-09-17T22:43:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.09e-07 2024-09-17T22:43:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540875+0000", 2024-09-17T22:43:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8990000000000001e-06 2024-09-17T22:43:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540877+0000", 2024-09-17T22:43:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1789999999999998e-06 2024-09-17T22:43:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570074+0000", 2024-09-17T22:43:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029197358 2024-09-17T22:43:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570111+0000", 2024-09-17T22:43:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6170000000000001e-05 2024-09-17T22:43:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570149+0000", 2024-09-17T22:43:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8170999999999998e-05 2024-09-17T22:43:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187440 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8527872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:16.739 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:43:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.151884870999993, 2024-09-17T22:43:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029359758999999999, 2024-09-17T22:43:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187440 2024-09-17T22:43:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:43:16.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:43:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540886+0000", 2024-09-17T22:43:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2700000000000001e-07 2024-09-17T22:43:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540888+0000", 2024-09-17T22:43:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.313e-06 2024-09-17T22:43:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540893+0000", 2024-09-17T22:43:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9110000000000001e-06 2024-09-17T22:43:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540895+0000", 2024-09-17T22:43:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.712e-06 2024-09-17T22:43:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570168+0000", 2024-09-17T22:43:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029273114999999999 2024-09-17T22:43:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570208+0000", 2024-09-17T22:43:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9227999999999998e-05 2024-09-17T22:43:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570246+0000", 2024-09-17T22:43:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8253000000000001e-05 2024-09-17T22:43:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.743 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187441 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8617984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:43:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.151871907, 2024-09-17T22:43:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029442211999999999, 2024-09-17T22:43:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:16.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187441 2024-09-17T22:43:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:43:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:43:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540899+0000", 2024-09-17T22:43:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1400000000000001e-07 2024-09-17T22:43:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540900+0000", 2024-09-17T22:43:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7999999999999993e-07 2024-09-17T22:43:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540903+0000", 2024-09-17T22:43:16.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9129999999999999e-06 2024-09-17T22:43:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540909+0000", 2024-09-17T22:43:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4269999999999999e-06 2024-09-17T22:43:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570267+0000", 2024-09-17T22:43:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029358308 2024-09-17T22:43:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570304+0000", 2024-09-17T22:43:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6860999999999998e-05 2024-09-17T22:43:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570341+0000", 2024-09-17T22:43:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7509000000000003e-05 2024-09-17T22:43:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187442 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8708096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:43:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.151859200000004, 2024-09-17T22:43:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029522606999999999, 2024-09-17T22:43:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.749 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187442 2024-09-17T22:43:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:43:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:43:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540912+0000", 2024-09-17T22:43:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.22e-07 2024-09-17T22:43:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540917+0000", 2024-09-17T22:43:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999998e-06 2024-09-17T22:43:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540920+0000", 2024-09-17T22:43:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4989999999999999e-06 2024-09-17T22:43:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540923+0000", 2024-09-17T22:43:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3870000000000002e-06 2024-09-17T22:43:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570361+0000", 2024-09-17T22:43:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029438345000000001 2024-09-17T22:43:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570397+0000", 2024-09-17T22:43:16.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5429000000000002e-05 2024-09-17T22:43:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570434+0000", 2024-09-17T22:43:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7824999999999998e-05 2024-09-17T22:43:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187443 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8798208~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:43:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.151845273000006, 2024-09-17T22:43:16.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029603640000000001, 2024-09-17T22:43:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187443 2024-09-17T22:43:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:43:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:43:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540926+0000", 2024-09-17T22:43:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.05e-07 2024-09-17T22:43:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540927+0000", 2024-09-17T22:43:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8299999999999995e-07 2024-09-17T22:43:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540930+0000", 2024-09-17T22:43:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9229999999999998e-06 2024-09-17T22:43:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540932+0000", 2024-09-17T22:43:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.294e-06 2024-09-17T22:43:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570454+0000", 2024-09-17T22:43:16.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029522230999999999 2024-09-17T22:43:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570491+0000", 2024-09-17T22:43:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6835999999999998e-05 2024-09-17T22:43:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570529+0000", 2024-09-17T22:43:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8167999999999998e-05 2024-09-17T22:43:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.757 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.758 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.758 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187444 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8888320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:16.758 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:43:16.758 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.151835798999997, 2024-09-17T22:43:16.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029687584999999999, 2024-09-17T22:43:16.758 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.758 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.758 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:16.758 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.758 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:16.758 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.759 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187444 2024-09-17T22:43:16.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.759 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:43:16.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:43:16.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540935+0000", 2024-09-17T22:43:16.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-09-17T22:43:16.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540936+0000", 2024-09-17T22:43:16.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3900000000000003e-07 2024-09-17T22:43:16.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540939+0000", 2024-09-17T22:43:16.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.994e-06 2024-09-17T22:43:16.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540942+0000", 2024-09-17T22:43:16.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1739999999999999e-06 2024-09-17T22:43:16.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570549+0000", 2024-09-17T22:43:16.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029607652000000002 2024-09-17T22:43:16.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570585+0000", 2024-09-17T22:43:16.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5775000000000002e-05 2024-09-17T22:43:16.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570623+0000", 2024-09-17T22:43:16.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7839999999999997e-05 2024-09-17T22:43:16.762 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.762 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.763 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.763 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187445 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 8978432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:16.763 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:43:16.763 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.151826911000001, 2024-09-17T22:43:16.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029772837999999999, 2024-09-17T22:43:16.763 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.763 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.763 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:16.763 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.763 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:16.763 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.763 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187445 2024-09-17T22:43:16.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.764 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:43:16.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:43:16.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540944+0000", 2024-09-17T22:43:16.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:43:16.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540945+0000", 2024-09-17T22:43:16.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3699999999999999e-07 2024-09-17T22:43:16.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540948+0000", 2024-09-17T22:43:16.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.898e-06 2024-09-17T22:43:16.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540950+0000", 2024-09-17T22:43:16.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2249999999999999e-06 2024-09-17T22:43:16.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570643+0000", 2024-09-17T22:43:16.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029692463999999998 2024-09-17T22:43:16.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570680+0000", 2024-09-17T22:43:16.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6772e-05 2024-09-17T22:43:16.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570717+0000", 2024-09-17T22:43:16.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7438999999999998e-05 2024-09-17T22:43:16.767 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.768 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.768 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.768 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187446 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9068544~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:16.768 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:43:16.768 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.151818172999995, 2024-09-17T22:43:16.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029856321000000002, 2024-09-17T22:43:16.768 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.769 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.769 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:16.769 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.769 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:16.769 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.769 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187446 2024-09-17T22:43:16.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.769 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:43:16.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:43:16.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540953+0000", 2024-09-17T22:43:16.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-09-17T22:43:16.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540958+0000", 2024-09-17T22:43:16.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3449999999999996e-06 2024-09-17T22:43:16.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540962+0000", 2024-09-17T22:43:16.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3529999999999999e-06 2024-09-17T22:43:16.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540964+0000", 2024-09-17T22:43:16.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3810000000000002e-06 2024-09-17T22:43:16.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570737+0000", 2024-09-17T22:43:16.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029772824 2024-09-17T22:43:16.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570770+0000", 2024-09-17T22:43:16.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2835e-05 2024-09-17T22:43:16.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570809+0000", 2024-09-17T22:43:16.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9379999999999999e-05 2024-09-17T22:43:16.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.773 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.773 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187447 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9158656~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:16.773 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:43:16.773 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.151804205000005, 2024-09-17T22:43:16.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02993262, 2024-09-17T22:43:16.773 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.773 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.773 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:16.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:16.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.774 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187447 2024-09-17T22:43:16.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.774 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:43:16.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:43:16.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540967+0000", 2024-09-17T22:43:16.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-09-17T22:43:16.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540968+0000", 2024-09-17T22:43:16.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6500000000000008e-07 2024-09-17T22:43:16.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540971+0000", 2024-09-17T22:43:16.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9869999999999999e-06 2024-09-17T22:43:16.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540973+0000", 2024-09-17T22:43:16.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3120000000000001e-06 2024-09-17T22:43:16.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570829+0000", 2024-09-17T22:43:16.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029855742000000001 2024-09-17T22:43:16.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570861+0000", 2024-09-17T22:43:16.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2152e-05 2024-09-17T22:43:16.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570899+0000", 2024-09-17T22:43:16.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8241999999999997e-05 2024-09-17T22:43:16.777 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.777 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.778 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.778 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187448 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9248768~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:16.778 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:43:16.778 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.151795175000004, 2024-09-17T22:43:16.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030014865000000002, 2024-09-17T22:43:16.778 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.778 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.778 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:16.778 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.778 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:16.778 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.779 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187448 2024-09-17T22:43:16.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.779 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:43:16.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:43:16.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540976+0000", 2024-09-17T22:43:16.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9600000000000001e-07 2024-09-17T22:43:16.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540977+0000", 2024-09-17T22:43:16.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.16e-07 2024-09-17T22:43:16.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540980+0000", 2024-09-17T22:43:16.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.892e-06 2024-09-17T22:43:16.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540982+0000", 2024-09-17T22:43:16.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3410000000000001e-06 2024-09-17T22:43:16.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570919+0000", 2024-09-17T22:43:16.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.029936800999999999 2024-09-17T22:43:16.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570953+0000", 2024-09-17T22:43:16.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3939000000000002e-05 2024-09-17T22:43:16.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.570991+0000", 2024-09-17T22:43:16.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7780000000000001e-05 2024-09-17T22:43:16.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.782 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.782 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187449 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9338880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:16.783 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:43:16.783 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.151785996000001, 2024-09-17T22:43:16.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030110238000000001, 2024-09-17T22:43:16.783 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.783 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.783 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:16.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:16.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.783 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187449 2024-09-17T22:43:16.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.783 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:43:16.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:43:16.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540985+0000", 2024-09-17T22:43:16.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1500000000000001e-07 2024-09-17T22:43:16.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540986+0000", 2024-09-17T22:43:16.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9500000000000001e-07 2024-09-17T22:43:16.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540989+0000", 2024-09-17T22:43:16.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7949999999999999e-06 2024-09-17T22:43:16.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540994+0000", 2024-09-17T22:43:16.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0359999999999997e-06 2024-09-17T22:43:16.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571011+0000", 2024-09-17T22:43:16.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030016915000000002 2024-09-17T22:43:16.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571056+0000", 2024-09-17T22:43:16.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5457999999999997e-05 2024-09-17T22:43:16.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571095+0000", 2024-09-17T22:43:16.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8924000000000002e-05 2024-09-17T22:43:16.787 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.787 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.787 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.787 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:187450 3.1a 3:5e609aa1:::rbd_data.115f65a1a5f0.0000000000000005:1a [read 9428992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T22:43:16.787 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:43:16.787 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 68.151774348999993, 2024-09-17T22:43:16.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030188204999999999, 2024-09-17T22:43:16.787 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:16.788 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:16.788 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:16.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:16.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:16.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:16.788 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 187450 2024-09-17T22:43:16.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.788 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:16.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:16.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:43:16.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:16.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:43:16.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:16.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:16.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540997+0000", 2024-09-17T22:43:16.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.16e-07 2024-09-17T22:43:16.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:16.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.540998+0000", 2024-09-17T22:43:16.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9000000000000005e-07 2024-09-17T22:43:16.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:16.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.541001+0000", 2024-09-17T22:43:16.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0680000000000001e-06 2024-09-17T22:43:16.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:16.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.541006+0000", 2024-09-17T22:43:16.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4720000000000003e-06 2024-09-17T22:43:16.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:16.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571110+0000", 2024-09-17T22:43:16.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.030103431 2024-09-17T22:43:16.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:16.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571147+0000", 2024-09-17T22:43:16.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6748999999999997e-05 2024-09-17T22:43:16.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:16.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:16.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:16.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:42:08.571185+0000", 2024-09-17T22:43:16.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8278999999999997e-05 2024-09-17T22:43:16.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.791 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:16.792 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:16.792 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:16.793 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T22:43:16.794 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:16.790+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:16.894 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:16.891+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:16.937 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:16.937 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:43:16.937 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:43:16.937 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:16.945 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T22:43:16.995 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:43:16.993+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:43:17.089 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:17.089 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:43:17.089 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-09-17T22:43:17.089 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-09-17T22:43:17.089 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:17.096 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:17.094+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:17.097 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T22:43:17.198 INFO:tasks.ceph.osd.4.smithi129.stderr:2024-09-17T22:43:17.195+0000 7f3eadb54640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33769) UID: 0 2024-09-17T22:43:17.260 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-09-17T22:43:17.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-09-17T22:43:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-09-17T22:43:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145246 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3850240~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:43:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.927526830000005, 2024-09-17T22:43:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052644991000000002, 2024-09-17T22:43:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145246 2024-09-17T22:43:17.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:43:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329898+0000", 2024-09-17T22:43:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329899+0000", 2024-09-17T22:43:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-09-17T22:43:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329901+0000", 2024-09-17T22:43:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1619999999999998e-06 2024-09-17T22:43:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329907+0000", 2024-09-17T22:43:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9059999999999999e-06 2024-09-17T22:43:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329911+0000", 2024-09-17T22:43:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7419999999999997e-06 2024-09-17T22:43:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382383+0000", 2024-09-17T22:43:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052471583000000002 2024-09-17T22:43:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382457+0000", 2024-09-17T22:43:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4309999999999998e-05 2024-09-17T22:43:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382543+0000", 2024-09-17T22:43:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5918e-05 2024-09-17T22:43:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145247 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3946496~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:43:17.265 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.927508180000004, 2024-09-17T22:43:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052805863000000001, 2024-09-17T22:43:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145247 2024-09-17T22:43:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:43:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:43:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329917+0000", 2024-09-17T22:43:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3200000000000001e-07 2024-09-17T22:43:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329919+0000", 2024-09-17T22:43:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.888e-06 2024-09-17T22:43:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329925+0000", 2024-09-17T22:43:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5369999999999997e-06 2024-09-17T22:43:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329929+0000", 2024-09-17T22:43:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.639e-06 2024-09-17T22:43:17.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382570+0000", 2024-09-17T22:43:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052641213999999999 2024-09-17T22:43:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382636+0000", 2024-09-17T22:43:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6008999999999998e-05 2024-09-17T22:43:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382723+0000", 2024-09-17T22:43:17.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6243999999999996e-05 2024-09-17T22:43:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145248 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3856384~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:43:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.927490132000003, 2024-09-17T22:43:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053002858, 2024-09-17T22:43:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.270 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145248 2024-09-17T22:43:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:43:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:43:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329935+0000", 2024-09-17T22:43:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.13e-07 2024-09-17T22:43:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329937+0000", 2024-09-17T22:43:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9300000000000002e-06 2024-09-17T22:43:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329943+0000", 2024-09-17T22:43:17.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-09-17T22:43:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329948+0000", 2024-09-17T22:43:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8579999999999999e-06 2024-09-17T22:43:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382763+0000", 2024-09-17T22:43:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052814537000000002 2024-09-17T22:43:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382840+0000", 2024-09-17T22:43:17.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6879999999999996e-05 2024-09-17T22:43:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382938+0000", 2024-09-17T22:43:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8159999999999995e-05 2024-09-17T22:43:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145249 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3952640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:43:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.927471230999998, 2024-09-17T22:43:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053188602000000001, 2024-09-17T22:43:17.274 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145249 2024-09-17T22:43:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:43:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:43:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329954+0000", 2024-09-17T22:43:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3700000000000001e-07 2024-09-17T22:43:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329956+0000", 2024-09-17T22:43:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1430000000000001e-06 2024-09-17T22:43:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329962+0000", 2024-09-17T22:43:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8270000000000003e-06 2024-09-17T22:43:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329967+0000", 2024-09-17T22:43:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7249999999999997e-06 2024-09-17T22:43:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.382969+0000", 2024-09-17T22:43:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053001952999999997 2024-09-17T22:43:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383040+0000", 2024-09-17T22:43:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1272999999999993e-05 2024-09-17T22:43:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383142+0000", 2024-09-17T22:43:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010234399999999999 2024-09-17T22:43:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145250 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3862528~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.278 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:43:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.927452431999995, 2024-09-17T22:43:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053359537999999998, 2024-09-17T22:43:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145250 2024-09-17T22:43:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:43:17.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:43:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329973+0000", 2024-09-17T22:43:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-09-17T22:43:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329975+0000", 2024-09-17T22:43:17.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2199999999999999e-06 2024-09-17T22:43:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329981+0000", 2024-09-17T22:43:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1020000000000002e-06 2024-09-17T22:43:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329991+0000", 2024-09-17T22:43:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6800000000000005e-06 2024-09-17T22:43:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383171+0000", 2024-09-17T22:43:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053180142 2024-09-17T22:43:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383242+0000", 2024-09-17T22:43:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1310000000000007e-05 2024-09-17T22:43:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383332+0000", 2024-09-17T22:43:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9703999999999994e-05 2024-09-17T22:43:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.282 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145251 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3958784~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:43:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.927426732000001, 2024-09-17T22:43:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053438537000000001, 2024-09-17T22:43:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145251 2024-09-17T22:43:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.283 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:43:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329998+0000", 2024-09-17T22:43:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.329999+0000", 2024-09-17T22:43:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-07 2024-09-17T22:43:17.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330001+0000", 2024-09-17T22:43:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2409999999999998e-06 2024-09-17T22:43:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330006+0000", 2024-09-17T22:43:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000004e-06 2024-09-17T22:43:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330011+0000", 2024-09-17T22:43:17.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4039999999999997e-06 2024-09-17T22:43:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383350+0000", 2024-09-17T22:43:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053339011999999998 2024-09-17T22:43:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383387+0000", 2024-09-17T22:43:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7085999999999997e-05 2024-09-17T22:43:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383437+0000", 2024-09-17T22:43:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9988000000000003e-05 2024-09-17T22:43:17.286 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145252 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3868672~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:43:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.927408284999999, 2024-09-17T22:43:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.053526724999999997, 2024-09-17T22:43:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.287 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145252 2024-09-17T22:43:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:43:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.330017+0000", 2024-09-17T22:43:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331054+0000", 2024-09-17T22:43:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010371670000000001 2024-09-17T22:43:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331059+0000", 2024-09-17T22:43:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1680000000000003e-06 2024-09-17T22:43:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331075+0000", 2024-09-17T22:43:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6028e-05 2024-09-17T22:43:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331084+0000", 2024-09-17T22:43:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3980000000000005e-06 2024-09-17T22:43:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383464+0000", 2024-09-17T22:43:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052379851999999998 2024-09-17T22:43:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383499+0000", 2024-09-17T22:43:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4610000000000002e-05 2024-09-17T22:43:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383543+0000", 2024-09-17T22:43:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4502000000000002e-05 2024-09-17T22:43:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145255 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3971072~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:43:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.926278193000002, 2024-09-17T22:43:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052677527000000002, 2024-09-17T22:43:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.291 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145255 2024-09-17T22:43:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:43:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:43:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331147+0000", 2024-09-17T22:43:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8800000000000002e-07 2024-09-17T22:43:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331150+0000", 2024-09-17T22:43:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1160000000000002e-06 2024-09-17T22:43:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331156+0000", 2024-09-17T22:43:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7469999999999999e-06 2024-09-17T22:43:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331162+0000", 2024-09-17T22:43:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5110000000000003e-06 2024-09-17T22:43:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383750+0000", 2024-09-17T22:43:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052588441999999999 2024-09-17T22:43:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383782+0000", 2024-09-17T22:43:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1610999999999999e-05 2024-09-17T22:43:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383824+0000", 2024-09-17T22:43:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2512000000000002e-05 2024-09-17T22:43:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145256 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3880960~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:43:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.926256772000002, 2024-09-17T22:43:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052744775000000001, 2024-09-17T22:43:17.295 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145256 2024-09-17T22:43:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:43:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331168+0000", 2024-09-17T22:43:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331169+0000", 2024-09-17T22:43:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-09-17T22:43:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331175+0000", 2024-09-17T22:43:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5660000000000003e-06 2024-09-17T22:43:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331182+0000", 2024-09-17T22:43:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0169999999999999e-06 2024-09-17T22:43:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331188+0000", 2024-09-17T22:43:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2750000000000003e-06 2024-09-17T22:43:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383839+0000", 2024-09-17T22:43:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052651728000000002 2024-09-17T22:43:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383872+0000", 2024-09-17T22:43:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2347e-05 2024-09-17T22:43:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383913+0000", 2024-09-17T22:43:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1362000000000001e-05 2024-09-17T22:43:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:145257 3.16 3:6f7fd393:::rbd_data.115f14f2d1d5.0000000000000000:e [read 3977216~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:43:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.926230724000007, 2024-09-17T22:43:17.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052809598999999999, 2024-09-17T22:43:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 145257 2024-09-17T22:43:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:43:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.331194+0000", 2024-09-17T22:43:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333051+0000", 2024-09-17T22:43:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018567189999999999 2024-09-17T22:43:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333056+0000", 2024-09-17T22:43:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.498e-06 2024-09-17T22:43:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333067+0000", 2024-09-17T22:43:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1704e-05 2024-09-17T22:43:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.333074+0000", 2024-09-17T22:43:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1280000000000002e-06 2024-09-17T22:43:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383927+0000", 2024-09-17T22:43:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050853094000000001 2024-09-17T22:43:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.383960+0000", 2024-09-17T22:43:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2891000000000001e-05 2024-09-17T22:43:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.384004+0000", 2024-09-17T22:43:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3565e-05 2024-09-17T22:43:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146168 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2467840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.303 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:43:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.862742103000002, 2024-09-17T22:43:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052652501999999997, 2024-09-17T22:43:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146168 2024-09-17T22:43:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:43:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394683+0000", 2024-09-17T22:43:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394989+0000", 2024-09-17T22:43:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030618300000000002 2024-09-17T22:43:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.394994+0000", 2024-09-17T22:43:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.566e-06 2024-09-17T22:43:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395003+0000", 2024-09-17T22:43:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3619999999999995e-06 2024-09-17T22:43:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395009+0000", 2024-09-17T22:43:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3589999999999996e-06 2024-09-17T22:43:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447180+0000", 2024-09-17T22:43:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052170643000000003 2024-09-17T22:43:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447251+0000", 2024-09-17T22:43:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1382000000000002e-05 2024-09-17T22:43:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447335+0000", 2024-09-17T22:43:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4006999999999997e-05 2024-09-17T22:43:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146172 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2480128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:43:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.862387217999995, 2024-09-17T22:43:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052714629999999998, 2024-09-17T22:43:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146172 2024-09-17T22:43:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.308 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:43:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395038+0000", 2024-09-17T22:43:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395486+0000", 2024-09-17T22:43:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00044787000000000001 2024-09-17T22:43:17.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395491+0000", 2024-09-17T22:43:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8550000000000001e-06 2024-09-17T22:43:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395503+0000", 2024-09-17T22:43:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2271e-05 2024-09-17T22:43:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395510+0000", 2024-09-17T22:43:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6320000000000008e-06 2024-09-17T22:43:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447565+0000", 2024-09-17T22:43:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052055016000000003 2024-09-17T22:43:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447644+0000", 2024-09-17T22:43:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8306000000000005e-05 2024-09-17T22:43:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.447752+0000", 2024-09-17T22:43:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010868 2024-09-17T22:43:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.311 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146181 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2504704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:43:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.861519827999999, 2024-09-17T22:43:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052850394000000002, 2024-09-17T22:43:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146181 2024-09-17T22:43:17.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:43:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.395905+0000", 2024-09-17T22:43:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396210+0000", 2024-09-17T22:43:17.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000305214 2024-09-17T22:43:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396214+0000", 2024-09-17T22:43:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9389999999999999e-06 2024-09-17T22:43:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396228+0000", 2024-09-17T22:43:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3365e-05 2024-09-17T22:43:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396235+0000", 2024-09-17T22:43:17.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2559999999999996e-06 2024-09-17T22:43:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448568+0000", 2024-09-17T22:43:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052333084000000002 2024-09-17T22:43:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448646+0000", 2024-09-17T22:43:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8287000000000004e-05 2024-09-17T22:43:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448756+0000", 2024-09-17T22:43:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000109249 2024-09-17T22:43:17.315 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146183 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2510848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:43:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.861179281000005, 2024-09-17T22:43:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052725453999999998, 2024-09-17T22:43:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.316 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146183 2024-09-17T22:43:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:43:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396246+0000", 2024-09-17T22:43:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396508+0000", 2024-09-17T22:43:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026271299999999998 2024-09-17T22:43:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396512+0000", 2024-09-17T22:43:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8039999999999999e-06 2024-09-17T22:43:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396522+0000", 2024-09-17T22:43:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9289999999999993e-06 2024-09-17T22:43:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.396528+0000", 2024-09-17T22:43:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2990000000000002e-06 2024-09-17T22:43:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448798+0000", 2024-09-17T22:43:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052269985999999997 2024-09-17T22:43:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448871+0000", 2024-09-17T22:43:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2581000000000002e-05 2024-09-17T22:43:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.448971+0000", 2024-09-17T22:43:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100142 2024-09-17T22:43:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146391 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:43:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.853451789000005, 2024-09-17T22:43:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052666719000000001, 2024-09-17T22:43:17.320 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146391 2024-09-17T22:43:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:43:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.403973+0000", 2024-09-17T22:43:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404098+0000", 2024-09-17T22:43:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000124509 2024-09-17T22:43:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404107+0000", 2024-09-17T22:43:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4220000000000006e-06 2024-09-17T22:43:17.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404117+0000", 2024-09-17T22:43:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.011e-05 2024-09-17T22:43:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404123+0000", 2024-09-17T22:43:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1530000000000002e-06 2024-09-17T22:43:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456481+0000", 2024-09-17T22:43:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052357923000000001 2024-09-17T22:43:17.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456551+0000", 2024-09-17T22:43:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9869000000000005e-05 2024-09-17T22:43:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456640+0000", 2024-09-17T22:43:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8733000000000006e-05 2024-09-17T22:43:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146393 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2748416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:43:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.853291725999995, 2024-09-17T22:43:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052739239, 2024-09-17T22:43:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146393 2024-09-17T22:43:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.325 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:43:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404133+0000", 2024-09-17T22:43:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404443+0000", 2024-09-17T22:43:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030977199999999999 2024-09-17T22:43:17.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404447+0000", 2024-09-17T22:43:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6370000000000001e-06 2024-09-17T22:43:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404469+0000", 2024-09-17T22:43:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2739999999999999e-05 2024-09-17T22:43:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404477+0000", 2024-09-17T22:43:17.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6559999999999992e-06 2024-09-17T22:43:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456734+0000", 2024-09-17T22:43:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052256889000000001 2024-09-17T22:43:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456796+0000", 2024-09-17T22:43:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2485999999999998e-05 2024-09-17T22:43:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.456873+0000", 2024-09-17T22:43:17.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6058999999999998e-05 2024-09-17T22:43:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146397 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2760704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:43:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.852915961999997, 2024-09-17T22:43:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052788928999999998, 2024-09-17T22:43:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146397 2024-09-17T22:43:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:43:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404509+0000", 2024-09-17T22:43:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404763+0000", 2024-09-17T22:43:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00025351399999999999 2024-09-17T22:43:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404767+0000", 2024-09-17T22:43:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2180000000000001e-06 2024-09-17T22:43:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404777+0000", 2024-09-17T22:43:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9310000000000003e-06 2024-09-17T22:43:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404783+0000", 2024-09-17T22:43:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.336e-06 2024-09-17T22:43:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457126+0000", 2024-09-17T22:43:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052342975999999999 2024-09-17T22:43:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457200+0000", 2024-09-17T22:43:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4146000000000005e-05 2024-09-17T22:43:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457298+0000", 2024-09-17T22:43:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7807999999999996e-05 2024-09-17T22:43:17.333 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146399 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2766848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:43:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.852625821000004, 2024-09-17T22:43:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052829779, 2024-09-17T22:43:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146399 2024-09-17T22:43:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:43:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404799+0000", 2024-09-17T22:43:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404800+0000", 2024-09-17T22:43:17.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8500000000000001e-07 2024-09-17T22:43:17.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404802+0000", 2024-09-17T22:43:17.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5349999999999999e-06 2024-09-17T22:43:17.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404811+0000", 2024-09-17T22:43:17.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9390000000000003e-06 2024-09-17T22:43:17.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404817+0000", 2024-09-17T22:43:17.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.524e-06 2024-09-17T22:43:17.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457461+0000", 2024-09-17T22:43:17.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052643784999999998 2024-09-17T22:43:17.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457539+0000", 2024-09-17T22:43:17.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8589000000000003e-05 2024-09-17T22:43:17.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457629+0000", 2024-09-17T22:43:17.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9821999999999995e-05 2024-09-17T22:43:17.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.338 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.338 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146401 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2772992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.338 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:43:17.338 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.852600573999993, 2024-09-17T22:43:17.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052910077, 2024-09-17T22:43:17.339 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.339 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.339 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.339 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146401 2024-09-17T22:43:17.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.339 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:43:17.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.404824+0000", 2024-09-17T22:43:17.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405106+0000", 2024-09-17T22:43:17.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000281286 2024-09-17T22:43:17.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405110+0000", 2024-09-17T22:43:17.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1459999999999998e-06 2024-09-17T22:43:17.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405120+0000", 2024-09-17T22:43:17.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0546e-05 2024-09-17T22:43:17.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405127+0000", 2024-09-17T22:43:17.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4300000000000003e-06 2024-09-17T22:43:17.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457647+0000", 2024-09-17T22:43:17.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052520229000000002 2024-09-17T22:43:17.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457684+0000", 2024-09-17T22:43:17.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6609999999999997e-05 2024-09-17T22:43:17.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457735+0000", 2024-09-17T22:43:17.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.083e-05 2024-09-17T22:43:17.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.342 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.343 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.343 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4447.0:146402 3.e 3:734467c7:::rbd_data.115f14f2d1d5.0000000000000002:e [read 2779136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-17T22:43:17.343 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:43:17.343 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 87.852287161999996, 2024-09-17T22:43:17.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052689812000000003, 2024-09-17T22:43:17.344 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-09-17T22:43:17.344 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-09-17T22:43:17.344 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-09-17T22:43:17.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-09-17T22:43:17.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4447", 2024-09-17T22:43:17.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.115:0/4149151034", 2024-09-17T22:43:17.345 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 146402 2024-09-17T22:43:17.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.345 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-09-17T22:43:17.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-09-17T22:43:17.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:43:17.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-09-17T22:43:17.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405138+0000", 2024-09-17T22:43:17.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-09-17T22:43:17.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-09-17T22:43:17.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405365+0000", 2024-09-17T22:43:17.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00022744599999999999 2024-09-17T22:43:17.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-09-17T22:43:17.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405370+0000", 2024-09-17T22:43:17.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2769999999999999e-06 2024-09-17T22:43:17.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-09-17T22:43:17.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405380+0000", 2024-09-17T22:43:17.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0349e-05 2024-09-17T22:43:17.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-09-17T22:43:17.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.405386+0000", 2024-09-17T22:43:17.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5169999999999996e-06 2024-09-17T22:43:17.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-09-17T22:43:17.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457750+0000", 2024-09-17T22:43:17.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.052363448999999999 2024-09-17T22:43:17.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-09-17T22:43:17.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457784+0000", 2024-09-17T22:43:17.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4363999999999998e-05 2024-09-17T22:43:17.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-09-17T22:43:17.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-09-17T22:43:17.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-09-17T22:43:17.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-09-17T22:41:49.457828+0000", 2024-09-17T22:43:17.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3409999999999999e-05 2024-09-17T22:43:17.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.348 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-09-17T22:43:17.348 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-17T22:43:17.348 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:17.349 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T22:43:17.350 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:17.347+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:17.405 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T22:43:17.450 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:17.447+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:17.455 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:17.455 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:43:17.455 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:43:17.456 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:17.465 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T22:43:17.510 INFO:teuthology.orchestra.run.smithi115.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-17T22:43:17.527 INFO:teuthology.orchestra.run.smithi115.stdout:osd.1: {} 2024-09-17T22:43:17.528 INFO:teuthology.orchestra.run.smithi115.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T22:43:17.545 INFO:teuthology.orchestra.run.smithi115.stdout:osd.2: {} 2024-09-17T22:43:17.545 INFO:teuthology.orchestra.run.smithi115.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T22:43:17.551 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:17.549+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:17.561 INFO:teuthology.orchestra.run.smithi115.stdout:osd.3: {} 2024-09-17T22:43:17.561 INFO:teuthology.orchestra.run.smithi115.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T22:43:17.579 INFO:teuthology.orchestra.run.smithi115.stdout:osd.4: {} 2024-09-17T22:43:17.579 INFO:teuthology.orchestra.run.smithi115.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T22:43:17.595 INFO:teuthology.orchestra.run.smithi115.stdout:osd.5: {} 2024-09-17T22:43:17.596 INFO:teuthology.orchestra.run.smithi115.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T22:43:17.610 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:17.610 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:17.612 INFO:teuthology.orchestra.run.smithi115.stdout:osd.6: {} 2024-09-17T22:43:17.613 INFO:teuthology.orchestra.run.smithi115.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-17T22:43:17.619 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T22:43:17.634 INFO:teuthology.orchestra.run.smithi115.stdout:osd.7: {} 2024-09-17T22:43:17.635 INFO:teuthology.orchestra.run.smithi115.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-17T22:43:17.637 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T22:43:17.638 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T22:43:17.678 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:17.676+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:17.736 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:17.742 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:17.746 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T22:43:17.790 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:17.788+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:17.854 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:17.851+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:17.854 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:17.855 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:17.864 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T22:43:17.955 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:17.953+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:18.006 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:18.006 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:18.015 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T22:43:18.056 INFO:tasks.ceph.osd.1.smithi115.stderr:2024-09-17T22:43:18.054+0000 7fc372ade640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33477) UID: 0 2024-09-17T22:43:18.118 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:18.118 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:18.126 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T22:43:18.169 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:18.167+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:18.243 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:18.243 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:18.248 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T22:43:18.253 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T22:43:18.294 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:18.292+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:18.359 INFO:tasks.ceph.osd.3.smithi115.stderr:2024-09-17T22:43:18.357+0000 7f253d616640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33484) UID: 0 2024-09-17T22:43:18.399 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:18.399 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:18.408 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T22:43:18.460 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:18.458+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:18.512 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:18.512 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:18.521 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-17T22:43:18.562 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:18.560+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:18.662 INFO:tasks.ceph.osd.5.smithi129.stderr:2024-09-17T22:43:18.659+0000 7f7976b33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33768) UID: 0 2024-09-17T22:43:18.665 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:18.665 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:18.674 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-17T22:43:18.762 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:18.759+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:18.816 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:18.816 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:18.825 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-17T22:43:18.863 INFO:tasks.ceph.osd.2.smithi115.stderr:2024-09-17T22:43:18.860+0000 7fc575554640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33485) UID: 0 2024-09-17T22:43:18.964 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:18.962+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:18.970 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:18.970 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:18.981 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-17T22:43:19.065 INFO:tasks.ceph.osd.7.smithi129.stderr:2024-09-17T22:43:19.063+0000 7f6688956640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33776) UID: 0 2024-09-17T22:43:19.124 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:19.124 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:19.124 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5, 6, 7] 2024-09-17T22:43:19.124 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T22:43:19.125 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:43:19.133 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-17T22:43:19.146 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is set 2024-09-17T22:43:19.174 INFO:tasks.ceph.osd.6.smithi129.stderr:2024-09-17T22:43:19.171+0000 7f2b01295640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33774) UID: 0 2024-09-17T22:43:19.229 INFO:teuthology.orchestra.run.smithi115.stdout:All operations completed A-OK! 2024-09-17T22:43:19.242 DEBUG:teuthology.parallel:result is None 2024-09-17T22:43:19.243 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-17T22:43:19.256 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-17T22:43:19.306 INFO:tasks.thrashosds:joining thrashosds 2024-09-17T22:43:19.307 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:19.307 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:19.307 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-17T22:43:19.411 INFO:teuthology.orchestra.run.smithi129.stdout:ERROR: (22) Invalid argument 2024-09-17T22:43:19.411 INFO:teuthology.orchestra.run.smithi129.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T22:43:19.415 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:43:19.415 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:43:19.427 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-09-17T22:43:19.140658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":237267545,"num_objects":881,"num_object_clones":172,"num_object_copies":1762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":337,"num_objects_misplaced":45,"num_objects_unfound":0,"num_objects_dirty":881,"num_whiteouts":664,"num_read":267598,"num_read_kb":801794,"num_write":18154,"num_write_kb":296086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":14661402,"num_keys_recovered":86,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16242,"ondisk_log_size":16242,"up":71,"acting":73,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":337812,"kb_used_data":21652,"kb_used_omap":12,"kb_used_meta":315955,"kb_avail":754636908,"statfs":{"total":773094113280,"available":772748193792,"internally_reserved":0,"allocated":22171648,"data_stored":89613045,"data_compressed":1789380,"data_compressed_allocated":7565312,"data_compressed_original":82260613,"omap_allocated":12750,"internal_metadata":323538482},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-136798018,"num_objects":-152,"num_object_clones":-40,"num_object_copies":-304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-13,"num_objects_misplaced":-8,"num_objects_unfound":0,"num_objects_dirty":-152,"num_whiteouts":-38,"num_read":1307,"num_read_kb":1116,"num_write":254,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.030980"},"pg_stats":[{"pgid":"3.1f","version":"128'840","reported_seq":7169,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T22:43:16.227880+0000","last_change":"2024-09-17T22:43:16.147248+0000","last_active":"2024-09-17T22:43:16.227880+0000","last_peered":"2024-09-17T22:43:16.227880+0000","last_clean":"2024-09-17T22:43:16.227880+0000","last_became_active":"2024-09-17T22:41:19.443658+0000","last_became_peered":"2024-09-17T22:41:19.443658+0000","last_unstale":"2024-09-17T22:43:16.227880+0000","last_undegraded":"2024-09-17T22:43:16.227880+0000","last_fullsized":"2024-09-17T22:43:16.227880+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:30:18.909603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046461700000000001,"stat_sum":{"num_bytes":61456,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":14,"num_read":8713,"num_read_kb":14816,"num_write":1090,"num_write_kb":4547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6","length":"23"}]},{"pgid":"3.1e","version":"128'779","reported_seq":7347,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T22:43:16.147539+0000","last_change":"2024-09-17T22:43:16.147539+0000","last_active":"2024-09-17T22:43:16.147539+0000","last_peered":"2024-09-17T22:43:16.147539+0000","last_clean":"2024-09-17T22:43:16.147539+0000","last_became_active":"2024-09-17T22:41:19.449357+0000","last_became_peered":"2024-09-17T22:41:19.449357+0000","last_unstale":"2024-09-17T22:43:16.147539+0000","last_undegraded":"2024-09-17T22:43:16.147539+0000","last_fullsized":"2024-09-17T22:43:16.147539+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:42:42.662801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071612900000000003,"stat_sum":{"num_bytes":4301352,"num_objects":27,"num_object_clones":3,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":23,"num_read":8865,"num_read_kb":26537,"num_write":1125,"num_write_kb":10958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6","length":"23"}]},{"pgid":"3.1d","version":"129'434","reported_seq":2318,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T22:43:17.144144+0000","last_change":"2024-09-17T22:43:17.144668+0000","last_active":"2024-09-17T22:43:17.144144+0000","last_peered":"2024-09-17T22:43:17.144144+0000","last_clean":"2024-09-17T22:43:17.144144+0000","last_became_active":"2024-09-17T22:42:58.002822+0000","last_became_peered":"2024-09-17T22:42:58.002822+0000","last_unstale":"2024-09-17T22:43:17.144144+0000","last_undegraded":"2024-09-17T22:43:17.144144+0000","last_fullsized":"2024-09-17T22:43:17.144144+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:56:26.931656+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0082220369999999997,"stat_sum":{"num_bytes":3545871,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":17641,"num_read_kb":48444,"num_write":376,"num_write_kb":8987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":7308630,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"5","length":"24"}]},{"pgid":"3.1c","version":"129'492","reported_seq":2756,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T22:43:17.196551+0000","last_change":"2024-09-17T22:43:17.152897+0000","last_active":"2024-09-17T22:43:17.196551+0000","last_peered":"2024-09-17T22:43:17.196551+0000","last_clean":"2024-09-17T22:43:17.196551+0000","last_became_active":"2024-09-17T22:42:34.310411+0000","last_became_peered":"2024-09-17T22:42:34.310411+0000","last_unstale":"2024-09-17T22:43:17.196551+0000","last_undegraded":"2024-09-17T22:43:17.196551+0000","last_fullsized":"2024-09-17T22:43:17.196551+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:15:25.804298+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.016440315,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":2605,"num_read_kb":14205,"num_write":595,"num_write_kb":12172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":7352772,"num_keys_recovered":74,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"5","length":"24"}]},{"pgid":"3.1b","version":"128'216","reported_seq":7750,"reported_epoch":128,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:43:16.189097+0000","last_change":"2024-09-17T22:41:54.118027+0000","last_active":"2024-09-17T22:43:16.189097+0000","last_peered":"2024-09-17T22:43:16.189097+0000","last_clean":"2024-09-17T22:41:52.368521+0000","last_became_active":"2024-09-17T22:41:54.118027+0000","last_became_peered":"2024-09-17T22:41:54.118027+0000","last_unstale":"2024-09-17T22:43:16.189097+0000","last_undegraded":"2024-09-17T22:41:54.116518+0000","last_fullsized":"2024-09-17T22:41:54.116411+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":35,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:28:09.296183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055813,"stat_sum":{"num_bytes":8866746,"num_objects":40,"num_object_clones":14,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":26,"num_read":7315,"num_read_kb":22822,"num_write":186,"num_write_kb":6215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":40}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"129'342","reported_seq":5100,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T22:43:17.261592+0000","last_change":"2024-09-17T22:43:17.170793+0000","last_active":"2024-09-17T22:43:17.261592+0000","last_peered":"2024-09-17T22:43:17.261592+0000","last_clean":"2024-09-17T22:43:17.261592+0000","last_became_active":"2024-09-17T22:41:19.446541+0000","last_became_peered":"2024-09-17T22:41:19.446541+0000","last_unstale":"2024-09-17T22:43:17.261592+0000","last_undegraded":"2024-09-17T22:43:17.261592+0000","last_fullsized":"2024-09-17T22:43:17.261592+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:59:19.633058+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.033926234999999999,"stat_sum":{"num_bytes":1328589,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":27,"num_read":4978,"num_read_kb":19586,"num_write":317,"num_write_kb":4178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"5","length":"24"}]},{"pgid":"3.19","version":"126'381","reported_seq":6687,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T22:43:16.149215+0000","last_change":"2024-09-17T22:43:16.149664+0000","last_active":"2024-09-17T22:43:16.149215+0000","last_peered":"2024-09-17T22:43:16.149215+0000","last_clean":"2024-09-17T22:43:16.149215+0000","last_became_active":"2024-09-17T22:41:19.449940+0000","last_became_peered":"2024-09-17T22:41:19.449940+0000","last_unstale":"2024-09-17T22:43:16.149215+0000","last_undegraded":"2024-09-17T22:43:16.149215+0000","last_fullsized":"2024-09-17T22:43:16.149215+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:12:33.060905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018798759999999999,"stat_sum":{"num_bytes":13928484,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":20,"num_read":6666,"num_read_kb":22142,"num_write":343,"num_write_kb":9035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"23"}]},{"pgid":"3.18","version":"128'379","reported_seq":3149,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T22:43:16.190242+0000","last_change":"2024-09-17T22:43:16.148168+0000","last_active":"2024-09-17T22:43:16.190242+0000","last_peered":"2024-09-17T22:43:16.190242+0000","last_clean":"2024-09-17T22:43:16.190242+0000","last_became_active":"2024-09-17T22:41:19.446287+0000","last_became_peered":"2024-09-17T22:41:19.446287+0000","last_unstale":"2024-09-17T22:43:16.190242+0000","last_undegraded":"2024-09-17T22:43:16.190242+0000","last_fullsized":"2024-09-17T22:43:16.190242+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:11:32.776442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012131959999999999,"stat_sum":{"num_bytes":376832,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":18,"num_read":3441,"num_read_kb":20083,"num_write":323,"num_write_kb":10930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6","length":"23"}]},{"pgid":"3.17","version":"128'502","reported_seq":2674,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T22:43:16.188966+0000","last_change":"2024-09-17T22:43:16.147635+0000","last_active":"2024-09-17T22:43:16.188966+0000","last_peered":"2024-09-17T22:43:16.188966+0000","last_clean":"2024-09-17T22:43:16.188966+0000","last_became_active":"2024-09-17T22:41:19.445923+0000","last_became_peered":"2024-09-17T22:41:19.445923+0000","last_unstale":"2024-09-17T22:43:16.188966+0000","last_undegraded":"2024-09-17T22:43:16.188966+0000","last_fullsized":"2024-09-17T22:43:16.188966+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:47:23.173714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074870799999999995,"stat_sum":{"num_bytes":3202188,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":20,"num_read":3445,"num_read_kb":15105,"num_write":512,"num_write_kb":8714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6","length":"23"}]},{"pgid":"3.16","version":"127'532","reported_seq":22272,"reported_epoch":127,"state":"active+clean+remapped","last_fresh":"2024-09-17T22:43:15.219431+0000","last_change":"2024-09-17T22:43:15.115822+0000","last_active":"2024-09-17T22:43:15.219431+0000","last_peered":"2024-09-17T22:43:15.219431+0000","last_clean":"2024-09-17T22:43:15.219431+0000","last_became_active":"2024-09-17T22:41:57.132018+0000","last_became_peered":"2024-09-17T22:41:57.132018+0000","last_unstale":"2024-09-17T22:43:15.219431+0000","last_undegraded":"2024-09-17T22:43:15.219431+0000","last_fullsized":"2024-09-17T22:43:15.219431+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:44:35.370584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015384929999999999,"stat_sum":{"num_bytes":2445312,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":29,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":21,"num_read":22766,"num_read_kb":63950,"num_write":593,"num_write_kb":12651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":29}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"22"}]},{"pgid":"3.15","version":"119'412","reported_seq":7316,"reported_epoch":128,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:43:16.147467+0000","last_change":"2024-09-17T22:41:54.119655+0000","last_active":"2024-09-17T22:43:16.147467+0000","last_peered":"2024-09-17T22:43:16.147467+0000","last_clean":"2024-09-17T22:41:52.245886+0000","last_became_active":"2024-09-17T22:41:54.119655+0000","last_became_peered":"2024-09-17T22:41:54.119655+0000","last_unstale":"2024-09-17T22:43:16.147467+0000","last_undegraded":"2024-09-17T22:41:54.119161+0000","last_fullsized":"2024-09-17T22:41:54.119071+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":35,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:21:57.371666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041055799999999999,"stat_sum":{"num_bytes":61057766,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":42,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":28,"num_read":8763,"num_read_kb":20352,"num_write":340,"num_write_kb":4822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":42}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"126'580","reported_seq":7773,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-17T22:43:15.114977+0000","last_change":"2024-09-17T22:43:15.115799+0000","last_active":"2024-09-17T22:43:15.114977+0000","last_peered":"2024-09-17T22:43:15.114977+0000","last_clean":"2024-09-17T22:43:15.114977+0000","last_became_active":"2024-09-17T22:41:19.451631+0000","last_became_peered":"2024-09-17T22:41:19.451631+0000","last_unstale":"2024-09-17T22:43:15.114977+0000","last_undegraded":"2024-09-17T22:43:15.114977+0000","last_fullsized":"2024-09-17T22:43:15.114977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:34:08.442619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015749819999999999,"stat_sum":{"num_bytes":1037547,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":14,"num_read":8321,"num_read_kb":19442,"num_write":735,"num_write_kb":19536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"22"}]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":127,"state":"active+undersized","last_fresh":"2024-09-17T22:43:15.113859+0000","last_change":"2024-09-17T22:41:54.118987+0000","last_active":"2024-09-17T22:43:15.113859+0000","last_peered":"2024-09-17T22:43:15.113859+0000","last_clean":"2024-09-17T22:41:52.098057+0000","last_became_active":"2024-09-17T22:41:54.118987+0000","last_became_peered":"2024-09-17T22:41:54.118987+0000","last_unstale":"2024-09-17T22:43:15.113859+0000","last_undegraded":"2024-09-17T22:43:15.113859+0000","last_fullsized":"2024-09-17T22:41:54.114557+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:06:47.941682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"128'566","reported_seq":5259,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T22:43:16.192667+0000","last_change":"2024-09-17T22:43:16.149288+0000","last_active":"2024-09-17T22:43:16.192667+0000","last_peered":"2024-09-17T22:43:16.192667+0000","last_clean":"2024-09-17T22:43:16.192667+0000","last_became_active":"2024-09-17T22:41:19.448966+0000","last_became_peered":"2024-09-17T22:41:19.448966+0000","last_unstale":"2024-09-17T22:43:16.192667+0000","last_undegraded":"2024-09-17T22:43:16.192667+0000","last_fullsized":"2024-09-17T22:43:16.192667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:54.235337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011619689999999999,"stat_sum":{"num_bytes":671988,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":15,"num_read":6258,"num_read_kb":29920,"num_write":539,"num_write_kb":6243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"23"}]},{"pgid":"2.6","version":"0'0","reported_seq":234,"reported_epoch":128,"state":"active+undersized","last_fresh":"2024-09-17T22:43:16.148242+0000","last_change":"2024-09-17T22:41:54.118378+0000","last_active":"2024-09-17T22:43:16.148242+0000","last_peered":"2024-09-17T22:43:16.148242+0000","last_clean":"2024-09-17T22:41:52.097696+0000","last_became_active":"2024-09-17T22:41:54.118378+0000","last_became_peered":"2024-09-17T22:41:54.118378+0000","last_unstale":"2024-09-17T22:43:16.148242+0000","last_undegraded":"2024-09-17T22:43:16.148242+0000","last_fullsized":"2024-09-17T22:41:54.116433+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:48:40.138581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"128'372","reported_seq":1680,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T22:43:16.148235+0000","last_change":"2024-09-17T22:43:16.148235+0000","last_active":"2024-09-17T22:43:16.148235+0000","last_peered":"2024-09-17T22:43:16.148235+0000","last_clean":"2024-09-17T22:43:16.148235+0000","last_became_active":"2024-09-17T22:41:19.451637+0000","last_became_peered":"2024-09-17T22:41:19.451637+0000","last_unstale":"2024-09-17T22:43:16.148235+0000","last_undegraded":"2024-09-17T22:43:16.148235+0000","last_fullsized":"2024-09-17T22:43:16.148235+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:41:19.972574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071414999999999998,"stat_sum":{"num_bytes":1064850,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":22,"num_read":1443,"num_read_kb":9951,"num_write":394,"num_write_kb":7019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"23"}]},{"pgid":"2.5","version":"0'0","reported_seq":237,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-17T22:43:15.154835+0000","last_change":"2024-09-17T22:43:15.154296+0000","last_active":"2024-09-17T22:43:15.154835+0000","last_peered":"2024-09-17T22:43:15.154835+0000","last_clean":"2024-09-17T22:43:15.154835+0000","last_became_active":"2024-09-17T22:43:15.153956+0000","last_became_peered":"2024-09-17T22:43:15.153956+0000","last_unstale":"2024-09-17T22:43:15.154835+0000","last_undegraded":"2024-09-17T22:43:15.154835+0000","last_fullsized":"2024-09-17T22:43:15.154835+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:19.340692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"128'469","reported_seq":1356,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T22:43:16.148937+0000","last_change":"2024-09-17T22:43:16.148937+0000","last_active":"2024-09-17T22:43:16.148937+0000","last_peered":"2024-09-17T22:43:16.148937+0000","last_clean":"2024-09-17T22:43:16.148937+0000","last_became_active":"2024-09-17T22:41:19.444893+0000","last_became_peered":"2024-09-17T22:41:19.444893+0000","last_unstale":"2024-09-17T22:43:16.148937+0000","last_undegraded":"2024-09-17T22:43:16.148937+0000","last_fullsized":"2024-09-17T22:43:16.148937+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:57:43.270775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001721633,"stat_sum":{"num_bytes":1920144,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":19,"num_read":1718,"num_read_kb":9540,"num_write":632,"num_write_kb":7900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"23"}]},{"pgid":"2.4","version":"0'0","reported_seq":233,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T22:43:16.146756+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:43:16.146756+0000","last_peered":"2024-09-17T22:43:16.146756+0000","last_clean":"2024-09-17T22:43:16.146756+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:43:16.146756+0000","last_undegraded":"2024-09-17T22:43:16.146756+0000","last_fullsized":"2024-09-17T22:43:16.146756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"127'384","reported_seq":15534,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-17T22:43:15.198516+0000","last_change":"2024-09-17T22:43:15.115901+0000","last_active":"2024-09-17T22:43:15.198516+0000","last_peered":"2024-09-17T22:43:15.198516+0000","last_clean":"2024-09-17T22:43:15.198516+0000","last_became_active":"2024-09-17T22:41:19.451723+0000","last_became_peered":"2024-09-17T22:41:19.451723+0000","last_unstale":"2024-09-17T22:43:15.198516+0000","last_undegraded":"2024-09-17T22:43:15.198516+0000","last_fullsized":"2024-09-17T22:43:15.198516+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:41:46.719468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070509299999999995,"stat_sum":{"num_bytes":18518032,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":11,"num_read":15368,"num_read_kb":33652,"num_write":423,"num_write_kb":8748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"22"}]},{"pgid":"3.3","version":"127'820","reported_seq":11740,"reported_epoch":127,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:43:15.195545+0000","last_change":"2024-09-17T22:41:54.119266+0000","last_active":"2024-09-17T22:43:15.195545+0000","last_peered":"2024-09-17T22:43:15.195545+0000","last_clean":"2024-09-17T22:41:52.262287+0000","last_became_active":"2024-09-17T22:41:54.119266+0000","last_became_peered":"2024-09-17T22:41:54.119266+0000","last_unstale":"2024-09-17T22:43:15.195545+0000","last_undegraded":"2024-09-17T22:41:54.116326+0000","last_fullsized":"2024-09-17T22:41:54.116242+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":820,"log_dups_size":0,"ondisk_log_size":820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":34,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:15:54.528023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063898899999999996,"stat_sum":{"num_bytes":21251504,"num_objects":69,"num_object_clones":21,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":69,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":46,"num_read":15301,"num_read_kb":33669,"num_write":839,"num_write_kb":8694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":69}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":86,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T22:43:16.147980+0000","last_change":"2024-09-17T22:42:48.420232+0000","last_active":"2024-09-17T22:43:16.147980+0000","last_peered":"2024-09-17T22:43:16.147980+0000","last_clean":"2024-09-17T22:43:16.147980+0000","last_became_active":"2024-09-17T22:42:48.419946+0000","last_became_peered":"2024-09-17T22:42:48.419946+0000","last_unstale":"2024-09-17T22:43:16.147980+0000","last_undegraded":"2024-09-17T22:43:16.147980+0000","last_fullsized":"2024-09-17T22:43:16.147980+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:11:29.985673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"128'366","reported_seq":4099,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T22:43:16.209335+0000","last_change":"2024-09-17T22:43:16.147267+0000","last_active":"2024-09-17T22:43:16.209335+0000","last_peered":"2024-09-17T22:43:16.209335+0000","last_clean":"2024-09-17T22:43:16.209335+0000","last_became_active":"2024-09-17T22:41:19.444314+0000","last_became_peered":"2024-09-17T22:41:19.444314+0000","last_unstale":"2024-09-17T22:43:16.209335+0000","last_undegraded":"2024-09-17T22:43:16.209335+0000","last_fullsized":"2024-09-17T22:43:16.209335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:51:17.635954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000464981,"stat_sum":{"num_bytes":531503,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":20,"num_read":3975,"num_read_kb":22330,"num_write":343,"num_write_kb":9348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6","length":"23"}]},{"pgid":"2.1","version":"0'0","reported_seq":234,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T22:43:16.112233+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:43:16.112233+0000","last_peered":"2024-09-17T22:43:16.112233+0000","last_clean":"2024-09-17T22:43:16.112233+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:43:16.112233+0000","last_undegraded":"2024-09-17T22:43:16.112233+0000","last_fullsized":"2024-09-17T22:43:16.112233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"128'476","reported_seq":7383,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T22:43:16.228597+0000","last_change":"2024-09-17T22:43:16.148556+0000","last_active":"2024-09-17T22:43:16.228597+0000","last_peered":"2024-09-17T22:43:16.228597+0000","last_clean":"2024-09-17T22:43:16.228597+0000","last_became_active":"2024-09-17T22:41:19.449791+0000","last_became_peered":"2024-09-17T22:41:19.449791+0000","last_unstale":"2024-09-17T22:43:16.228597+0000","last_undegraded":"2024-09-17T22:43:16.228597+0000","last_fullsized":"2024-09-17T22:43:16.228597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:22:28.344220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069221799999999998,"stat_sum":{"num_bytes":155648,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":14,"num_read":8092,"num_read_kb":9247,"num_write":518,"num_write_kb":4605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"23"}]},{"pgid":"2.0","version":"0'0","reported_seq":232,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-17T22:43:15.113359+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:43:15.113359+0000","last_peered":"2024-09-17T22:43:15.113359+0000","last_clean":"2024-09-17T22:43:15.113359+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:43:15.113359+0000","last_undegraded":"2024-09-17T22:43:15.113359+0000","last_fullsized":"2024-09-17T22:43:15.113359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"126'740","reported_seq":8656,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T22:43:16.148314+0000","last_change":"2024-09-17T22:43:16.148776+0000","last_active":"2024-09-17T22:43:16.148314+0000","last_peered":"2024-09-17T22:43:16.148314+0000","last_clean":"2024-09-17T22:43:16.148314+0000","last_became_active":"2024-09-17T22:41:19.449279+0000","last_became_peered":"2024-09-17T22:41:19.449279+0000","last_unstale":"2024-09-17T22:43:16.148314+0000","last_undegraded":"2024-09-17T22:43:16.148314+0000","last_fullsized":"2024-09-17T22:43:16.148314+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:37:47.325464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098880999999999991,"stat_sum":{"num_bytes":4792348,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":21,"num_read":11288,"num_read_kb":22946,"num_write":954,"num_write_kb":2860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"23"}]},{"pgid":"2.3","version":"19'1","reported_seq":71,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-17T22:43:15.114346+0000","last_change":"2024-09-17T22:42:48.420535+0000","last_active":"2024-09-17T22:43:15.114346+0000","last_peered":"2024-09-17T22:43:15.114346+0000","last_clean":"2024-09-17T22:43:15.114346+0000","last_became_active":"2024-09-17T22:42:48.420210+0000","last_became_peered":"2024-09-17T22:42:48.420210+0000","last_unstale":"2024-09-17T22:43:15.114346+0000","last_undegraded":"2024-09-17T22:43:15.114346+0000","last_fullsized":"2024-09-17T22:43:15.114346+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:36.249795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":387,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T22:43:17.136363+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:43:17.136363+0000","last_peered":"2024-09-17T22:43:17.136363+0000","last_clean":"2024-09-17T22:43:17.136363+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:43:17.136363+0000","last_undegraded":"2024-09-17T22:43:17.136363+0000","last_fullsized":"2024-09-17T22:43:17.136363+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"128'609","reported_seq":11879,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T22:43:16.191737+0000","last_change":"2024-09-17T22:43:16.149151+0000","last_active":"2024-09-17T22:43:16.191737+0000","last_peered":"2024-09-17T22:43:16.191737+0000","last_clean":"2024-09-17T22:43:16.191737+0000","last_became_active":"2024-09-17T22:41:19.448956+0000","last_became_peered":"2024-09-17T22:41:19.448956+0000","last_unstale":"2024-09-17T22:43:16.191737+0000","last_undegraded":"2024-09-17T22:43:16.191737+0000","last_fullsized":"2024-09-17T22:43:16.191737+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:58.167326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001239628,"stat_sum":{"num_bytes":159744,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":18,"num_read":13252,"num_read_kb":46911,"num_write":655,"num_write_kb":25175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"23"}]},{"pgid":"3.9","version":"124'756","reported_seq":14655,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-17T22:43:15.114093+0000","last_change":"2024-09-17T22:43:15.115796+0000","last_active":"2024-09-17T22:43:15.114093+0000","last_peered":"2024-09-17T22:43:15.114093+0000","last_clean":"2024-09-17T22:43:15.114093+0000","last_became_active":"2024-09-17T22:41:19.449003+0000","last_became_peered":"2024-09-17T22:41:19.449003+0000","last_unstale":"2024-09-17T22:43:15.114093+0000","last_undegraded":"2024-09-17T22:43:15.114093+0000","last_fullsized":"2024-09-17T22:43:15.114093+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:14:30.848637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013449169999999999,"stat_sum":{"num_bytes":2095580,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":17,"num_read":16204,"num_read_kb":43229,"num_write":1030,"num_write_kb":9843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"22"}]},{"pgid":"3.a","version":"129'469","reported_seq":8450,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T22:43:17.170042+0000","last_change":"2024-09-17T22:43:17.170042+0000","last_active":"2024-09-17T22:43:17.170042+0000","last_peered":"2024-09-17T22:43:17.170042+0000","last_clean":"2024-09-17T22:43:17.170042+0000","last_became_active":"2024-09-17T22:42:48.419427+0000","last_became_peered":"2024-09-17T22:42:48.419427+0000","last_unstale":"2024-09-17T22:43:17.170042+0000","last_undegraded":"2024-09-17T22:43:17.170042+0000","last_fullsized":"2024-09-17T22:43:17.170042+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:51:23.434130+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.033078979000000001,"stat_sum":{"num_bytes":13440345,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":15,"num_read":8191,"num_read_kb":41081,"num_write":472,"num_write_kb":9769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"5","length":"24"}]},{"pgid":"3.b","version":"128'414","reported_seq":2660,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-17T22:43:16.112712+0000","last_change":"2024-09-17T22:43:16.112712+0000","last_active":"2024-09-17T22:43:16.112712+0000","last_peered":"2024-09-17T22:43:16.112712+0000","last_clean":"2024-09-17T22:43:16.112712+0000","last_became_active":"2024-09-17T22:41:19.447215+0000","last_became_peered":"2024-09-17T22:41:19.447215+0000","last_unstale":"2024-09-17T22:43:16.112712+0000","last_undegraded":"2024-09-17T22:43:16.112712+0000","last_fullsized":"2024-09-17T22:43:16.112712+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:05:34.399860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044034599999999999,"stat_sum":{"num_bytes":3260936,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":18,"num_read":3409,"num_read_kb":14480,"num_write":403,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"6","length":"23"}]},{"pgid":"3.c","version":"128'309","reported_seq":21277,"reported_epoch":128,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:43:16.186931+0000","last_change":"2024-09-17T22:41:54.119555+0000","last_active":"2024-09-17T22:43:16.186931+0000","last_peered":"2024-09-17T22:43:16.186931+0000","last_clean":"2024-09-17T22:41:52.249181+0000","last_became_active":"2024-09-17T22:41:54.119555+0000","last_became_peered":"2024-09-17T22:41:54.119555+0000","last_unstale":"2024-09-17T22:43:16.186931+0000","last_undegraded":"2024-09-17T22:41:54.118897+0000","last_fullsized":"2024-09-17T22:41:54.118785+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":35,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:34:01.899779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000395712,"stat_sum":{"num_bytes":31918040,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":31,"num_read":21316,"num_read_kb":63555,"num_write":323,"num_write_kb":14934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":43}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"129'486","reported_seq":2006,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T22:43:17.166913+0000","last_change":"2024-09-17T22:43:17.167285+0000","last_active":"2024-09-17T22:43:17.166913+0000","last_peered":"2024-09-17T22:43:17.166913+0000","last_clean":"2024-09-17T22:43:17.166913+0000","last_became_active":"2024-09-17T22:42:48.419360+0000","last_became_peered":"2024-09-17T22:42:48.419360+0000","last_unstale":"2024-09-17T22:43:17.166913+0000","last_undegraded":"2024-09-17T22:43:17.166913+0000","last_fullsized":"2024-09-17T22:43:17.166913+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:57:05.359086+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.030609778000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2216,"num_read_kb":10204,"num_write":574,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"5","length":"24"}]},{"pgid":"3.e","version":"124'342","reported_seq":8489,"reported_epoch":127,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:43:15.114571+0000","last_change":"2024-09-17T22:41:54.119016+0000","last_active":"2024-09-17T22:43:15.114571+0000","last_peered":"2024-09-17T22:43:15.114571+0000","last_clean":"2024-09-17T22:41:52.398679+0000","last_became_active":"2024-09-17T22:41:54.119016+0000","last_became_peered":"2024-09-17T22:41:54.119016+0000","last_unstale":"2024-09-17T22:43:15.114571+0000","last_undegraded":"2024-09-17T22:41:54.116346+0000","last_fullsized":"2024-09-17T22:41:54.116297+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":34,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:08:07.251520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038538199999999998,"stat_sum":{"num_bytes":7058838,"num_objects":47,"num_object_clones":15,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":32,"num_read":8804,"num_read_kb":23106,"num_write":291,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":47}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"128'322","reported_seq":1556,"reported_epoch":129,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:43:17.137042+0000","last_change":"2024-09-17T22:41:54.113116+0000","last_active":"2024-09-17T22:43:17.137042+0000","last_peered":"2024-09-17T22:43:17.137042+0000","last_clean":"2024-09-17T22:41:52.346319+0000","last_became_active":"2024-09-17T22:41:54.113116+0000","last_became_peered":"2024-09-17T22:41:54.113116+0000","last_unstale":"2024-09-17T22:43:17.137042+0000","last_undegraded":"2024-09-17T22:41:54.112692+0000","last_fullsized":"2024-09-17T22:41:54.112614+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":36,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:18:29.375139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000602379,"stat_sum":{"num_bytes":20935789,"num_objects":52,"num_object_clones":15,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":52,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":36,"num_read":1634,"num_read_kb":5501,"num_write":379,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":52}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"129'349","reported_seq":2108,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-17T22:43:17.172325+0000","last_change":"2024-09-17T22:43:17.173095+0000","last_active":"2024-09-17T22:43:17.172325+0000","last_peered":"2024-09-17T22:43:17.172325+0000","last_clean":"2024-09-17T22:43:17.172325+0000","last_became_active":"2024-09-17T22:41:19.447419+0000","last_became_peered":"2024-09-17T22:41:19.447419+0000","last_unstale":"2024-09-17T22:43:17.172325+0000","last_undegraded":"2024-09-17T22:43:17.172325+0000","last_fullsized":"2024-09-17T22:43:17.172325+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:37:52.888741+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.036055242000000001,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3110,"num_read_kb":8489,"num_write":286,"num_write_kb":3161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"5","length":"24"}]},{"pgid":"3.11","version":"126'606","reported_seq":3596,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-17T22:43:15.113722+0000","last_change":"2024-09-17T22:43:15.114913+0000","last_active":"2024-09-17T22:43:15.113722+0000","last_peered":"2024-09-17T22:43:15.113722+0000","last_clean":"2024-09-17T22:43:15.113722+0000","last_became_active":"2024-09-17T22:41:19.454151+0000","last_became_peered":"2024-09-17T22:41:19.454151+0000","last_unstale":"2024-09-17T22:43:15.113722+0000","last_undegraded":"2024-09-17T22:43:15.113722+0000","last_fullsized":"2024-09-17T22:43:15.113722+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:41:18.585369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014397469999999999,"stat_sum":{"num_bytes":1038336,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":20,"num_read":5266,"num_read_kb":13546,"num_write":658,"num_write_kb":3731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"7","length":"22"}]},{"pgid":"3.12","version":"128'906","reported_seq":11463,"reported_epoch":128,"state":"active+clean+remapped","last_fresh":"2024-09-17T22:43:16.193703+0000","last_change":"2024-09-17T22:43:16.149178+0000","last_active":"2024-09-17T22:43:16.193703+0000","last_peered":"2024-09-17T22:43:16.193703+0000","last_clean":"2024-09-17T22:43:16.193703+0000","last_became_active":"2024-09-17T22:41:57.131666+0000","last_became_peered":"2024-09-17T22:41:57.131666+0000","last_unstale":"2024-09-17T22:43:16.193703+0000","last_undegraded":"2024-09-17T22:43:16.193703+0000","last_fullsized":"2024-09-17T22:43:16.193703+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:44:34.522785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00121609,"stat_sum":{"num_bytes":2621488,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":13,"num_read":14640,"num_read_kb":44073,"num_write":1140,"num_write_kb":22161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6,3],"avail_no_missing":["6","3"],"object_location_counts":[{"shards":"3,6","objects":16}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"6","length":"23"}]},{"pgid":"3.13","version":"127'472","reported_seq":2280,"reported_epoch":127,"state":"active+undersized+degraded","last_fresh":"2024-09-17T22:43:15.217455+0000","last_change":"2024-09-17T22:41:54.118972+0000","last_active":"2024-09-17T22:43:15.217455+0000","last_peered":"2024-09-17T22:43:15.217455+0000","last_clean":"2024-09-17T22:41:52.267782+0000","last_became_active":"2024-09-17T22:41:54.118972+0000","last_became_peered":"2024-09-17T22:41:54.118972+0000","last_unstale":"2024-09-17T22:43:15.217455+0000","last_undegraded":"2024-09-17T22:41:54.114427+0000","last_fullsized":"2024-09-17T22:41:54.114366+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":34,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:41.657962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062146200000000003,"stat_sum":{"num_bytes":5089902,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":44,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":33,"num_read":2478,"num_read_kb":8780,"num_write":605,"num_write_kb":8380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":44}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":236677158,"num_objects":877,"num_object_clones":172,"num_object_copies":1754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":337,"num_objects_misplaced":45,"num_objects_unfound":0,"num_objects_dirty":877,"num_whiteouts":664,"num_read":267482,"num_read_kb":801694,"num_write":17993,"num_write_kb":293668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":14661402,"num_keys_recovered":86,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8994816,"data_stored":63434889,"data_compressed":1510037,"data_compressed_allocated":5914624,"data_compressed_original":61647089,"omap_allocated":0,"internal_metadata":0},"log_size":16122,"ondisk_log_size":16122,"up":55,"acting":57,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":14,"acting":14,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542175,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29236,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342604,"statfs":{"total":96636764160,"available":96606826496,"internally_reserved":0,"allocated":2252800,"data_stored":5438699,"data_compressed":105385,"data_compressed_allocated":446464,"data_compressed_original":4534272,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542156,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34156,"kb_used_data":2972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337684,"statfs":{"total":96636764160,"available":96601788416,"internally_reserved":0,"allocated":3043328,"data_stored":23926169,"data_compressed":437861,"data_compressed_allocated":1896448,"data_compressed_original":23315877,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542173,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41668,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330172,"statfs":{"total":96636764160,"available":96594096128,"internally_reserved":0,"allocated":2211840,"data_stored":5034377,"data_compressed":138528,"data_compressed_allocated":450560,"data_compressed_original":4161536,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542173,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30136,"kb_used_data":3108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341704,"statfs":{"total":96636764160,"available":96605904896,"internally_reserved":0,"allocated":3182592,"data_stored":13879314,"data_compressed":337868,"data_compressed_allocated":1216512,"data_compressed_original":12897060,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542173,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47676,"kb_used_data":2600,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324164,"statfs":{"total":96636764160,"available":96587943936,"internally_reserved":0,"allocated":2662400,"data_stored":10374120,"data_compressed":130679,"data_compressed_allocated":798720,"data_compressed_original":9440880,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46292,"kb_used_data":2624,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325548,"statfs":{"total":96636764160,"available":96589361152,"internally_reserved":0,"allocated":2686976,"data_stored":8265472,"data_compressed":228866,"data_compressed_allocated":770048,"data_compressed_original":7307264,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56236,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315604,"statfs":{"total":96636764160,"available":96579178496,"internally_reserved":0,"allocated":2883584,"data_stored":8224816,"data_compressed":210250,"data_compressed_allocated":823296,"data_compressed_original":7193420,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52412,"kb_used_data":3172,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319428,"statfs":{"total":96636764160,"available":96583094272,"internally_reserved":0,"allocated":3248128,"data_stored":14470078,"data_compressed":199943,"data_compressed_allocated":1163264,"data_compressed_original":13410304,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2514944,"data_stored":23924031,"data_compressed":438892,"data_compressed_allocated":1921024,"data_compressed_original":23549349,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315392,"data_stored":2062309,"data_compressed":72257,"data_compressed_allocated":212992,"data_compressed_original":2002944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1740800,"data_stored":13746249,"data_compressed":199943,"data_compressed_allocated":1163264,"data_compressed_original":13410304,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1363968,"data_stored":7494624,"data_compressed":210250,"data_compressed_allocated":823296,"data_compressed_original":7193420,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1044480,"data_stored":6665942,"data_compressed":201632,"data_compressed_allocated":675840,"data_compressed_original":6459392,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":507904,"data_stored":2116055,"data_compressed":33744,"data_compressed_allocated":229376,"data_compressed_original":1961984,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1032192,"data_stored":5370785,"data_compressed":244542,"data_compressed_allocated":626688,"data_compressed_original":5136384,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":475136,"data_stored":2054894,"data_compressed":108777,"data_compressed_allocated":262144,"data_compressed_original":1933312,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:43:19.429 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 3.15 3.3 3.c 3.e 3.f 3.13 2024-09-17T22:43:19.711 INFO:teuthology.orchestra.run.smithi115.stderr:instructing pg(s) [3.f] on osd.1 to force-backfill; instructing pg(s) [3.3,3.e] on osd.2 to force-backfill; instructing pg(s) [3.13] on osd.3 to force-backfill; instructing pg(s) [3.15] on osd.6 to force-backfill; instructing pg(s) [3.c] on osd.7 to force-backfill; 2024-09-17T22:43:19.711 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-09-17T22:43:21.163 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T22:43:22.149 INFO:teuthology.orchestra.run.smithi115.stderr:noscrub is unset 2024-09-17T22:43:22.168 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T22:43:23.167 INFO:teuthology.orchestra.run.smithi115.stderr:nodeep-scrub is unset 2024-09-17T22:43:24.724 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-17T22:43:24.724 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-17T22:43:24.724 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T22:43:24.724 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-17T22:43:24.727 INFO:tasks.ceph.osd.0:Started 2024-09-17T22:43:24.728 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:43:24.837 INFO:teuthology.orchestra.run.smithi115.stderr:no valid command found; 10 closest matches: 2024-09-17T22:43:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:0 2024-09-17T22:43:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:1 2024-09-17T22:43:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:2 2024-09-17T22:43:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:abort 2024-09-17T22:43:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:assert 2024-09-17T22:43:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:config diff 2024-09-17T22:43:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:config diff get 2024-09-17T22:43:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:config get 2024-09-17T22:43:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:config help [] 2024-09-17T22:43:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:config set ... 2024-09-17T22:43:24.838 INFO:teuthology.orchestra.run.smithi115.stderr:admin_socket: invalid command 2024-09-17T22:43:24.840 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:43:24.840 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-17T22:43:25.911 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:43:25.908+0000 7f41f8fdd640 -1 Falling back to public interface 2024-09-17T22:43:29.455 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:43:29.452+0000 7f41f8fdd640 -1 osd.0 50 log_to_monitors true 2024-09-17T22:43:29.841 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:43:29.947 INFO:teuthology.orchestra.run.smithi115.stderr:no valid command found; 10 closest matches: 2024-09-17T22:43:29.948 INFO:teuthology.orchestra.run.smithi115.stderr:0 2024-09-17T22:43:29.948 INFO:teuthology.orchestra.run.smithi115.stderr:1 2024-09-17T22:43:29.948 INFO:teuthology.orchestra.run.smithi115.stderr:2 2024-09-17T22:43:29.948 INFO:teuthology.orchestra.run.smithi115.stderr:abort 2024-09-17T22:43:29.948 INFO:teuthology.orchestra.run.smithi115.stderr:assert 2024-09-17T22:43:29.948 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs debug_inject_read_zeros 2024-09-17T22:43:29.948 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs files list 2024-09-17T22:43:29.948 INFO:teuthology.orchestra.run.smithi115.stderr:bluefs stats 2024-09-17T22:43:29.948 INFO:teuthology.orchestra.run.smithi115.stderr:bluestore allocator dump block 2024-09-17T22:43:29.949 INFO:teuthology.orchestra.run.smithi115.stderr:bluestore allocator fragmentation block 2024-09-17T22:43:29.949 INFO:teuthology.orchestra.run.smithi115.stderr:admin_socket: invalid command 2024-09-17T22:43:29.950 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T22:43:29.951 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-17T22:43:30.223 INFO:tasks.ceph.osd.0.smithi115.stderr:2024-09-17T22:43:30.220+0000 7f41ecd8c640 -1 osd.0 134 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T22:43:34.952 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T22:43:35.060 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:43:35.060 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-09-17T22:43:35.060 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-09-17T22:43:35.061 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:43:35.069 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-17T22:43:35.180 INFO:teuthology.orchestra.run.smithi115.stdout:mon.a: {} 2024-09-17T22:43:35.181 INFO:teuthology.orchestra.run.smithi115.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T22:43:35.191 INFO:teuthology.orchestra.run.smithi115.stdout:mon.b: {} 2024-09-17T22:43:35.191 INFO:teuthology.orchestra.run.smithi115.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T22:43:35.199 INFO:teuthology.orchestra.run.smithi115.stdout:mon.c: {} 2024-09-17T22:43:35.199 INFO:teuthology.orchestra.run.smithi115.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T22:43:35.210 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-17T22:43:36.245 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T22:43:36.261 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-17T22:43:37.251 INFO:teuthology.orchestra.run.smithi115.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-17T22:43:37.267 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-17T22:43:38.254 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T22:43:38.270 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-17T22:43:39.263 INFO:teuthology.orchestra.run.smithi115.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-17T22:43:39.279 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-17T22:43:39.998 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T22:43:40.012 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-17T22:43:41.273 INFO:teuthology.orchestra.run.smithi115.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-17T22:43:41.292 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T22:43:42.278 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T22:43:42.296 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-17T22:43:43.279 INFO:teuthology.orchestra.run.smithi115.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-17T22:43:43.295 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-17T22:43:44.299 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T22:43:44.315 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-17T22:43:45.309 INFO:teuthology.orchestra.run.smithi115.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-17T22:43:45.324 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-17T22:43:46.321 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.6 to 1 (10000) 2024-09-17T22:43:46.337 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-17T22:43:47.323 INFO:teuthology.orchestra.run.smithi115.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-17T22:43:47.339 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-17T22:43:48.335 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.7 to 1 (10000) 2024-09-17T22:43:48.351 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-17T22:43:49.344 INFO:teuthology.orchestra.run.smithi115.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-17T22:43:49.360 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-17T22:43:50.349 INFO:teuthology.orchestra.run.smithi115.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T22:43:50.364 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-17T22:43:51.351 INFO:teuthology.orchestra.run.smithi115.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-17T22:43:51.370 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-17T22:43:51.370 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-17T22:43:51.370 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-17T22:43:51.370 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-17T22:43:51.371 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T22:43:51.371 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T22:43:51.652 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:43:51.652 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":155,"fsid":"551c5ed8-e790-485a-bc95-a9578a29275c","created":"2024-09-17T22:40:50.189230+0000","modified":"2024-09-17T22:43:51.348625+0000","last_up_change":"2024-09-17T22:43:31.180198+0000","last_in_change":"2024-09-17T22:40:52.260526+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T22:41:02.338356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T22:41:08.097861+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T22:41:17.501542+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":40,"snap_epoch":131,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3bd5c11f-e7c2-4968-839a-2799c00d93a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":50,"up_from":135,"up_thru":23,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":4198232340},{"type":"v1","addr":"172.21.15.115:6825","nonce":4198232340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":4198232340},{"type":"v1","addr":"172.21.15.115:6827","nonce":4198232340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":4198232340},{"type":"v1","addr":"172.21.15.115:6831","nonce":4198232340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4198232340},{"type":"v1","addr":"172.21.15.115:6829","nonce":4198232340}]},"public_addr":"172.21.15.115:6825/4198232340","cluster_addr":"172.21.15.115:6827/4198232340","heartbeat_back_addr":"172.21.15.115:6831/4198232340","heartbeat_front_addr":"172.21.15.115:6829/4198232340","state":["exists","up"]},{"osd":1,"uuid":"02918203-cf2e-49fc-b994-743b5d81b4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6817","nonce":1941780390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6819","nonce":1941780390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6823","nonce":1941780390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6821","nonce":1941780390}]},"public_addr":"172.21.15.115:6817/1941780390","cluster_addr":"172.21.15.115:6819/1941780390","heartbeat_back_addr":"172.21.15.115:6823/1941780390","heartbeat_front_addr":"172.21.15.115:6821/1941780390","state":["exists","up"]},{"osd":2,"uuid":"22ee34ef-522e-4121-85b7-38bbafeca5de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6809","nonce":689014698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6811","nonce":689014698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6815","nonce":689014698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6813","nonce":689014698}]},"public_addr":"172.21.15.115:6809/689014698","cluster_addr":"172.21.15.115:6811/689014698","heartbeat_back_addr":"172.21.15.115:6815/689014698","heartbeat_front_addr":"172.21.15.115:6813/689014698","state":["exists","up"]},{"osd":3,"uuid":"5d464921-5696-43c5-b89d-430a54ddefb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6801","nonce":453192890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6803","nonce":453192890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6807","nonce":453192890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6805","nonce":453192890}]},"public_addr":"172.21.15.115:6801/453192890","cluster_addr":"172.21.15.115:6803/453192890","heartbeat_back_addr":"172.21.15.115:6807/453192890","heartbeat_front_addr":"172.21.15.115:6805/453192890","state":["exists","up"]},{"osd":4,"uuid":"4cb6973a-63ae-4f48-af94-37610aee3c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6801","nonce":380388860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6803","nonce":380388860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6807","nonce":380388860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6805","nonce":380388860}]},"public_addr":"172.21.15.129:6801/380388860","cluster_addr":"172.21.15.129:6803/380388860","heartbeat_back_addr":"172.21.15.129:6807/380388860","heartbeat_front_addr":"172.21.15.129:6805/380388860","state":["exists","up"]},{"osd":5,"uuid":"ef9fe11b-b719-48f4-9c6c-e1b1b355acdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6809","nonce":943488949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6811","nonce":943488949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6815","nonce":943488949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6813","nonce":943488949}]},"public_addr":"172.21.15.129:6809/943488949","cluster_addr":"172.21.15.129:6811/943488949","heartbeat_back_addr":"172.21.15.129:6815/943488949","heartbeat_front_addr":"172.21.15.129:6813/943488949","state":["exists","up"]},{"osd":6,"uuid":"e4045aec-22c1-401b-bfbf-64f115bc32af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6817","nonce":1873501077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6819","nonce":1873501077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6823","nonce":1873501077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6821","nonce":1873501077}]},"public_addr":"172.21.15.129:6817/1873501077","cluster_addr":"172.21.15.129:6819/1873501077","heartbeat_back_addr":"172.21.15.129:6823/1873501077","heartbeat_front_addr":"172.21.15.129:6821/1873501077","state":["exists","up"]},{"osd":7,"uuid":"4a05a5d7-8281-4572-a9d0-6095ee5e168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6825","nonce":3609357666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6827","nonce":3609357666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6831","nonce":3609357666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6829","nonce":3609357666}]},"public_addr":"172.21.15.129:6825/3609357666","cluster_addr":"172.21.15.129:6827/3609357666","heartbeat_back_addr":"172.21.15.129:6831/3609357666","heartbeat_front_addr":"172.21.15.129:6829/3609357666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:59.351758+0000","dead_epoch":51},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:59.058915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.922383+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.802379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.881681+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.616649+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.589555+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:59.047783+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.1c","osds":[1,3]},{"pgid":"3.1d","osds":[1,6]}],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":4,"to":7}]},{"pgid":"3.12","mappings":[{"from":3,"to":0}]},{"pgid":"3.16","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T22:43:51.665 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T22:43:51.665 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T22:43:51.941 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:43:51.942 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":155,"fsid":"551c5ed8-e790-485a-bc95-a9578a29275c","created":"2024-09-17T22:40:50.189230+0000","modified":"2024-09-17T22:43:51.348625+0000","last_up_change":"2024-09-17T22:43:31.180198+0000","last_in_change":"2024-09-17T22:40:52.260526+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T22:41:02.338356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T22:41:08.097861+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T22:41:17.501542+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":40,"snap_epoch":131,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3bd5c11f-e7c2-4968-839a-2799c00d93a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":50,"up_from":135,"up_thru":23,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":4198232340},{"type":"v1","addr":"172.21.15.115:6825","nonce":4198232340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":4198232340},{"type":"v1","addr":"172.21.15.115:6827","nonce":4198232340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":4198232340},{"type":"v1","addr":"172.21.15.115:6831","nonce":4198232340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4198232340},{"type":"v1","addr":"172.21.15.115:6829","nonce":4198232340}]},"public_addr":"172.21.15.115:6825/4198232340","cluster_addr":"172.21.15.115:6827/4198232340","heartbeat_back_addr":"172.21.15.115:6831/4198232340","heartbeat_front_addr":"172.21.15.115:6829/4198232340","state":["exists","up"]},{"osd":1,"uuid":"02918203-cf2e-49fc-b994-743b5d81b4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6817","nonce":1941780390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6819","nonce":1941780390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6823","nonce":1941780390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6821","nonce":1941780390}]},"public_addr":"172.21.15.115:6817/1941780390","cluster_addr":"172.21.15.115:6819/1941780390","heartbeat_back_addr":"172.21.15.115:6823/1941780390","heartbeat_front_addr":"172.21.15.115:6821/1941780390","state":["exists","up"]},{"osd":2,"uuid":"22ee34ef-522e-4121-85b7-38bbafeca5de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6809","nonce":689014698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6811","nonce":689014698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6815","nonce":689014698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6813","nonce":689014698}]},"public_addr":"172.21.15.115:6809/689014698","cluster_addr":"172.21.15.115:6811/689014698","heartbeat_back_addr":"172.21.15.115:6815/689014698","heartbeat_front_addr":"172.21.15.115:6813/689014698","state":["exists","up"]},{"osd":3,"uuid":"5d464921-5696-43c5-b89d-430a54ddefb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6801","nonce":453192890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6803","nonce":453192890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6807","nonce":453192890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6805","nonce":453192890}]},"public_addr":"172.21.15.115:6801/453192890","cluster_addr":"172.21.15.115:6803/453192890","heartbeat_back_addr":"172.21.15.115:6807/453192890","heartbeat_front_addr":"172.21.15.115:6805/453192890","state":["exists","up"]},{"osd":4,"uuid":"4cb6973a-63ae-4f48-af94-37610aee3c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6801","nonce":380388860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6803","nonce":380388860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6807","nonce":380388860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6805","nonce":380388860}]},"public_addr":"172.21.15.129:6801/380388860","cluster_addr":"172.21.15.129:6803/380388860","heartbeat_back_addr":"172.21.15.129:6807/380388860","heartbeat_front_addr":"172.21.15.129:6805/380388860","state":["exists","up"]},{"osd":5,"uuid":"ef9fe11b-b719-48f4-9c6c-e1b1b355acdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6809","nonce":943488949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6811","nonce":943488949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6815","nonce":943488949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6813","nonce":943488949}]},"public_addr":"172.21.15.129:6809/943488949","cluster_addr":"172.21.15.129:6811/943488949","heartbeat_back_addr":"172.21.15.129:6815/943488949","heartbeat_front_addr":"172.21.15.129:6813/943488949","state":["exists","up"]},{"osd":6,"uuid":"e4045aec-22c1-401b-bfbf-64f115bc32af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6817","nonce":1873501077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6819","nonce":1873501077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6823","nonce":1873501077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6821","nonce":1873501077}]},"public_addr":"172.21.15.129:6817/1873501077","cluster_addr":"172.21.15.129:6819/1873501077","heartbeat_back_addr":"172.21.15.129:6823/1873501077","heartbeat_front_addr":"172.21.15.129:6821/1873501077","state":["exists","up"]},{"osd":7,"uuid":"4a05a5d7-8281-4572-a9d0-6095ee5e168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6825","nonce":3609357666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6827","nonce":3609357666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6831","nonce":3609357666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6829","nonce":3609357666}]},"public_addr":"172.21.15.129:6825/3609357666","cluster_addr":"172.21.15.129:6827/3609357666","heartbeat_back_addr":"172.21.15.129:6831/3609357666","heartbeat_front_addr":"172.21.15.129:6829/3609357666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:59.351758+0000","dead_epoch":51},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:59.058915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.922383+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.802379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.881681+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.616649+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.589555+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:59.047783+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.1c","osds":[1,3]},{"pgid":"3.1d","osds":[1,6]}],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":4,"to":7}]},{"pgid":"3.12","mappings":[{"from":3,"to":0}]},{"pgid":"3.16","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T22:43:51.955 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T22:43:51.955 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T22:43:51.956 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T22:43:51.956 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T22:43:51.956 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T22:43:51.957 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T22:43:51.957 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-17T22:43:51.958 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-17T22:43:52.196 INFO:teuthology.orchestra.run.smithi115.stdout:579820584966 2024-09-17T22:43:52.197 INFO:teuthology.orchestra.run.smithi115.stdout:60129542181 2024-09-17T22:43:52.197 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T22:43:52.198 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T22:43:52.199 INFO:teuthology.orchestra.run.smithi115.stdout:55834574886 2024-09-17T22:43:52.199 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T22:43:52.200 INFO:teuthology.orchestra.run.smithi115.stdout:55834574886 2024-09-17T22:43:52.200 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T22:43:52.201 INFO:teuthology.orchestra.run.smithi115.stdout:60129542182 2024-09-17T22:43:52.201 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T22:43:52.202 INFO:teuthology.orchestra.run.smithi115.stdout:60129542181 2024-09-17T22:43:52.203 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T22:43:52.203 INFO:teuthology.orchestra.run.smithi115.stdout:60129542181 2024-09-17T22:43:52.204 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T22:43:52.205 INFO:teuthology.orchestra.run.smithi115.stdout:60129542181 2024-09-17T22:43:52.205 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T22:43:52.754 INFO:teuthology.orchestra.run.smithi115.stdout:60129542181 2024-09-17T22:43:52.774 INFO:teuthology.orchestra.run.smithi115.stdout:579820584966 2024-09-17T22:43:52.776 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542181 got 60129542181 for osd.7 2024-09-17T22:43:52.776 DEBUG:teuthology.parallel:result is None 2024-09-17T22:43:52.793 INFO:tasks.ceph.ceph_manager.ceph:need seq 579820584966 got 579820584966 for osd.0 2024-09-17T22:43:52.793 DEBUG:teuthology.parallel:result is None 2024-09-17T22:43:52.798 INFO:teuthology.orchestra.run.smithi115.stdout:55834574886 2024-09-17T22:43:52.812 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574886 got 55834574886 for osd.3 2024-09-17T22:43:52.812 DEBUG:teuthology.parallel:result is None 2024-09-17T22:43:52.820 INFO:teuthology.orchestra.run.smithi115.stdout:60129542181 2024-09-17T22:43:52.827 INFO:teuthology.orchestra.run.smithi115.stdout:60129542181 2024-09-17T22:43:52.835 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542181 got 60129542181 for osd.4 2024-09-17T22:43:52.835 DEBUG:teuthology.parallel:result is None 2024-09-17T22:43:52.840 INFO:teuthology.orchestra.run.smithi115.stdout:60129542182 2024-09-17T22:43:52.844 INFO:teuthology.orchestra.run.smithi115.stdout:60129542181 2024-09-17T22:43:52.846 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542181 got 60129542181 for osd.5 2024-09-17T22:43:52.846 DEBUG:teuthology.parallel:result is None 2024-09-17T22:43:52.849 INFO:teuthology.orchestra.run.smithi115.stdout:55834574886 2024-09-17T22:43:52.859 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542182 got 60129542182 for osd.1 2024-09-17T22:43:52.859 DEBUG:teuthology.parallel:result is None 2024-09-17T22:43:52.859 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542181 got 60129542181 for osd.6 2024-09-17T22:43:52.860 DEBUG:teuthology.parallel:result is None 2024-09-17T22:43:52.865 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574886 got 55834574886 for osd.2 2024-09-17T22:43:52.865 DEBUG:teuthology.parallel:result is None 2024-09-17T22:43:52.865 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T22:43:52.865 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:43:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:43:53.182 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:43:53.194 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2024-09-17T22:43:52.373922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":500,"num_object_clones":0,"num_object_copies":1000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":500,"num_whiteouts":492,"num_read":268221,"num_read_kb":802336,"num_write":18302,"num_write_kb":296121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":30918130,"num_keys_recovered":86,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16753,"ondisk_log_size":16753,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":315860,"kb_used_data":14608,"kb_used_omap":32,"kb_used_meta":301023,"kb_avail":754658860,"statfs":{"total":773094113280,"available":772770672640,"internally_reserved":0,"allocated":14958592,"data_stored":8373156,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":33450,"internal_metadata":308247894},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-114,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035823"},"pg_stats":[{"pgid":"3.1f","version":"131'860","reported_seq":7310,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749216+0000","last_change":"2024-09-17T22:43:19.191421+0000","last_active":"2024-09-17T22:43:51.749216+0000","last_peered":"2024-09-17T22:43:51.749216+0000","last_clean":"2024-09-17T22:43:51.749216+0000","last_became_active":"2024-09-17T22:41:19.443658+0000","last_became_peered":"2024-09-17T22:41:19.443658+0000","last_unstale":"2024-09-17T22:43:51.749216+0000","last_undegraded":"2024-09-17T22:43:51.749216+0000","last_fullsized":"2024-09-17T22:43:51.749216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:58.423329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001073428,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8816,"num_read_kb":14906,"num_write":1102,"num_write_kb":4548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"130'786","reported_seq":7422,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.748986+0000","last_change":"2024-09-17T22:43:19.190807+0000","last_active":"2024-09-17T22:43:51.748986+0000","last_peered":"2024-09-17T22:43:51.748986+0000","last_clean":"2024-09-17T22:43:51.748986+0000","last_became_active":"2024-09-17T22:41:19.449357+0000","last_became_peered":"2024-09-17T22:41:19.449357+0000","last_unstale":"2024-09-17T22:43:51.748986+0000","last_undegraded":"2024-09-17T22:43:51.748986+0000","last_fullsized":"2024-09-17T22:43:51.748986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:20.300342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051110000000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":8865,"num_read_kb":26537,"num_write":1126,"num_write_kb":10958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"131'435","reported_seq":2380,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357945+0000","last_change":"2024-09-17T22:43:19.144579+0000","last_active":"2024-09-17T22:43:51.357945+0000","last_peered":"2024-09-17T22:43:51.357945+0000","last_clean":"2024-09-17T22:43:51.357945+0000","last_became_active":"2024-09-17T22:42:58.002822+0000","last_became_peered":"2024-09-17T22:42:58.002822+0000","last_unstale":"2024-09-17T22:43:51.357945+0000","last_undegraded":"2024-09-17T22:43:51.357945+0000","last_fullsized":"2024-09-17T22:43:51.357945+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:18:33.730453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022657670000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":17641,"num_read_kb":48444,"num_write":377,"num_write_kb":8987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":7308630,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"131'494","reported_seq":2826,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357881+0000","last_change":"2024-09-17T22:43:19.146655+0000","last_active":"2024-09-17T22:43:51.357881+0000","last_peered":"2024-09-17T22:43:51.357881+0000","last_clean":"2024-09-17T22:43:51.357881+0000","last_became_active":"2024-09-17T22:42:34.310411+0000","last_became_peered":"2024-09-17T22:42:34.310411+0000","last_unstale":"2024-09-17T22:43:51.357881+0000","last_undegraded":"2024-09-17T22:43:51.357881+0000","last_fullsized":"2024-09-17T22:43:51.357881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:34.577953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039390110000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":2615,"num_read_kb":14215,"num_write":599,"num_write_kb":12172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":7352772,"num_keys_recovered":74,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"139'244","reported_seq":8116,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352818+0000","last_change":"2024-09-17T22:43:35.308387+0000","last_active":"2024-09-17T22:43:51.352818+0000","last_peered":"2024-09-17T22:43:51.352818+0000","last_clean":"2024-09-17T22:43:51.352818+0000","last_became_active":"2024-09-17T22:43:35.215952+0000","last_became_peered":"2024-09-17T22:43:35.215952+0000","last_unstale":"2024-09-17T22:43:51.352818+0000","last_undegraded":"2024-09-17T22:43:51.352818+0000","last_fullsized":"2024-09-17T22:43:51.352818+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:45.376055+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.091482261999999995,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7315,"num_read_kb":22822,"num_write":186,"num_write_kb":6215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":3640098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"131'360","reported_seq":5241,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357959+0000","last_change":"2024-09-17T22:43:19.145987+0000","last_active":"2024-09-17T22:43:51.357959+0000","last_peered":"2024-09-17T22:43:51.357959+0000","last_clean":"2024-09-17T22:43:51.357959+0000","last_became_active":"2024-09-17T22:41:19.446541+0000","last_became_peered":"2024-09-17T22:41:19.446541+0000","last_unstale":"2024-09-17T22:43:51.357959+0000","last_undegraded":"2024-09-17T22:43:51.357959+0000","last_fullsized":"2024-09-17T22:43:51.357959+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:12:51.399521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035192800000000001,"stat_sum":{"num_bytes":19,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":5091,"num_read_kb":19688,"num_write":329,"num_write_kb":4179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"130'391","reported_seq":6764,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352819+0000","last_change":"2024-09-17T22:43:19.150923+0000","last_active":"2024-09-17T22:43:51.352819+0000","last_peered":"2024-09-17T22:43:51.352819+0000","last_clean":"2024-09-17T22:43:51.352819+0000","last_became_active":"2024-09-17T22:41:19.449940+0000","last_became_peered":"2024-09-17T22:41:19.449940+0000","last_unstale":"2024-09-17T22:43:51.352819+0000","last_undegraded":"2024-09-17T22:43:51.352819+0000","last_fullsized":"2024-09-17T22:43:51.352819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:57:32.786125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002665995,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6668,"num_read_kb":22143,"num_write":345,"num_write_kb":9035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"129'383","reported_seq":3221,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749001+0000","last_change":"2024-09-17T22:43:19.191965+0000","last_active":"2024-09-17T22:43:51.749001+0000","last_peered":"2024-09-17T22:43:51.749001+0000","last_clean":"2024-09-17T22:43:51.749001+0000","last_became_active":"2024-09-17T22:41:19.446287+0000","last_became_peered":"2024-09-17T22:41:19.446287+0000","last_unstale":"2024-09-17T22:43:51.749001+0000","last_undegraded":"2024-09-17T22:43:51.749001+0000","last_fullsized":"2024-09-17T22:43:51.749001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:23:52.909972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001462812,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3441,"num_read_kb":20083,"num_write":323,"num_write_kb":10930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"131'513","reported_seq":2752,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749198+0000","last_change":"2024-09-17T22:43:19.191786+0000","last_active":"2024-09-17T22:43:51.749198+0000","last_peered":"2024-09-17T22:43:51.749198+0000","last_clean":"2024-09-17T22:43:51.749198+0000","last_became_active":"2024-09-17T22:41:19.445923+0000","last_became_peered":"2024-09-17T22:41:19.445923+0000","last_unstale":"2024-09-17T22:43:51.749198+0000","last_undegraded":"2024-09-17T22:43:51.749198+0000","last_fullsized":"2024-09-17T22:43:51.749198+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:02:33.097995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001414085,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3445,"num_read_kb":15105,"num_write":513,"num_write_kb":8714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"129'562","reported_seq":22388,"reported_epoch":155,"state":"peering","last_fresh":"2024-09-17T22:43:51.355701+0000","last_change":"2024-09-17T22:43:51.355701+0000","last_active":"2024-09-17T22:43:31.187738+0000","last_peered":"2024-09-17T22:43:30.548859+0000","last_clean":"2024-09-17T22:43:30.548859+0000","last_became_active":"2024-09-17T22:41:57.132018+0000","last_became_peered":"2024-09-17T22:41:57.132018+0000","last_unstale":"2024-09-17T22:43:51.355701+0000","last_undegraded":"2024-09-17T22:43:51.355701+0000","last_fullsized":"2024-09-17T22:43:51.355701+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016446880000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":22869,"num_read_kb":64042,"num_write":605,"num_write_kb":12652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"139'437","reported_seq":7649,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T22:43:50.359256+0000","last_change":"2024-09-17T22:43:35.287778+0000","last_active":"2024-09-17T22:43:50.359256+0000","last_peered":"2024-09-17T22:43:50.359256+0000","last_clean":"2024-09-17T22:43:50.359256+0000","last_became_active":"2024-09-17T22:43:35.217210+0000","last_became_peered":"2024-09-17T22:43:35.217210+0000","last_unstale":"2024-09-17T22:43:50.359256+0000","last_undegraded":"2024-09-17T22:43:50.359256+0000","last_fullsized":"2024-09-17T22:43:50.359256+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:38:24.310736+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.069534181,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8763,"num_read_kb":20352,"num_write":343,"num_write_kb":4822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2823905,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"129'589","reported_seq":7855,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352780+0000","last_change":"2024-09-17T22:43:19.143328+0000","last_active":"2024-09-17T22:43:51.352780+0000","last_peered":"2024-09-17T22:43:51.352780+0000","last_clean":"2024-09-17T22:43:51.352780+0000","last_became_active":"2024-09-17T22:41:19.451631+0000","last_became_peered":"2024-09-17T22:41:19.451631+0000","last_unstale":"2024-09-17T22:43:51.352780+0000","last_undegraded":"2024-09-17T22:43:51.352780+0000","last_fullsized":"2024-09-17T22:43:51.352780+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:31.809865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090157099999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8321,"num_read_kb":19442,"num_write":736,"num_write_kb":19536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":293,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352731+0000","last_change":"2024-09-17T22:43:32.195958+0000","last_active":"2024-09-17T22:43:51.352731+0000","last_peered":"2024-09-17T22:43:51.352731+0000","last_clean":"2024-09-17T22:43:51.352731+0000","last_became_active":"2024-09-17T22:43:32.195676+0000","last_became_peered":"2024-09-17T22:43:32.195676+0000","last_unstale":"2024-09-17T22:43:51.352731+0000","last_undegraded":"2024-09-17T22:43:51.352731+0000","last_fullsized":"2024-09-17T22:43:51.352731+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:41:52.669174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"131'579","reported_seq":5338,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352815+0000","last_change":"2024-09-17T22:43:19.150601+0000","last_active":"2024-09-17T22:43:51.352815+0000","last_peered":"2024-09-17T22:43:51.352815+0000","last_clean":"2024-09-17T22:43:51.352815+0000","last_became_active":"2024-09-17T22:41:19.448966+0000","last_became_peered":"2024-09-17T22:41:19.448966+0000","last_unstale":"2024-09-17T22:43:51.352815+0000","last_undegraded":"2024-09-17T22:43:51.352815+0000","last_fullsized":"2024-09-17T22:43:51.352815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:20:32.636962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026117269999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6258,"num_read_kb":29920,"num_write":539,"num_write_kb":6243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":292,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352861+0000","last_change":"2024-09-17T22:43:32.195712+0000","last_active":"2024-09-17T22:43:51.352861+0000","last_peered":"2024-09-17T22:43:51.352861+0000","last_clean":"2024-09-17T22:43:51.352861+0000","last_became_active":"2024-09-17T22:43:32.195451+0000","last_became_peered":"2024-09-17T22:43:32.195451+0000","last_unstale":"2024-09-17T22:43:51.352861+0000","last_undegraded":"2024-09-17T22:43:51.352861+0000","last_fullsized":"2024-09-17T22:43:51.352861+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:52.062601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"131'396","reported_seq":1773,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352783+0000","last_change":"2024-09-17T22:43:19.149604+0000","last_active":"2024-09-17T22:43:51.352783+0000","last_peered":"2024-09-17T22:43:51.352783+0000","last_clean":"2024-09-17T22:43:51.352783+0000","last_became_active":"2024-09-17T22:41:19.451637+0000","last_became_peered":"2024-09-17T22:41:19.451637+0000","last_unstale":"2024-09-17T22:43:51.352783+0000","last_undegraded":"2024-09-17T22:43:51.352783+0000","last_fullsized":"2024-09-17T22:43:51.352783+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:16.254887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001593059,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1474,"num_read_kb":9975,"num_write":408,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":291,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352906+0000","last_change":"2024-09-17T22:43:15.154296+0000","last_active":"2024-09-17T22:43:51.352906+0000","last_peered":"2024-09-17T22:43:51.352906+0000","last_clean":"2024-09-17T22:43:51.352906+0000","last_became_active":"2024-09-17T22:43:15.153956+0000","last_became_peered":"2024-09-17T22:43:15.153956+0000","last_unstale":"2024-09-17T22:43:51.352906+0000","last_undegraded":"2024-09-17T22:43:51.352906+0000","last_fullsized":"2024-09-17T22:43:51.352906+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:19.340692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"131'478","reported_seq":1431,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357944+0000","last_change":"2024-09-17T22:43:19.151215+0000","last_active":"2024-09-17T22:43:51.357944+0000","last_peered":"2024-09-17T22:43:51.357944+0000","last_clean":"2024-09-17T22:43:51.357944+0000","last_became_active":"2024-09-17T22:41:19.444893+0000","last_became_peered":"2024-09-17T22:41:19.444893+0000","last_unstale":"2024-09-17T22:43:51.357944+0000","last_undegraded":"2024-09-17T22:43:51.357944+0000","last_fullsized":"2024-09-17T22:43:51.357944+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:34:31.498807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001189308,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1718,"num_read_kb":9540,"num_write":633,"num_write_kb":7900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":286,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749190+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:43:51.749190+0000","last_peered":"2024-09-17T22:43:51.749190+0000","last_clean":"2024-09-17T22:43:51.749190+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:43:51.749190+0000","last_undegraded":"2024-09-17T22:43:51.749190+0000","last_fullsized":"2024-09-17T22:43:51.749190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"130'393","reported_seq":15617,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352661+0000","last_change":"2024-09-17T22:43:19.187091+0000","last_active":"2024-09-17T22:43:51.352661+0000","last_peered":"2024-09-17T22:43:51.352661+0000","last_clean":"2024-09-17T22:43:51.352661+0000","last_became_active":"2024-09-17T22:41:19.451723+0000","last_became_peered":"2024-09-17T22:41:19.451723+0000","last_unstale":"2024-09-17T22:43:51.352661+0000","last_undegraded":"2024-09-17T22:43:51.352661+0000","last_fullsized":"2024-09-17T22:43:51.352661+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:50:14.037236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061483000000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":15368,"num_read_kb":33652,"num_write":428,"num_write_kb":8748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"139'879","reported_seq":12150,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-17T22:43:48.852590+0000","last_change":"2024-09-17T22:43:35.334520+0000","last_active":"2024-09-17T22:43:48.852590+0000","last_peered":"2024-09-17T22:43:48.852590+0000","last_clean":"2024-09-17T22:43:48.852590+0000","last_became_active":"2024-09-17T22:43:35.220905+0000","last_became_peered":"2024-09-17T22:43:35.220905+0000","last_unstale":"2024-09-17T22:43:48.852590+0000","last_undegraded":"2024-09-17T22:43:48.852590+0000","last_fullsized":"2024-09-17T22:43:48.852590+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:11:27.480976+0000","scrub_duration":0,"objects_trimmed":42,"snaptrim_duration":0.112643386,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":15325,"num_read_kb":33693,"num_write":856,"num_write_kb":8706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":4229520,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":202,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352969+0000","last_change":"2024-09-17T22:43:42.285641+0000","last_active":"2024-09-17T22:43:51.352969+0000","last_peered":"2024-09-17T22:43:51.352969+0000","last_clean":"2024-09-17T22:43:51.352969+0000","last_became_active":"2024-09-17T22:43:42.285307+0000","last_became_peered":"2024-09-17T22:43:42.285307+0000","last_unstale":"2024-09-17T22:43:51.352969+0000","last_undegraded":"2024-09-17T22:43:51.352969+0000","last_fullsized":"2024-09-17T22:43:51.352969+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:39.203508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"130'385","reported_seq":4190,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749152+0000","last_change":"2024-09-17T22:43:19.191554+0000","last_active":"2024-09-17T22:43:51.749152+0000","last_peered":"2024-09-17T22:43:51.749152+0000","last_clean":"2024-09-17T22:43:51.749152+0000","last_became_active":"2024-09-17T22:41:19.444314+0000","last_became_peered":"2024-09-17T22:41:19.444314+0000","last_unstale":"2024-09-17T22:43:51.749152+0000","last_undegraded":"2024-09-17T22:43:51.749152+0000","last_fullsized":"2024-09-17T22:43:51.749152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:07:00.374588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011857370000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4006,"num_read_kb":22354,"num_write":358,"num_write_kb":9353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":286,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.358312+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:43:51.358312+0000","last_peered":"2024-09-17T22:43:51.358312+0000","last_clean":"2024-09-17T22:43:51.358312+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:43:51.358312+0000","last_undegraded":"2024-09-17T22:43:51.358312+0000","last_fullsized":"2024-09-17T22:43:51.358312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"130'481","reported_seq":7455,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.358137+0000","last_change":"2024-09-17T22:43:19.150805+0000","last_active":"2024-09-17T22:43:51.358137+0000","last_peered":"2024-09-17T22:43:51.358137+0000","last_clean":"2024-09-17T22:43:51.358137+0000","last_became_active":"2024-09-17T22:41:19.449791+0000","last_became_peered":"2024-09-17T22:41:19.449791+0000","last_unstale":"2024-09-17T22:43:51.358137+0000","last_undegraded":"2024-09-17T22:43:51.358137+0000","last_fullsized":"2024-09-17T22:43:51.358137+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:48.110050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088469700000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8092,"num_read_kb":9247,"num_write":519,"num_write_kb":4605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":286,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352571+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:43:51.352571+0000","last_peered":"2024-09-17T22:43:51.352571+0000","last_clean":"2024-09-17T22:43:51.352571+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:43:51.352571+0000","last_undegraded":"2024-09-17T22:43:51.352571+0000","last_fullsized":"2024-09-17T22:43:51.352571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"131'756","reported_seq":8739,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352735+0000","last_change":"2024-09-17T22:43:19.148879+0000","last_active":"2024-09-17T22:43:51.352735+0000","last_peered":"2024-09-17T22:43:51.352735+0000","last_clean":"2024-09-17T22:43:51.352735+0000","last_became_active":"2024-09-17T22:41:19.449279+0000","last_became_peered":"2024-09-17T22:41:19.449279+0000","last_unstale":"2024-09-17T22:43:51.352735+0000","last_undegraded":"2024-09-17T22:43:51.352735+0000","last_fullsized":"2024-09-17T22:43:51.352735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:51:59.677897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089414599999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":11291,"num_read_kb":22947,"num_write":956,"num_write_kb":2860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":201,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.353170+0000","last_change":"2024-09-17T22:43:42.286959+0000","last_active":"2024-09-17T22:43:51.353170+0000","last_peered":"2024-09-17T22:43:51.353170+0000","last_clean":"2024-09-17T22:43:51.353170+0000","last_became_active":"2024-09-17T22:43:42.286652+0000","last_became_peered":"2024-09-17T22:43:42.286652+0000","last_unstale":"2024-09-17T22:43:51.353170+0000","last_undegraded":"2024-09-17T22:43:51.353170+0000","last_fullsized":"2024-09-17T22:43:51.353170+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:50:39.824946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":438,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357877+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:43:51.357877+0000","last_peered":"2024-09-17T22:43:51.357877+0000","last_clean":"2024-09-17T22:43:51.357877+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:43:51.357877+0000","last_undegraded":"2024-09-17T22:43:51.357877+0000","last_fullsized":"2024-09-17T22:43:51.357877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"129'613","reported_seq":11950,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352720+0000","last_change":"2024-09-17T22:43:19.149769+0000","last_active":"2024-09-17T22:43:51.352720+0000","last_peered":"2024-09-17T22:43:51.352720+0000","last_clean":"2024-09-17T22:43:51.352720+0000","last_became_active":"2024-09-17T22:41:19.448956+0000","last_became_peered":"2024-09-17T22:41:19.448956+0000","last_unstale":"2024-09-17T22:43:51.352720+0000","last_undegraded":"2024-09-17T22:43:51.352720+0000","last_fullsized":"2024-09-17T22:43:51.352720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:28:45.951042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017463789999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13252,"num_read_kb":46911,"num_write":655,"num_write_kb":25175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"129'785","reported_seq":14759,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352803+0000","last_change":"2024-09-17T22:43:19.145329+0000","last_active":"2024-09-17T22:43:51.352803+0000","last_peered":"2024-09-17T22:43:51.352803+0000","last_clean":"2024-09-17T22:43:51.352803+0000","last_became_active":"2024-09-17T22:41:19.449003+0000","last_became_peered":"2024-09-17T22:41:19.449003+0000","last_unstale":"2024-09-17T22:43:51.352803+0000","last_undegraded":"2024-09-17T22:43:51.352803+0000","last_fullsized":"2024-09-17T22:43:51.352803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:13:00.810670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910964,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":16223,"num_read_kb":43243,"num_write":1040,"num_write_kb":9843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"130'470","reported_seq":8484,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352888+0000","last_change":"2024-09-17T22:43:42.285286+0000","last_active":"2024-09-17T22:43:51.352888+0000","last_peered":"2024-09-17T22:43:51.352888+0000","last_clean":"2024-09-17T22:43:51.352888+0000","last_became_active":"2024-09-17T22:43:42.284094+0000","last_became_peered":"2024-09-17T22:43:42.284094+0000","last_unstale":"2024-09-17T22:43:51.352888+0000","last_undegraded":"2024-09-17T22:43:51.352888+0000","last_fullsized":"2024-09-17T22:43:51.352888+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:36:47.430783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001830772,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":8191,"num_read_kb":41081,"num_write":473,"num_write_kb":9769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"131'423","reported_seq":2735,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.358360+0000","last_change":"2024-09-17T22:43:19.142462+0000","last_active":"2024-09-17T22:43:51.358360+0000","last_peered":"2024-09-17T22:43:51.358360+0000","last_clean":"2024-09-17T22:43:51.358360+0000","last_became_active":"2024-09-17T22:41:19.447215+0000","last_became_peered":"2024-09-17T22:41:19.447215+0000","last_unstale":"2024-09-17T22:43:51.358360+0000","last_undegraded":"2024-09-17T22:43:51.358360+0000","last_fullsized":"2024-09-17T22:43:51.358360+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:37:48.721121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066798900000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3409,"num_read_kb":14480,"num_write":404,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"139'353","reported_seq":21720,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T22:43:50.359231+0000","last_change":"2024-09-17T22:43:35.288547+0000","last_active":"2024-09-17T22:43:50.359231+0000","last_peered":"2024-09-17T22:43:50.359231+0000","last_clean":"2024-09-17T22:43:50.359231+0000","last_became_active":"2024-09-17T22:43:35.216950+0000","last_became_peered":"2024-09-17T22:43:35.216950+0000","last_unstale":"2024-09-17T22:43:50.359231+0000","last_undegraded":"2024-09-17T22:43:50.359231+0000","last_fullsized":"2024-09-17T22:43:50.359231+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:01:37.332938+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.070463135999999996,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":21473,"num_read_kb":63694,"num_write":340,"num_write_kb":14940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":1192963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"129'486","reported_seq":2029,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352846+0000","last_change":"2024-09-17T22:43:42.283323+0000","last_active":"2024-09-17T22:43:51.352846+0000","last_peered":"2024-09-17T22:43:51.352846+0000","last_clean":"2024-09-17T22:43:51.352846+0000","last_became_active":"2024-09-17T22:43:42.282252+0000","last_became_peered":"2024-09-17T22:43:42.282252+0000","last_unstale":"2024-09-17T22:43:51.352846+0000","last_undegraded":"2024-09-17T22:43:51.352846+0000","last_fullsized":"2024-09-17T22:43:51.352846+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:31.315273+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.002173033,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2216,"num_read_kb":10204,"num_write":574,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"139'372","reported_seq":8837,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352997+0000","last_change":"2024-09-17T22:43:35.323373+0000","last_active":"2024-09-17T22:43:51.352997+0000","last_peered":"2024-09-17T22:43:51.352997+0000","last_clean":"2024-09-17T22:43:51.352997+0000","last_became_active":"2024-09-17T22:43:35.224141+0000","last_became_peered":"2024-09-17T22:43:35.224141+0000","last_unstale":"2024-09-17T22:43:51.352997+0000","last_undegraded":"2024-09-17T22:43:51.352997+0000","last_fullsized":"2024-09-17T22:43:51.352997+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:01:02.683062+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.098411079999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8804,"num_read_kb":23106,"num_write":291,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":2514411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"139'353","reported_seq":1880,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T22:43:46.318517+0000","last_change":"2024-09-17T22:43:35.320798+0000","last_active":"2024-09-17T22:43:46.318517+0000","last_peered":"2024-09-17T22:43:46.318517+0000","last_clean":"2024-09-17T22:43:46.318517+0000","last_became_active":"2024-09-17T22:43:35.217622+0000","last_became_peered":"2024-09-17T22:43:35.217622+0000","last_unstale":"2024-09-17T22:43:46.318517+0000","last_undegraded":"2024-09-17T22:43:46.318517+0000","last_fullsized":"2024-09-17T22:43:46.318517+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:52:51.375191+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.102357168,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1634,"num_read_kb":5501,"num_write":380,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1563844,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"129'349","reported_seq":2170,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357762+0000","last_change":"2024-09-17T22:43:19.144272+0000","last_active":"2024-09-17T22:43:51.357762+0000","last_peered":"2024-09-17T22:43:51.357762+0000","last_clean":"2024-09-17T22:43:51.357762+0000","last_became_active":"2024-09-17T22:41:19.447419+0000","last_became_peered":"2024-09-17T22:41:19.447419+0000","last_unstale":"2024-09-17T22:43:51.357762+0000","last_undegraded":"2024-09-17T22:43:51.357762+0000","last_fullsized":"2024-09-17T22:43:51.357762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:48.842752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00132408,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3110,"num_read_kb":8489,"num_write":286,"num_write_kb":3161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"129'616","reported_seq":3679,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352697+0000","last_change":"2024-09-17T22:43:19.144657+0000","last_active":"2024-09-17T22:43:51.352697+0000","last_peered":"2024-09-17T22:43:51.352697+0000","last_clean":"2024-09-17T22:43:51.352697+0000","last_became_active":"2024-09-17T22:41:19.454151+0000","last_became_peered":"2024-09-17T22:41:19.454151+0000","last_unstale":"2024-09-17T22:43:51.352697+0000","last_undegraded":"2024-09-17T22:43:51.352697+0000","last_fullsized":"2024-09-17T22:43:51.352697+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:39:51.641693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001718979,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5266,"num_read_kb":13546,"num_write":658,"num_write_kb":3731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"130'910","reported_seq":11717,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357917+0000","last_change":"2024-09-17T22:43:35.218645+0000","last_active":"2024-09-17T22:43:51.357917+0000","last_peered":"2024-09-17T22:43:51.357917+0000","last_clean":"2024-09-17T22:43:51.357917+0000","last_became_active":"2024-09-17T22:43:35.217045+0000","last_became_peered":"2024-09-17T22:43:35.217045+0000","last_unstale":"2024-09-17T22:43:51.357917+0000","last_undegraded":"2024-09-17T22:43:51.357917+0000","last_fullsized":"2024-09-17T22:43:51.357917+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:20.169030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001497735,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14642,"num_read_kb":44074,"num_write":1142,"num_write_kb":22161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"139'502","reported_seq":2628,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.353075+0000","last_change":"2024-09-17T22:43:35.301824+0000","last_active":"2024-09-17T22:43:51.353075+0000","last_peered":"2024-09-17T22:43:51.353075+0000","last_clean":"2024-09-17T22:43:51.353075+0000","last_became_active":"2024-09-17T22:43:35.217551+0000","last_became_peered":"2024-09-17T22:43:35.217551+0000","last_unstale":"2024-09-17T22:43:51.353075+0000","last_undegraded":"2024-09-17T22:43:51.353075+0000","last_fullsized":"2024-09-17T22:43:51.353075+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:10.204124+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.082900606000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":2503,"num_read_kb":8800,"num_write":617,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":291987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":492,"num_read":268105,"num_read_kb":802236,"num_write":18141,"num_write_kb":293703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":30918130,"num_keys_recovered":86,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11879,"internal_metadata":0},"log_size":16633,"ondisk_log_size":16633,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542183,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28908,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342932,"statfs":{"total":96636764160,"available":96607162368,"internally_reserved":0,"allocated":1921024,"data_stored":1492409,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":135,"seq":579820584966,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34724,"kb_used_data":1804,"kb_used_omap":21,"kb_used_meta":32874,"kb_avail":94337116,"statfs":{"total":96636764160,"available":96601206784,"internally_reserved":0,"allocated":1847296,"data_stored":896064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22295,"internal_metadata":33663209},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542181,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41320,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330520,"statfs":{"total":96636764160,"available":96594452480,"internally_reserved":0,"allocated":1847296,"data_stored":896064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542181,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28848,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342992,"statfs":{"total":96636764160,"available":96607223808,"internally_reserved":0,"allocated":1863680,"data_stored":902041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542181,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46948,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324892,"statfs":{"total":96636764160,"available":96588689408,"internally_reserved":0,"allocated":1908736,"data_stored":1486451,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28848,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342992,"statfs":{"total":96636764160,"available":96607223808,"internally_reserved":0,"allocated":1859584,"data_stored":902022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55212,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316628,"statfs":{"total":96636764160,"available":96580227072,"internally_reserved":0,"allocated":1851392,"data_stored":896083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51052,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320788,"statfs":{"total":96636764160,"available":96584486912,"internally_reserved":0,"allocated":1859584,"data_stored":902022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11879,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:43:53.195 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:43:53.475 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:43:53.476 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:43:53.487 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2024-09-17T22:43:52.373922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":500,"num_object_clones":0,"num_object_copies":1000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":500,"num_whiteouts":492,"num_read":268221,"num_read_kb":802336,"num_write":18302,"num_write_kb":296121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":30918130,"num_keys_recovered":86,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16753,"ondisk_log_size":16753,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":315860,"kb_used_data":14608,"kb_used_omap":32,"kb_used_meta":301023,"kb_avail":754658860,"statfs":{"total":773094113280,"available":772770672640,"internally_reserved":0,"allocated":14958592,"data_stored":8373156,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":33450,"internal_metadata":308247894},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-114,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035823"},"pg_stats":[{"pgid":"3.1f","version":"131'860","reported_seq":7310,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749216+0000","last_change":"2024-09-17T22:43:19.191421+0000","last_active":"2024-09-17T22:43:51.749216+0000","last_peered":"2024-09-17T22:43:51.749216+0000","last_clean":"2024-09-17T22:43:51.749216+0000","last_became_active":"2024-09-17T22:41:19.443658+0000","last_became_peered":"2024-09-17T22:41:19.443658+0000","last_unstale":"2024-09-17T22:43:51.749216+0000","last_undegraded":"2024-09-17T22:43:51.749216+0000","last_fullsized":"2024-09-17T22:43:51.749216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:58.423329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001073428,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8816,"num_read_kb":14906,"num_write":1102,"num_write_kb":4548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"130'786","reported_seq":7422,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.748986+0000","last_change":"2024-09-17T22:43:19.190807+0000","last_active":"2024-09-17T22:43:51.748986+0000","last_peered":"2024-09-17T22:43:51.748986+0000","last_clean":"2024-09-17T22:43:51.748986+0000","last_became_active":"2024-09-17T22:41:19.449357+0000","last_became_peered":"2024-09-17T22:41:19.449357+0000","last_unstale":"2024-09-17T22:43:51.748986+0000","last_undegraded":"2024-09-17T22:43:51.748986+0000","last_fullsized":"2024-09-17T22:43:51.748986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:20.300342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051110000000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":8865,"num_read_kb":26537,"num_write":1126,"num_write_kb":10958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"131'435","reported_seq":2380,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357945+0000","last_change":"2024-09-17T22:43:19.144579+0000","last_active":"2024-09-17T22:43:51.357945+0000","last_peered":"2024-09-17T22:43:51.357945+0000","last_clean":"2024-09-17T22:43:51.357945+0000","last_became_active":"2024-09-17T22:42:58.002822+0000","last_became_peered":"2024-09-17T22:42:58.002822+0000","last_unstale":"2024-09-17T22:43:51.357945+0000","last_undegraded":"2024-09-17T22:43:51.357945+0000","last_fullsized":"2024-09-17T22:43:51.357945+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:18:33.730453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022657670000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":17641,"num_read_kb":48444,"num_write":377,"num_write_kb":8987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":7308630,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"131'494","reported_seq":2826,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357881+0000","last_change":"2024-09-17T22:43:19.146655+0000","last_active":"2024-09-17T22:43:51.357881+0000","last_peered":"2024-09-17T22:43:51.357881+0000","last_clean":"2024-09-17T22:43:51.357881+0000","last_became_active":"2024-09-17T22:42:34.310411+0000","last_became_peered":"2024-09-17T22:42:34.310411+0000","last_unstale":"2024-09-17T22:43:51.357881+0000","last_undegraded":"2024-09-17T22:43:51.357881+0000","last_fullsized":"2024-09-17T22:43:51.357881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:34.577953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039390110000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":2615,"num_read_kb":14215,"num_write":599,"num_write_kb":12172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":7352772,"num_keys_recovered":74,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"139'244","reported_seq":8116,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352818+0000","last_change":"2024-09-17T22:43:35.308387+0000","last_active":"2024-09-17T22:43:51.352818+0000","last_peered":"2024-09-17T22:43:51.352818+0000","last_clean":"2024-09-17T22:43:51.352818+0000","last_became_active":"2024-09-17T22:43:35.215952+0000","last_became_peered":"2024-09-17T22:43:35.215952+0000","last_unstale":"2024-09-17T22:43:51.352818+0000","last_undegraded":"2024-09-17T22:43:51.352818+0000","last_fullsized":"2024-09-17T22:43:51.352818+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:45.376055+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.091482261999999995,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7315,"num_read_kb":22822,"num_write":186,"num_write_kb":6215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":3640098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"131'360","reported_seq":5241,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357959+0000","last_change":"2024-09-17T22:43:19.145987+0000","last_active":"2024-09-17T22:43:51.357959+0000","last_peered":"2024-09-17T22:43:51.357959+0000","last_clean":"2024-09-17T22:43:51.357959+0000","last_became_active":"2024-09-17T22:41:19.446541+0000","last_became_peered":"2024-09-17T22:41:19.446541+0000","last_unstale":"2024-09-17T22:43:51.357959+0000","last_undegraded":"2024-09-17T22:43:51.357959+0000","last_fullsized":"2024-09-17T22:43:51.357959+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:12:51.399521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035192800000000001,"stat_sum":{"num_bytes":19,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":5091,"num_read_kb":19688,"num_write":329,"num_write_kb":4179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"130'391","reported_seq":6764,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352819+0000","last_change":"2024-09-17T22:43:19.150923+0000","last_active":"2024-09-17T22:43:51.352819+0000","last_peered":"2024-09-17T22:43:51.352819+0000","last_clean":"2024-09-17T22:43:51.352819+0000","last_became_active":"2024-09-17T22:41:19.449940+0000","last_became_peered":"2024-09-17T22:41:19.449940+0000","last_unstale":"2024-09-17T22:43:51.352819+0000","last_undegraded":"2024-09-17T22:43:51.352819+0000","last_fullsized":"2024-09-17T22:43:51.352819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:57:32.786125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002665995,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6668,"num_read_kb":22143,"num_write":345,"num_write_kb":9035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"129'383","reported_seq":3221,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749001+0000","last_change":"2024-09-17T22:43:19.191965+0000","last_active":"2024-09-17T22:43:51.749001+0000","last_peered":"2024-09-17T22:43:51.749001+0000","last_clean":"2024-09-17T22:43:51.749001+0000","last_became_active":"2024-09-17T22:41:19.446287+0000","last_became_peered":"2024-09-17T22:41:19.446287+0000","last_unstale":"2024-09-17T22:43:51.749001+0000","last_undegraded":"2024-09-17T22:43:51.749001+0000","last_fullsized":"2024-09-17T22:43:51.749001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:23:52.909972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001462812,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3441,"num_read_kb":20083,"num_write":323,"num_write_kb":10930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"131'513","reported_seq":2752,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749198+0000","last_change":"2024-09-17T22:43:19.191786+0000","last_active":"2024-09-17T22:43:51.749198+0000","last_peered":"2024-09-17T22:43:51.749198+0000","last_clean":"2024-09-17T22:43:51.749198+0000","last_became_active":"2024-09-17T22:41:19.445923+0000","last_became_peered":"2024-09-17T22:41:19.445923+0000","last_unstale":"2024-09-17T22:43:51.749198+0000","last_undegraded":"2024-09-17T22:43:51.749198+0000","last_fullsized":"2024-09-17T22:43:51.749198+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:02:33.097995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001414085,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3445,"num_read_kb":15105,"num_write":513,"num_write_kb":8714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"129'562","reported_seq":22388,"reported_epoch":155,"state":"peering","last_fresh":"2024-09-17T22:43:51.355701+0000","last_change":"2024-09-17T22:43:51.355701+0000","last_active":"2024-09-17T22:43:31.187738+0000","last_peered":"2024-09-17T22:43:30.548859+0000","last_clean":"2024-09-17T22:43:30.548859+0000","last_became_active":"2024-09-17T22:41:57.132018+0000","last_became_peered":"2024-09-17T22:41:57.132018+0000","last_unstale":"2024-09-17T22:43:51.355701+0000","last_undegraded":"2024-09-17T22:43:51.355701+0000","last_fullsized":"2024-09-17T22:43:51.355701+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016446880000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":22869,"num_read_kb":64042,"num_write":605,"num_write_kb":12652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"139'437","reported_seq":7649,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T22:43:50.359256+0000","last_change":"2024-09-17T22:43:35.287778+0000","last_active":"2024-09-17T22:43:50.359256+0000","last_peered":"2024-09-17T22:43:50.359256+0000","last_clean":"2024-09-17T22:43:50.359256+0000","last_became_active":"2024-09-17T22:43:35.217210+0000","last_became_peered":"2024-09-17T22:43:35.217210+0000","last_unstale":"2024-09-17T22:43:50.359256+0000","last_undegraded":"2024-09-17T22:43:50.359256+0000","last_fullsized":"2024-09-17T22:43:50.359256+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:38:24.310736+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.069534181,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8763,"num_read_kb":20352,"num_write":343,"num_write_kb":4822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2823905,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"129'589","reported_seq":7855,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352780+0000","last_change":"2024-09-17T22:43:19.143328+0000","last_active":"2024-09-17T22:43:51.352780+0000","last_peered":"2024-09-17T22:43:51.352780+0000","last_clean":"2024-09-17T22:43:51.352780+0000","last_became_active":"2024-09-17T22:41:19.451631+0000","last_became_peered":"2024-09-17T22:41:19.451631+0000","last_unstale":"2024-09-17T22:43:51.352780+0000","last_undegraded":"2024-09-17T22:43:51.352780+0000","last_fullsized":"2024-09-17T22:43:51.352780+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:31.809865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090157099999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8321,"num_read_kb":19442,"num_write":736,"num_write_kb":19536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":293,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352731+0000","last_change":"2024-09-17T22:43:32.195958+0000","last_active":"2024-09-17T22:43:51.352731+0000","last_peered":"2024-09-17T22:43:51.352731+0000","last_clean":"2024-09-17T22:43:51.352731+0000","last_became_active":"2024-09-17T22:43:32.195676+0000","last_became_peered":"2024-09-17T22:43:32.195676+0000","last_unstale":"2024-09-17T22:43:51.352731+0000","last_undegraded":"2024-09-17T22:43:51.352731+0000","last_fullsized":"2024-09-17T22:43:51.352731+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:41:52.669174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"131'579","reported_seq":5338,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352815+0000","last_change":"2024-09-17T22:43:19.150601+0000","last_active":"2024-09-17T22:43:51.352815+0000","last_peered":"2024-09-17T22:43:51.352815+0000","last_clean":"2024-09-17T22:43:51.352815+0000","last_became_active":"2024-09-17T22:41:19.448966+0000","last_became_peered":"2024-09-17T22:41:19.448966+0000","last_unstale":"2024-09-17T22:43:51.352815+0000","last_undegraded":"2024-09-17T22:43:51.352815+0000","last_fullsized":"2024-09-17T22:43:51.352815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:20:32.636962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026117269999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6258,"num_read_kb":29920,"num_write":539,"num_write_kb":6243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":292,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352861+0000","last_change":"2024-09-17T22:43:32.195712+0000","last_active":"2024-09-17T22:43:51.352861+0000","last_peered":"2024-09-17T22:43:51.352861+0000","last_clean":"2024-09-17T22:43:51.352861+0000","last_became_active":"2024-09-17T22:43:32.195451+0000","last_became_peered":"2024-09-17T22:43:32.195451+0000","last_unstale":"2024-09-17T22:43:51.352861+0000","last_undegraded":"2024-09-17T22:43:51.352861+0000","last_fullsized":"2024-09-17T22:43:51.352861+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:52.062601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"131'396","reported_seq":1773,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352783+0000","last_change":"2024-09-17T22:43:19.149604+0000","last_active":"2024-09-17T22:43:51.352783+0000","last_peered":"2024-09-17T22:43:51.352783+0000","last_clean":"2024-09-17T22:43:51.352783+0000","last_became_active":"2024-09-17T22:41:19.451637+0000","last_became_peered":"2024-09-17T22:41:19.451637+0000","last_unstale":"2024-09-17T22:43:51.352783+0000","last_undegraded":"2024-09-17T22:43:51.352783+0000","last_fullsized":"2024-09-17T22:43:51.352783+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:16.254887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001593059,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1474,"num_read_kb":9975,"num_write":408,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":291,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352906+0000","last_change":"2024-09-17T22:43:15.154296+0000","last_active":"2024-09-17T22:43:51.352906+0000","last_peered":"2024-09-17T22:43:51.352906+0000","last_clean":"2024-09-17T22:43:51.352906+0000","last_became_active":"2024-09-17T22:43:15.153956+0000","last_became_peered":"2024-09-17T22:43:15.153956+0000","last_unstale":"2024-09-17T22:43:51.352906+0000","last_undegraded":"2024-09-17T22:43:51.352906+0000","last_fullsized":"2024-09-17T22:43:51.352906+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:19.340692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"131'478","reported_seq":1431,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357944+0000","last_change":"2024-09-17T22:43:19.151215+0000","last_active":"2024-09-17T22:43:51.357944+0000","last_peered":"2024-09-17T22:43:51.357944+0000","last_clean":"2024-09-17T22:43:51.357944+0000","last_became_active":"2024-09-17T22:41:19.444893+0000","last_became_peered":"2024-09-17T22:41:19.444893+0000","last_unstale":"2024-09-17T22:43:51.357944+0000","last_undegraded":"2024-09-17T22:43:51.357944+0000","last_fullsized":"2024-09-17T22:43:51.357944+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:34:31.498807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001189308,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1718,"num_read_kb":9540,"num_write":633,"num_write_kb":7900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":286,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749190+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:43:51.749190+0000","last_peered":"2024-09-17T22:43:51.749190+0000","last_clean":"2024-09-17T22:43:51.749190+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:43:51.749190+0000","last_undegraded":"2024-09-17T22:43:51.749190+0000","last_fullsized":"2024-09-17T22:43:51.749190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"130'393","reported_seq":15617,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352661+0000","last_change":"2024-09-17T22:43:19.187091+0000","last_active":"2024-09-17T22:43:51.352661+0000","last_peered":"2024-09-17T22:43:51.352661+0000","last_clean":"2024-09-17T22:43:51.352661+0000","last_became_active":"2024-09-17T22:41:19.451723+0000","last_became_peered":"2024-09-17T22:41:19.451723+0000","last_unstale":"2024-09-17T22:43:51.352661+0000","last_undegraded":"2024-09-17T22:43:51.352661+0000","last_fullsized":"2024-09-17T22:43:51.352661+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:50:14.037236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061483000000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":15368,"num_read_kb":33652,"num_write":428,"num_write_kb":8748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"139'879","reported_seq":12150,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-17T22:43:48.852590+0000","last_change":"2024-09-17T22:43:35.334520+0000","last_active":"2024-09-17T22:43:48.852590+0000","last_peered":"2024-09-17T22:43:48.852590+0000","last_clean":"2024-09-17T22:43:48.852590+0000","last_became_active":"2024-09-17T22:43:35.220905+0000","last_became_peered":"2024-09-17T22:43:35.220905+0000","last_unstale":"2024-09-17T22:43:48.852590+0000","last_undegraded":"2024-09-17T22:43:48.852590+0000","last_fullsized":"2024-09-17T22:43:48.852590+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:11:27.480976+0000","scrub_duration":0,"objects_trimmed":42,"snaptrim_duration":0.112643386,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":15325,"num_read_kb":33693,"num_write":856,"num_write_kb":8706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":4229520,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":202,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352969+0000","last_change":"2024-09-17T22:43:42.285641+0000","last_active":"2024-09-17T22:43:51.352969+0000","last_peered":"2024-09-17T22:43:51.352969+0000","last_clean":"2024-09-17T22:43:51.352969+0000","last_became_active":"2024-09-17T22:43:42.285307+0000","last_became_peered":"2024-09-17T22:43:42.285307+0000","last_unstale":"2024-09-17T22:43:51.352969+0000","last_undegraded":"2024-09-17T22:43:51.352969+0000","last_fullsized":"2024-09-17T22:43:51.352969+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:39.203508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"130'385","reported_seq":4190,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749152+0000","last_change":"2024-09-17T22:43:19.191554+0000","last_active":"2024-09-17T22:43:51.749152+0000","last_peered":"2024-09-17T22:43:51.749152+0000","last_clean":"2024-09-17T22:43:51.749152+0000","last_became_active":"2024-09-17T22:41:19.444314+0000","last_became_peered":"2024-09-17T22:41:19.444314+0000","last_unstale":"2024-09-17T22:43:51.749152+0000","last_undegraded":"2024-09-17T22:43:51.749152+0000","last_fullsized":"2024-09-17T22:43:51.749152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:07:00.374588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011857370000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4006,"num_read_kb":22354,"num_write":358,"num_write_kb":9353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":286,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.358312+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:43:51.358312+0000","last_peered":"2024-09-17T22:43:51.358312+0000","last_clean":"2024-09-17T22:43:51.358312+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:43:51.358312+0000","last_undegraded":"2024-09-17T22:43:51.358312+0000","last_fullsized":"2024-09-17T22:43:51.358312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"130'481","reported_seq":7455,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.358137+0000","last_change":"2024-09-17T22:43:19.150805+0000","last_active":"2024-09-17T22:43:51.358137+0000","last_peered":"2024-09-17T22:43:51.358137+0000","last_clean":"2024-09-17T22:43:51.358137+0000","last_became_active":"2024-09-17T22:41:19.449791+0000","last_became_peered":"2024-09-17T22:41:19.449791+0000","last_unstale":"2024-09-17T22:43:51.358137+0000","last_undegraded":"2024-09-17T22:43:51.358137+0000","last_fullsized":"2024-09-17T22:43:51.358137+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:48.110050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088469700000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8092,"num_read_kb":9247,"num_write":519,"num_write_kb":4605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":286,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352571+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:43:51.352571+0000","last_peered":"2024-09-17T22:43:51.352571+0000","last_clean":"2024-09-17T22:43:51.352571+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:43:51.352571+0000","last_undegraded":"2024-09-17T22:43:51.352571+0000","last_fullsized":"2024-09-17T22:43:51.352571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"131'756","reported_seq":8739,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352735+0000","last_change":"2024-09-17T22:43:19.148879+0000","last_active":"2024-09-17T22:43:51.352735+0000","last_peered":"2024-09-17T22:43:51.352735+0000","last_clean":"2024-09-17T22:43:51.352735+0000","last_became_active":"2024-09-17T22:41:19.449279+0000","last_became_peered":"2024-09-17T22:41:19.449279+0000","last_unstale":"2024-09-17T22:43:51.352735+0000","last_undegraded":"2024-09-17T22:43:51.352735+0000","last_fullsized":"2024-09-17T22:43:51.352735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:51:59.677897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089414599999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":11291,"num_read_kb":22947,"num_write":956,"num_write_kb":2860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":201,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.353170+0000","last_change":"2024-09-17T22:43:42.286959+0000","last_active":"2024-09-17T22:43:51.353170+0000","last_peered":"2024-09-17T22:43:51.353170+0000","last_clean":"2024-09-17T22:43:51.353170+0000","last_became_active":"2024-09-17T22:43:42.286652+0000","last_became_peered":"2024-09-17T22:43:42.286652+0000","last_unstale":"2024-09-17T22:43:51.353170+0000","last_undegraded":"2024-09-17T22:43:51.353170+0000","last_fullsized":"2024-09-17T22:43:51.353170+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:50:39.824946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":438,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357877+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:43:51.357877+0000","last_peered":"2024-09-17T22:43:51.357877+0000","last_clean":"2024-09-17T22:43:51.357877+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:43:51.357877+0000","last_undegraded":"2024-09-17T22:43:51.357877+0000","last_fullsized":"2024-09-17T22:43:51.357877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"129'613","reported_seq":11950,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352720+0000","last_change":"2024-09-17T22:43:19.149769+0000","last_active":"2024-09-17T22:43:51.352720+0000","last_peered":"2024-09-17T22:43:51.352720+0000","last_clean":"2024-09-17T22:43:51.352720+0000","last_became_active":"2024-09-17T22:41:19.448956+0000","last_became_peered":"2024-09-17T22:41:19.448956+0000","last_unstale":"2024-09-17T22:43:51.352720+0000","last_undegraded":"2024-09-17T22:43:51.352720+0000","last_fullsized":"2024-09-17T22:43:51.352720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:28:45.951042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017463789999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13252,"num_read_kb":46911,"num_write":655,"num_write_kb":25175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"129'785","reported_seq":14759,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352803+0000","last_change":"2024-09-17T22:43:19.145329+0000","last_active":"2024-09-17T22:43:51.352803+0000","last_peered":"2024-09-17T22:43:51.352803+0000","last_clean":"2024-09-17T22:43:51.352803+0000","last_became_active":"2024-09-17T22:41:19.449003+0000","last_became_peered":"2024-09-17T22:41:19.449003+0000","last_unstale":"2024-09-17T22:43:51.352803+0000","last_undegraded":"2024-09-17T22:43:51.352803+0000","last_fullsized":"2024-09-17T22:43:51.352803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:13:00.810670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910964,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":16223,"num_read_kb":43243,"num_write":1040,"num_write_kb":9843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"130'470","reported_seq":8484,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352888+0000","last_change":"2024-09-17T22:43:42.285286+0000","last_active":"2024-09-17T22:43:51.352888+0000","last_peered":"2024-09-17T22:43:51.352888+0000","last_clean":"2024-09-17T22:43:51.352888+0000","last_became_active":"2024-09-17T22:43:42.284094+0000","last_became_peered":"2024-09-17T22:43:42.284094+0000","last_unstale":"2024-09-17T22:43:51.352888+0000","last_undegraded":"2024-09-17T22:43:51.352888+0000","last_fullsized":"2024-09-17T22:43:51.352888+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:36:47.430783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001830772,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":8191,"num_read_kb":41081,"num_write":473,"num_write_kb":9769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"131'423","reported_seq":2735,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.358360+0000","last_change":"2024-09-17T22:43:19.142462+0000","last_active":"2024-09-17T22:43:51.358360+0000","last_peered":"2024-09-17T22:43:51.358360+0000","last_clean":"2024-09-17T22:43:51.358360+0000","last_became_active":"2024-09-17T22:41:19.447215+0000","last_became_peered":"2024-09-17T22:41:19.447215+0000","last_unstale":"2024-09-17T22:43:51.358360+0000","last_undegraded":"2024-09-17T22:43:51.358360+0000","last_fullsized":"2024-09-17T22:43:51.358360+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:37:48.721121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066798900000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3409,"num_read_kb":14480,"num_write":404,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"139'353","reported_seq":21720,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T22:43:50.359231+0000","last_change":"2024-09-17T22:43:35.288547+0000","last_active":"2024-09-17T22:43:50.359231+0000","last_peered":"2024-09-17T22:43:50.359231+0000","last_clean":"2024-09-17T22:43:50.359231+0000","last_became_active":"2024-09-17T22:43:35.216950+0000","last_became_peered":"2024-09-17T22:43:35.216950+0000","last_unstale":"2024-09-17T22:43:50.359231+0000","last_undegraded":"2024-09-17T22:43:50.359231+0000","last_fullsized":"2024-09-17T22:43:50.359231+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:01:37.332938+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.070463135999999996,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":21473,"num_read_kb":63694,"num_write":340,"num_write_kb":14940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":1192963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"129'486","reported_seq":2029,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352846+0000","last_change":"2024-09-17T22:43:42.283323+0000","last_active":"2024-09-17T22:43:51.352846+0000","last_peered":"2024-09-17T22:43:51.352846+0000","last_clean":"2024-09-17T22:43:51.352846+0000","last_became_active":"2024-09-17T22:43:42.282252+0000","last_became_peered":"2024-09-17T22:43:42.282252+0000","last_unstale":"2024-09-17T22:43:51.352846+0000","last_undegraded":"2024-09-17T22:43:51.352846+0000","last_fullsized":"2024-09-17T22:43:51.352846+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:31.315273+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.002173033,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2216,"num_read_kb":10204,"num_write":574,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"139'372","reported_seq":8837,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352997+0000","last_change":"2024-09-17T22:43:35.323373+0000","last_active":"2024-09-17T22:43:51.352997+0000","last_peered":"2024-09-17T22:43:51.352997+0000","last_clean":"2024-09-17T22:43:51.352997+0000","last_became_active":"2024-09-17T22:43:35.224141+0000","last_became_peered":"2024-09-17T22:43:35.224141+0000","last_unstale":"2024-09-17T22:43:51.352997+0000","last_undegraded":"2024-09-17T22:43:51.352997+0000","last_fullsized":"2024-09-17T22:43:51.352997+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:01:02.683062+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.098411079999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8804,"num_read_kb":23106,"num_write":291,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":2514411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"139'353","reported_seq":1880,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T22:43:46.318517+0000","last_change":"2024-09-17T22:43:35.320798+0000","last_active":"2024-09-17T22:43:46.318517+0000","last_peered":"2024-09-17T22:43:46.318517+0000","last_clean":"2024-09-17T22:43:46.318517+0000","last_became_active":"2024-09-17T22:43:35.217622+0000","last_became_peered":"2024-09-17T22:43:35.217622+0000","last_unstale":"2024-09-17T22:43:46.318517+0000","last_undegraded":"2024-09-17T22:43:46.318517+0000","last_fullsized":"2024-09-17T22:43:46.318517+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:52:51.375191+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.102357168,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1634,"num_read_kb":5501,"num_write":380,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1563844,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"129'349","reported_seq":2170,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357762+0000","last_change":"2024-09-17T22:43:19.144272+0000","last_active":"2024-09-17T22:43:51.357762+0000","last_peered":"2024-09-17T22:43:51.357762+0000","last_clean":"2024-09-17T22:43:51.357762+0000","last_became_active":"2024-09-17T22:41:19.447419+0000","last_became_peered":"2024-09-17T22:41:19.447419+0000","last_unstale":"2024-09-17T22:43:51.357762+0000","last_undegraded":"2024-09-17T22:43:51.357762+0000","last_fullsized":"2024-09-17T22:43:51.357762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:48.842752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00132408,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3110,"num_read_kb":8489,"num_write":286,"num_write_kb":3161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"129'616","reported_seq":3679,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352697+0000","last_change":"2024-09-17T22:43:19.144657+0000","last_active":"2024-09-17T22:43:51.352697+0000","last_peered":"2024-09-17T22:43:51.352697+0000","last_clean":"2024-09-17T22:43:51.352697+0000","last_became_active":"2024-09-17T22:41:19.454151+0000","last_became_peered":"2024-09-17T22:41:19.454151+0000","last_unstale":"2024-09-17T22:43:51.352697+0000","last_undegraded":"2024-09-17T22:43:51.352697+0000","last_fullsized":"2024-09-17T22:43:51.352697+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:39:51.641693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001718979,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5266,"num_read_kb":13546,"num_write":658,"num_write_kb":3731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"130'910","reported_seq":11717,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357917+0000","last_change":"2024-09-17T22:43:35.218645+0000","last_active":"2024-09-17T22:43:51.357917+0000","last_peered":"2024-09-17T22:43:51.357917+0000","last_clean":"2024-09-17T22:43:51.357917+0000","last_became_active":"2024-09-17T22:43:35.217045+0000","last_became_peered":"2024-09-17T22:43:35.217045+0000","last_unstale":"2024-09-17T22:43:51.357917+0000","last_undegraded":"2024-09-17T22:43:51.357917+0000","last_fullsized":"2024-09-17T22:43:51.357917+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:20.169030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001497735,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14642,"num_read_kb":44074,"num_write":1142,"num_write_kb":22161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"139'502","reported_seq":2628,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.353075+0000","last_change":"2024-09-17T22:43:35.301824+0000","last_active":"2024-09-17T22:43:51.353075+0000","last_peered":"2024-09-17T22:43:51.353075+0000","last_clean":"2024-09-17T22:43:51.353075+0000","last_became_active":"2024-09-17T22:43:35.217551+0000","last_became_peered":"2024-09-17T22:43:35.217551+0000","last_unstale":"2024-09-17T22:43:51.353075+0000","last_undegraded":"2024-09-17T22:43:51.353075+0000","last_fullsized":"2024-09-17T22:43:51.353075+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:10.204124+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.082900606000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":2503,"num_read_kb":8800,"num_write":617,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":291987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":492,"num_read":268105,"num_read_kb":802236,"num_write":18141,"num_write_kb":293703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":30918130,"num_keys_recovered":86,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11879,"internal_metadata":0},"log_size":16633,"ondisk_log_size":16633,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542183,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28908,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342932,"statfs":{"total":96636764160,"available":96607162368,"internally_reserved":0,"allocated":1921024,"data_stored":1492409,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":135,"seq":579820584966,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34724,"kb_used_data":1804,"kb_used_omap":21,"kb_used_meta":32874,"kb_avail":94337116,"statfs":{"total":96636764160,"available":96601206784,"internally_reserved":0,"allocated":1847296,"data_stored":896064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22295,"internal_metadata":33663209},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542181,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41320,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330520,"statfs":{"total":96636764160,"available":96594452480,"internally_reserved":0,"allocated":1847296,"data_stored":896064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542181,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28848,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342992,"statfs":{"total":96636764160,"available":96607223808,"internally_reserved":0,"allocated":1863680,"data_stored":902041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542181,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46948,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324892,"statfs":{"total":96636764160,"available":96588689408,"internally_reserved":0,"allocated":1908736,"data_stored":1486451,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28848,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342992,"statfs":{"total":96636764160,"available":96607223808,"internally_reserved":0,"allocated":1859584,"data_stored":902022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55212,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316628,"statfs":{"total":96636764160,"available":96580227072,"internally_reserved":0,"allocated":1851392,"data_stored":896083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51052,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320788,"statfs":{"total":96636764160,"available":96584486912,"internally_reserved":0,"allocated":1859584,"data_stored":902022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11879,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:43:53.489 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T22:43:53.827 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:43:53.828 INFO:teuthology.orchestra.run.smithi115.stdout:{"fsid":"551c5ed8-e790-485a-bc95-a9578a29275c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":183,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":156,"num_osds":8,"num_up_osds":8,"osd_up_since":1726613011,"num_in_osds":8,"osd_in_since":1726612852,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":40},{"state_name":"peering","count":1}],"num_pgs":41,"num_pools":3,"num_objects":500,"data_bytes":590406,"bytes_used":323440640,"bytes_avail":772770672640,"bytes_total":773094113280,"inactive_pgs_ratio":0.024390242993831635},"fsmap":{"epoch":1,"btime":"2024-09-17T22:40:50:181442+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-17T22:42:04.316075+0000","services":{}},"progress_events":{}} 2024-09-17T22:43:53.840 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T22:43:53.840 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:43:54.115 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:43:54.116 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:43:54.127 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2024-09-17T22:43:52.373922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":500,"num_object_clones":0,"num_object_copies":1000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":500,"num_whiteouts":492,"num_read":268221,"num_read_kb":802336,"num_write":18302,"num_write_kb":296121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":30918130,"num_keys_recovered":86,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16753,"ondisk_log_size":16753,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":315860,"kb_used_data":14608,"kb_used_omap":32,"kb_used_meta":301023,"kb_avail":754658860,"statfs":{"total":773094113280,"available":772770672640,"internally_reserved":0,"allocated":14958592,"data_stored":8373156,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":33450,"internal_metadata":308247894},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-114,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035823"},"pg_stats":[{"pgid":"3.1f","version":"131'860","reported_seq":7310,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749216+0000","last_change":"2024-09-17T22:43:19.191421+0000","last_active":"2024-09-17T22:43:51.749216+0000","last_peered":"2024-09-17T22:43:51.749216+0000","last_clean":"2024-09-17T22:43:51.749216+0000","last_became_active":"2024-09-17T22:41:19.443658+0000","last_became_peered":"2024-09-17T22:41:19.443658+0000","last_unstale":"2024-09-17T22:43:51.749216+0000","last_undegraded":"2024-09-17T22:43:51.749216+0000","last_fullsized":"2024-09-17T22:43:51.749216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:58.423329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001073428,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8816,"num_read_kb":14906,"num_write":1102,"num_write_kb":4548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"130'786","reported_seq":7422,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.748986+0000","last_change":"2024-09-17T22:43:19.190807+0000","last_active":"2024-09-17T22:43:51.748986+0000","last_peered":"2024-09-17T22:43:51.748986+0000","last_clean":"2024-09-17T22:43:51.748986+0000","last_became_active":"2024-09-17T22:41:19.449357+0000","last_became_peered":"2024-09-17T22:41:19.449357+0000","last_unstale":"2024-09-17T22:43:51.748986+0000","last_undegraded":"2024-09-17T22:43:51.748986+0000","last_fullsized":"2024-09-17T22:43:51.748986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:20.300342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051110000000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":8865,"num_read_kb":26537,"num_write":1126,"num_write_kb":10958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"131'435","reported_seq":2380,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357945+0000","last_change":"2024-09-17T22:43:19.144579+0000","last_active":"2024-09-17T22:43:51.357945+0000","last_peered":"2024-09-17T22:43:51.357945+0000","last_clean":"2024-09-17T22:43:51.357945+0000","last_became_active":"2024-09-17T22:42:58.002822+0000","last_became_peered":"2024-09-17T22:42:58.002822+0000","last_unstale":"2024-09-17T22:43:51.357945+0000","last_undegraded":"2024-09-17T22:43:51.357945+0000","last_fullsized":"2024-09-17T22:43:51.357945+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:18:33.730453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022657670000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":17641,"num_read_kb":48444,"num_write":377,"num_write_kb":8987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":7308630,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"131'494","reported_seq":2826,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357881+0000","last_change":"2024-09-17T22:43:19.146655+0000","last_active":"2024-09-17T22:43:51.357881+0000","last_peered":"2024-09-17T22:43:51.357881+0000","last_clean":"2024-09-17T22:43:51.357881+0000","last_became_active":"2024-09-17T22:42:34.310411+0000","last_became_peered":"2024-09-17T22:42:34.310411+0000","last_unstale":"2024-09-17T22:43:51.357881+0000","last_undegraded":"2024-09-17T22:43:51.357881+0000","last_fullsized":"2024-09-17T22:43:51.357881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:34.577953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039390110000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":2615,"num_read_kb":14215,"num_write":599,"num_write_kb":12172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":7352772,"num_keys_recovered":74,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"139'244","reported_seq":8116,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352818+0000","last_change":"2024-09-17T22:43:35.308387+0000","last_active":"2024-09-17T22:43:51.352818+0000","last_peered":"2024-09-17T22:43:51.352818+0000","last_clean":"2024-09-17T22:43:51.352818+0000","last_became_active":"2024-09-17T22:43:35.215952+0000","last_became_peered":"2024-09-17T22:43:35.215952+0000","last_unstale":"2024-09-17T22:43:51.352818+0000","last_undegraded":"2024-09-17T22:43:51.352818+0000","last_fullsized":"2024-09-17T22:43:51.352818+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:45.376055+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.091482261999999995,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7315,"num_read_kb":22822,"num_write":186,"num_write_kb":6215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":3640098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"131'360","reported_seq":5241,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357959+0000","last_change":"2024-09-17T22:43:19.145987+0000","last_active":"2024-09-17T22:43:51.357959+0000","last_peered":"2024-09-17T22:43:51.357959+0000","last_clean":"2024-09-17T22:43:51.357959+0000","last_became_active":"2024-09-17T22:41:19.446541+0000","last_became_peered":"2024-09-17T22:41:19.446541+0000","last_unstale":"2024-09-17T22:43:51.357959+0000","last_undegraded":"2024-09-17T22:43:51.357959+0000","last_fullsized":"2024-09-17T22:43:51.357959+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:12:51.399521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035192800000000001,"stat_sum":{"num_bytes":19,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":5091,"num_read_kb":19688,"num_write":329,"num_write_kb":4179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"130'391","reported_seq":6764,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352819+0000","last_change":"2024-09-17T22:43:19.150923+0000","last_active":"2024-09-17T22:43:51.352819+0000","last_peered":"2024-09-17T22:43:51.352819+0000","last_clean":"2024-09-17T22:43:51.352819+0000","last_became_active":"2024-09-17T22:41:19.449940+0000","last_became_peered":"2024-09-17T22:41:19.449940+0000","last_unstale":"2024-09-17T22:43:51.352819+0000","last_undegraded":"2024-09-17T22:43:51.352819+0000","last_fullsized":"2024-09-17T22:43:51.352819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:57:32.786125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002665995,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6668,"num_read_kb":22143,"num_write":345,"num_write_kb":9035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"129'383","reported_seq":3221,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749001+0000","last_change":"2024-09-17T22:43:19.191965+0000","last_active":"2024-09-17T22:43:51.749001+0000","last_peered":"2024-09-17T22:43:51.749001+0000","last_clean":"2024-09-17T22:43:51.749001+0000","last_became_active":"2024-09-17T22:41:19.446287+0000","last_became_peered":"2024-09-17T22:41:19.446287+0000","last_unstale":"2024-09-17T22:43:51.749001+0000","last_undegraded":"2024-09-17T22:43:51.749001+0000","last_fullsized":"2024-09-17T22:43:51.749001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:23:52.909972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001462812,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3441,"num_read_kb":20083,"num_write":323,"num_write_kb":10930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"131'513","reported_seq":2752,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749198+0000","last_change":"2024-09-17T22:43:19.191786+0000","last_active":"2024-09-17T22:43:51.749198+0000","last_peered":"2024-09-17T22:43:51.749198+0000","last_clean":"2024-09-17T22:43:51.749198+0000","last_became_active":"2024-09-17T22:41:19.445923+0000","last_became_peered":"2024-09-17T22:41:19.445923+0000","last_unstale":"2024-09-17T22:43:51.749198+0000","last_undegraded":"2024-09-17T22:43:51.749198+0000","last_fullsized":"2024-09-17T22:43:51.749198+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:02:33.097995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001414085,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3445,"num_read_kb":15105,"num_write":513,"num_write_kb":8714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"129'562","reported_seq":22388,"reported_epoch":155,"state":"peering","last_fresh":"2024-09-17T22:43:51.355701+0000","last_change":"2024-09-17T22:43:51.355701+0000","last_active":"2024-09-17T22:43:31.187738+0000","last_peered":"2024-09-17T22:43:30.548859+0000","last_clean":"2024-09-17T22:43:30.548859+0000","last_became_active":"2024-09-17T22:41:57.132018+0000","last_became_peered":"2024-09-17T22:41:57.132018+0000","last_unstale":"2024-09-17T22:43:51.355701+0000","last_undegraded":"2024-09-17T22:43:51.355701+0000","last_fullsized":"2024-09-17T22:43:51.355701+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016446880000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":22869,"num_read_kb":64042,"num_write":605,"num_write_kb":12652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"139'437","reported_seq":7649,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T22:43:50.359256+0000","last_change":"2024-09-17T22:43:35.287778+0000","last_active":"2024-09-17T22:43:50.359256+0000","last_peered":"2024-09-17T22:43:50.359256+0000","last_clean":"2024-09-17T22:43:50.359256+0000","last_became_active":"2024-09-17T22:43:35.217210+0000","last_became_peered":"2024-09-17T22:43:35.217210+0000","last_unstale":"2024-09-17T22:43:50.359256+0000","last_undegraded":"2024-09-17T22:43:50.359256+0000","last_fullsized":"2024-09-17T22:43:50.359256+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:38:24.310736+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.069534181,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8763,"num_read_kb":20352,"num_write":343,"num_write_kb":4822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2823905,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"129'589","reported_seq":7855,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352780+0000","last_change":"2024-09-17T22:43:19.143328+0000","last_active":"2024-09-17T22:43:51.352780+0000","last_peered":"2024-09-17T22:43:51.352780+0000","last_clean":"2024-09-17T22:43:51.352780+0000","last_became_active":"2024-09-17T22:41:19.451631+0000","last_became_peered":"2024-09-17T22:41:19.451631+0000","last_unstale":"2024-09-17T22:43:51.352780+0000","last_undegraded":"2024-09-17T22:43:51.352780+0000","last_fullsized":"2024-09-17T22:43:51.352780+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:31.809865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090157099999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8321,"num_read_kb":19442,"num_write":736,"num_write_kb":19536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":293,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352731+0000","last_change":"2024-09-17T22:43:32.195958+0000","last_active":"2024-09-17T22:43:51.352731+0000","last_peered":"2024-09-17T22:43:51.352731+0000","last_clean":"2024-09-17T22:43:51.352731+0000","last_became_active":"2024-09-17T22:43:32.195676+0000","last_became_peered":"2024-09-17T22:43:32.195676+0000","last_unstale":"2024-09-17T22:43:51.352731+0000","last_undegraded":"2024-09-17T22:43:51.352731+0000","last_fullsized":"2024-09-17T22:43:51.352731+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:41:52.669174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"131'579","reported_seq":5338,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352815+0000","last_change":"2024-09-17T22:43:19.150601+0000","last_active":"2024-09-17T22:43:51.352815+0000","last_peered":"2024-09-17T22:43:51.352815+0000","last_clean":"2024-09-17T22:43:51.352815+0000","last_became_active":"2024-09-17T22:41:19.448966+0000","last_became_peered":"2024-09-17T22:41:19.448966+0000","last_unstale":"2024-09-17T22:43:51.352815+0000","last_undegraded":"2024-09-17T22:43:51.352815+0000","last_fullsized":"2024-09-17T22:43:51.352815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:20:32.636962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026117269999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6258,"num_read_kb":29920,"num_write":539,"num_write_kb":6243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":292,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352861+0000","last_change":"2024-09-17T22:43:32.195712+0000","last_active":"2024-09-17T22:43:51.352861+0000","last_peered":"2024-09-17T22:43:51.352861+0000","last_clean":"2024-09-17T22:43:51.352861+0000","last_became_active":"2024-09-17T22:43:32.195451+0000","last_became_peered":"2024-09-17T22:43:32.195451+0000","last_unstale":"2024-09-17T22:43:51.352861+0000","last_undegraded":"2024-09-17T22:43:51.352861+0000","last_fullsized":"2024-09-17T22:43:51.352861+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:52.062601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"131'396","reported_seq":1773,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352783+0000","last_change":"2024-09-17T22:43:19.149604+0000","last_active":"2024-09-17T22:43:51.352783+0000","last_peered":"2024-09-17T22:43:51.352783+0000","last_clean":"2024-09-17T22:43:51.352783+0000","last_became_active":"2024-09-17T22:41:19.451637+0000","last_became_peered":"2024-09-17T22:41:19.451637+0000","last_unstale":"2024-09-17T22:43:51.352783+0000","last_undegraded":"2024-09-17T22:43:51.352783+0000","last_fullsized":"2024-09-17T22:43:51.352783+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:16.254887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001593059,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1474,"num_read_kb":9975,"num_write":408,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":291,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352906+0000","last_change":"2024-09-17T22:43:15.154296+0000","last_active":"2024-09-17T22:43:51.352906+0000","last_peered":"2024-09-17T22:43:51.352906+0000","last_clean":"2024-09-17T22:43:51.352906+0000","last_became_active":"2024-09-17T22:43:15.153956+0000","last_became_peered":"2024-09-17T22:43:15.153956+0000","last_unstale":"2024-09-17T22:43:51.352906+0000","last_undegraded":"2024-09-17T22:43:51.352906+0000","last_fullsized":"2024-09-17T22:43:51.352906+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:19.340692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"131'478","reported_seq":1431,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357944+0000","last_change":"2024-09-17T22:43:19.151215+0000","last_active":"2024-09-17T22:43:51.357944+0000","last_peered":"2024-09-17T22:43:51.357944+0000","last_clean":"2024-09-17T22:43:51.357944+0000","last_became_active":"2024-09-17T22:41:19.444893+0000","last_became_peered":"2024-09-17T22:41:19.444893+0000","last_unstale":"2024-09-17T22:43:51.357944+0000","last_undegraded":"2024-09-17T22:43:51.357944+0000","last_fullsized":"2024-09-17T22:43:51.357944+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:34:31.498807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001189308,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1718,"num_read_kb":9540,"num_write":633,"num_write_kb":7900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":286,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749190+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:43:51.749190+0000","last_peered":"2024-09-17T22:43:51.749190+0000","last_clean":"2024-09-17T22:43:51.749190+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:43:51.749190+0000","last_undegraded":"2024-09-17T22:43:51.749190+0000","last_fullsized":"2024-09-17T22:43:51.749190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"130'393","reported_seq":15617,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352661+0000","last_change":"2024-09-17T22:43:19.187091+0000","last_active":"2024-09-17T22:43:51.352661+0000","last_peered":"2024-09-17T22:43:51.352661+0000","last_clean":"2024-09-17T22:43:51.352661+0000","last_became_active":"2024-09-17T22:41:19.451723+0000","last_became_peered":"2024-09-17T22:41:19.451723+0000","last_unstale":"2024-09-17T22:43:51.352661+0000","last_undegraded":"2024-09-17T22:43:51.352661+0000","last_fullsized":"2024-09-17T22:43:51.352661+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:50:14.037236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061483000000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":15368,"num_read_kb":33652,"num_write":428,"num_write_kb":8748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"139'879","reported_seq":12150,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-17T22:43:48.852590+0000","last_change":"2024-09-17T22:43:35.334520+0000","last_active":"2024-09-17T22:43:48.852590+0000","last_peered":"2024-09-17T22:43:48.852590+0000","last_clean":"2024-09-17T22:43:48.852590+0000","last_became_active":"2024-09-17T22:43:35.220905+0000","last_became_peered":"2024-09-17T22:43:35.220905+0000","last_unstale":"2024-09-17T22:43:48.852590+0000","last_undegraded":"2024-09-17T22:43:48.852590+0000","last_fullsized":"2024-09-17T22:43:48.852590+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:11:27.480976+0000","scrub_duration":0,"objects_trimmed":42,"snaptrim_duration":0.112643386,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":15325,"num_read_kb":33693,"num_write":856,"num_write_kb":8706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":4229520,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":202,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352969+0000","last_change":"2024-09-17T22:43:42.285641+0000","last_active":"2024-09-17T22:43:51.352969+0000","last_peered":"2024-09-17T22:43:51.352969+0000","last_clean":"2024-09-17T22:43:51.352969+0000","last_became_active":"2024-09-17T22:43:42.285307+0000","last_became_peered":"2024-09-17T22:43:42.285307+0000","last_unstale":"2024-09-17T22:43:51.352969+0000","last_undegraded":"2024-09-17T22:43:51.352969+0000","last_fullsized":"2024-09-17T22:43:51.352969+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:39.203508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"130'385","reported_seq":4190,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749152+0000","last_change":"2024-09-17T22:43:19.191554+0000","last_active":"2024-09-17T22:43:51.749152+0000","last_peered":"2024-09-17T22:43:51.749152+0000","last_clean":"2024-09-17T22:43:51.749152+0000","last_became_active":"2024-09-17T22:41:19.444314+0000","last_became_peered":"2024-09-17T22:41:19.444314+0000","last_unstale":"2024-09-17T22:43:51.749152+0000","last_undegraded":"2024-09-17T22:43:51.749152+0000","last_fullsized":"2024-09-17T22:43:51.749152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:07:00.374588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011857370000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4006,"num_read_kb":22354,"num_write":358,"num_write_kb":9353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":286,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.358312+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:43:51.358312+0000","last_peered":"2024-09-17T22:43:51.358312+0000","last_clean":"2024-09-17T22:43:51.358312+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:43:51.358312+0000","last_undegraded":"2024-09-17T22:43:51.358312+0000","last_fullsized":"2024-09-17T22:43:51.358312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"130'481","reported_seq":7455,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.358137+0000","last_change":"2024-09-17T22:43:19.150805+0000","last_active":"2024-09-17T22:43:51.358137+0000","last_peered":"2024-09-17T22:43:51.358137+0000","last_clean":"2024-09-17T22:43:51.358137+0000","last_became_active":"2024-09-17T22:41:19.449791+0000","last_became_peered":"2024-09-17T22:41:19.449791+0000","last_unstale":"2024-09-17T22:43:51.358137+0000","last_undegraded":"2024-09-17T22:43:51.358137+0000","last_fullsized":"2024-09-17T22:43:51.358137+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:48.110050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088469700000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8092,"num_read_kb":9247,"num_write":519,"num_write_kb":4605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":286,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352571+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:43:51.352571+0000","last_peered":"2024-09-17T22:43:51.352571+0000","last_clean":"2024-09-17T22:43:51.352571+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:43:51.352571+0000","last_undegraded":"2024-09-17T22:43:51.352571+0000","last_fullsized":"2024-09-17T22:43:51.352571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"131'756","reported_seq":8739,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352735+0000","last_change":"2024-09-17T22:43:19.148879+0000","last_active":"2024-09-17T22:43:51.352735+0000","last_peered":"2024-09-17T22:43:51.352735+0000","last_clean":"2024-09-17T22:43:51.352735+0000","last_became_active":"2024-09-17T22:41:19.449279+0000","last_became_peered":"2024-09-17T22:41:19.449279+0000","last_unstale":"2024-09-17T22:43:51.352735+0000","last_undegraded":"2024-09-17T22:43:51.352735+0000","last_fullsized":"2024-09-17T22:43:51.352735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:51:59.677897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089414599999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":11291,"num_read_kb":22947,"num_write":956,"num_write_kb":2860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":201,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.353170+0000","last_change":"2024-09-17T22:43:42.286959+0000","last_active":"2024-09-17T22:43:51.353170+0000","last_peered":"2024-09-17T22:43:51.353170+0000","last_clean":"2024-09-17T22:43:51.353170+0000","last_became_active":"2024-09-17T22:43:42.286652+0000","last_became_peered":"2024-09-17T22:43:42.286652+0000","last_unstale":"2024-09-17T22:43:51.353170+0000","last_undegraded":"2024-09-17T22:43:51.353170+0000","last_fullsized":"2024-09-17T22:43:51.353170+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:50:39.824946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":438,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357877+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:43:51.357877+0000","last_peered":"2024-09-17T22:43:51.357877+0000","last_clean":"2024-09-17T22:43:51.357877+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:43:51.357877+0000","last_undegraded":"2024-09-17T22:43:51.357877+0000","last_fullsized":"2024-09-17T22:43:51.357877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"129'613","reported_seq":11950,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352720+0000","last_change":"2024-09-17T22:43:19.149769+0000","last_active":"2024-09-17T22:43:51.352720+0000","last_peered":"2024-09-17T22:43:51.352720+0000","last_clean":"2024-09-17T22:43:51.352720+0000","last_became_active":"2024-09-17T22:41:19.448956+0000","last_became_peered":"2024-09-17T22:41:19.448956+0000","last_unstale":"2024-09-17T22:43:51.352720+0000","last_undegraded":"2024-09-17T22:43:51.352720+0000","last_fullsized":"2024-09-17T22:43:51.352720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:28:45.951042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017463789999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13252,"num_read_kb":46911,"num_write":655,"num_write_kb":25175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"129'785","reported_seq":14759,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352803+0000","last_change":"2024-09-17T22:43:19.145329+0000","last_active":"2024-09-17T22:43:51.352803+0000","last_peered":"2024-09-17T22:43:51.352803+0000","last_clean":"2024-09-17T22:43:51.352803+0000","last_became_active":"2024-09-17T22:41:19.449003+0000","last_became_peered":"2024-09-17T22:41:19.449003+0000","last_unstale":"2024-09-17T22:43:51.352803+0000","last_undegraded":"2024-09-17T22:43:51.352803+0000","last_fullsized":"2024-09-17T22:43:51.352803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:13:00.810670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910964,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":16223,"num_read_kb":43243,"num_write":1040,"num_write_kb":9843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"130'470","reported_seq":8484,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352888+0000","last_change":"2024-09-17T22:43:42.285286+0000","last_active":"2024-09-17T22:43:51.352888+0000","last_peered":"2024-09-17T22:43:51.352888+0000","last_clean":"2024-09-17T22:43:51.352888+0000","last_became_active":"2024-09-17T22:43:42.284094+0000","last_became_peered":"2024-09-17T22:43:42.284094+0000","last_unstale":"2024-09-17T22:43:51.352888+0000","last_undegraded":"2024-09-17T22:43:51.352888+0000","last_fullsized":"2024-09-17T22:43:51.352888+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:36:47.430783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001830772,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":8191,"num_read_kb":41081,"num_write":473,"num_write_kb":9769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"131'423","reported_seq":2735,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.358360+0000","last_change":"2024-09-17T22:43:19.142462+0000","last_active":"2024-09-17T22:43:51.358360+0000","last_peered":"2024-09-17T22:43:51.358360+0000","last_clean":"2024-09-17T22:43:51.358360+0000","last_became_active":"2024-09-17T22:41:19.447215+0000","last_became_peered":"2024-09-17T22:41:19.447215+0000","last_unstale":"2024-09-17T22:43:51.358360+0000","last_undegraded":"2024-09-17T22:43:51.358360+0000","last_fullsized":"2024-09-17T22:43:51.358360+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:37:48.721121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066798900000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3409,"num_read_kb":14480,"num_write":404,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"139'353","reported_seq":21720,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-17T22:43:50.359231+0000","last_change":"2024-09-17T22:43:35.288547+0000","last_active":"2024-09-17T22:43:50.359231+0000","last_peered":"2024-09-17T22:43:50.359231+0000","last_clean":"2024-09-17T22:43:50.359231+0000","last_became_active":"2024-09-17T22:43:35.216950+0000","last_became_peered":"2024-09-17T22:43:35.216950+0000","last_unstale":"2024-09-17T22:43:50.359231+0000","last_undegraded":"2024-09-17T22:43:50.359231+0000","last_fullsized":"2024-09-17T22:43:50.359231+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:01:37.332938+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.070463135999999996,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":21473,"num_read_kb":63694,"num_write":340,"num_write_kb":14940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":1192963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"129'486","reported_seq":2029,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352846+0000","last_change":"2024-09-17T22:43:42.283323+0000","last_active":"2024-09-17T22:43:51.352846+0000","last_peered":"2024-09-17T22:43:51.352846+0000","last_clean":"2024-09-17T22:43:51.352846+0000","last_became_active":"2024-09-17T22:43:42.282252+0000","last_became_peered":"2024-09-17T22:43:42.282252+0000","last_unstale":"2024-09-17T22:43:51.352846+0000","last_undegraded":"2024-09-17T22:43:51.352846+0000","last_fullsized":"2024-09-17T22:43:51.352846+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:31.315273+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.002173033,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2216,"num_read_kb":10204,"num_write":574,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"139'372","reported_seq":8837,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352997+0000","last_change":"2024-09-17T22:43:35.323373+0000","last_active":"2024-09-17T22:43:51.352997+0000","last_peered":"2024-09-17T22:43:51.352997+0000","last_clean":"2024-09-17T22:43:51.352997+0000","last_became_active":"2024-09-17T22:43:35.224141+0000","last_became_peered":"2024-09-17T22:43:35.224141+0000","last_unstale":"2024-09-17T22:43:51.352997+0000","last_undegraded":"2024-09-17T22:43:51.352997+0000","last_fullsized":"2024-09-17T22:43:51.352997+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:01:02.683062+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.098411079999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8804,"num_read_kb":23106,"num_write":291,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":2514411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"139'353","reported_seq":1880,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-17T22:43:46.318517+0000","last_change":"2024-09-17T22:43:35.320798+0000","last_active":"2024-09-17T22:43:46.318517+0000","last_peered":"2024-09-17T22:43:46.318517+0000","last_clean":"2024-09-17T22:43:46.318517+0000","last_became_active":"2024-09-17T22:43:35.217622+0000","last_became_peered":"2024-09-17T22:43:35.217622+0000","last_unstale":"2024-09-17T22:43:46.318517+0000","last_undegraded":"2024-09-17T22:43:46.318517+0000","last_fullsized":"2024-09-17T22:43:46.318517+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:52:51.375191+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.102357168,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1634,"num_read_kb":5501,"num_write":380,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1563844,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"129'349","reported_seq":2170,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357762+0000","last_change":"2024-09-17T22:43:19.144272+0000","last_active":"2024-09-17T22:43:51.357762+0000","last_peered":"2024-09-17T22:43:51.357762+0000","last_clean":"2024-09-17T22:43:51.357762+0000","last_became_active":"2024-09-17T22:41:19.447419+0000","last_became_peered":"2024-09-17T22:41:19.447419+0000","last_unstale":"2024-09-17T22:43:51.357762+0000","last_undegraded":"2024-09-17T22:43:51.357762+0000","last_fullsized":"2024-09-17T22:43:51.357762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:48.842752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00132408,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3110,"num_read_kb":8489,"num_write":286,"num_write_kb":3161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"129'616","reported_seq":3679,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352697+0000","last_change":"2024-09-17T22:43:19.144657+0000","last_active":"2024-09-17T22:43:51.352697+0000","last_peered":"2024-09-17T22:43:51.352697+0000","last_clean":"2024-09-17T22:43:51.352697+0000","last_became_active":"2024-09-17T22:41:19.454151+0000","last_became_peered":"2024-09-17T22:41:19.454151+0000","last_unstale":"2024-09-17T22:43:51.352697+0000","last_undegraded":"2024-09-17T22:43:51.352697+0000","last_fullsized":"2024-09-17T22:43:51.352697+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:39:51.641693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001718979,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5266,"num_read_kb":13546,"num_write":658,"num_write_kb":3731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"130'910","reported_seq":11717,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357917+0000","last_change":"2024-09-17T22:43:35.218645+0000","last_active":"2024-09-17T22:43:51.357917+0000","last_peered":"2024-09-17T22:43:51.357917+0000","last_clean":"2024-09-17T22:43:51.357917+0000","last_became_active":"2024-09-17T22:43:35.217045+0000","last_became_peered":"2024-09-17T22:43:35.217045+0000","last_unstale":"2024-09-17T22:43:51.357917+0000","last_undegraded":"2024-09-17T22:43:51.357917+0000","last_fullsized":"2024-09-17T22:43:51.357917+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:20.169030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001497735,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14642,"num_read_kb":44074,"num_write":1142,"num_write_kb":22161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"139'502","reported_seq":2628,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.353075+0000","last_change":"2024-09-17T22:43:35.301824+0000","last_active":"2024-09-17T22:43:51.353075+0000","last_peered":"2024-09-17T22:43:51.353075+0000","last_clean":"2024-09-17T22:43:51.353075+0000","last_became_active":"2024-09-17T22:43:35.217551+0000","last_became_peered":"2024-09-17T22:43:35.217551+0000","last_unstale":"2024-09-17T22:43:51.353075+0000","last_undegraded":"2024-09-17T22:43:51.353075+0000","last_fullsized":"2024-09-17T22:43:51.353075+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:10.204124+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.082900606000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":2503,"num_read_kb":8800,"num_write":617,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":291987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":492,"num_read":268105,"num_read_kb":802236,"num_write":18141,"num_write_kb":293703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":30918130,"num_keys_recovered":86,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11879,"internal_metadata":0},"log_size":16633,"ondisk_log_size":16633,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542183,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28908,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342932,"statfs":{"total":96636764160,"available":96607162368,"internally_reserved":0,"allocated":1921024,"data_stored":1492409,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":135,"seq":579820584966,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34724,"kb_used_data":1804,"kb_used_omap":21,"kb_used_meta":32874,"kb_avail":94337116,"statfs":{"total":96636764160,"available":96601206784,"internally_reserved":0,"allocated":1847296,"data_stored":896064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22295,"internal_metadata":33663209},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542181,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41320,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330520,"statfs":{"total":96636764160,"available":96594452480,"internally_reserved":0,"allocated":1847296,"data_stored":896064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542181,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28848,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342992,"statfs":{"total":96636764160,"available":96607223808,"internally_reserved":0,"allocated":1863680,"data_stored":902041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542181,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46948,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324892,"statfs":{"total":96636764160,"available":96588689408,"internally_reserved":0,"allocated":1908736,"data_stored":1486451,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28848,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342992,"statfs":{"total":96636764160,"available":96607223808,"internally_reserved":0,"allocated":1859584,"data_stored":902022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55212,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316628,"statfs":{"total":96636764160,"available":96580227072,"internally_reserved":0,"allocated":1851392,"data_stored":896083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51052,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320788,"statfs":{"total":96636764160,"available":96584486912,"internally_reserved":0,"allocated":1859584,"data_stored":902022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11879,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:43:57.130 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:43:57.422 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:43:57.423 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:43:57.434 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-17T22:43:56.339636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":500,"num_object_clones":0,"num_object_copies":1000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":500,"num_whiteouts":492,"num_read":268221,"num_read_kb":802336,"num_write":18302,"num_write_kb":296121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":30918130,"num_keys_recovered":86,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16753,"ondisk_log_size":16753,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":315920,"kb_used_data":14668,"kb_used_omap":32,"kb_used_meta":301023,"kb_avail":754658800,"statfs":{"total":773094113280,"available":772770611200,"internally_reserved":0,"allocated":15020032,"data_stored":8402946,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":33450,"internal_metadata":308247894},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-114,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001178"},"pg_stats":[{"pgid":"3.1f","version":"131'860","reported_seq":7310,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749216+0000","last_change":"2024-09-17T22:43:19.191421+0000","last_active":"2024-09-17T22:43:51.749216+0000","last_peered":"2024-09-17T22:43:51.749216+0000","last_clean":"2024-09-17T22:43:51.749216+0000","last_became_active":"2024-09-17T22:41:19.443658+0000","last_became_peered":"2024-09-17T22:41:19.443658+0000","last_unstale":"2024-09-17T22:43:51.749216+0000","last_undegraded":"2024-09-17T22:43:51.749216+0000","last_fullsized":"2024-09-17T22:43:51.749216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:58.423329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001073428,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8816,"num_read_kb":14906,"num_write":1102,"num_write_kb":4548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"130'786","reported_seq":7422,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.748986+0000","last_change":"2024-09-17T22:43:19.190807+0000","last_active":"2024-09-17T22:43:51.748986+0000","last_peered":"2024-09-17T22:43:51.748986+0000","last_clean":"2024-09-17T22:43:51.748986+0000","last_became_active":"2024-09-17T22:41:19.449357+0000","last_became_peered":"2024-09-17T22:41:19.449357+0000","last_unstale":"2024-09-17T22:43:51.748986+0000","last_undegraded":"2024-09-17T22:43:51.748986+0000","last_fullsized":"2024-09-17T22:43:51.748986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:20.300342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051110000000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":8865,"num_read_kb":26537,"num_write":1126,"num_write_kb":10958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"131'435","reported_seq":2380,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357945+0000","last_change":"2024-09-17T22:43:19.144579+0000","last_active":"2024-09-17T22:43:51.357945+0000","last_peered":"2024-09-17T22:43:51.357945+0000","last_clean":"2024-09-17T22:43:51.357945+0000","last_became_active":"2024-09-17T22:42:58.002822+0000","last_became_peered":"2024-09-17T22:42:58.002822+0000","last_unstale":"2024-09-17T22:43:51.357945+0000","last_undegraded":"2024-09-17T22:43:51.357945+0000","last_fullsized":"2024-09-17T22:43:51.357945+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:18:33.730453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022657670000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":17641,"num_read_kb":48444,"num_write":377,"num_write_kb":8987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":7308630,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"131'494","reported_seq":2826,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357881+0000","last_change":"2024-09-17T22:43:19.146655+0000","last_active":"2024-09-17T22:43:51.357881+0000","last_peered":"2024-09-17T22:43:51.357881+0000","last_clean":"2024-09-17T22:43:51.357881+0000","last_became_active":"2024-09-17T22:42:34.310411+0000","last_became_peered":"2024-09-17T22:42:34.310411+0000","last_unstale":"2024-09-17T22:43:51.357881+0000","last_undegraded":"2024-09-17T22:43:51.357881+0000","last_fullsized":"2024-09-17T22:43:51.357881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:34.577953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039390110000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":2615,"num_read_kb":14215,"num_write":599,"num_write_kb":12172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":7352772,"num_keys_recovered":74,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"139'244","reported_seq":8116,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352818+0000","last_change":"2024-09-17T22:43:35.308387+0000","last_active":"2024-09-17T22:43:51.352818+0000","last_peered":"2024-09-17T22:43:51.352818+0000","last_clean":"2024-09-17T22:43:51.352818+0000","last_became_active":"2024-09-17T22:43:35.215952+0000","last_became_peered":"2024-09-17T22:43:35.215952+0000","last_unstale":"2024-09-17T22:43:51.352818+0000","last_undegraded":"2024-09-17T22:43:51.352818+0000","last_fullsized":"2024-09-17T22:43:51.352818+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:45.376055+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.091482261999999995,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7315,"num_read_kb":22822,"num_write":186,"num_write_kb":6215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":3640098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"131'360","reported_seq":5241,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357959+0000","last_change":"2024-09-17T22:43:19.145987+0000","last_active":"2024-09-17T22:43:51.357959+0000","last_peered":"2024-09-17T22:43:51.357959+0000","last_clean":"2024-09-17T22:43:51.357959+0000","last_became_active":"2024-09-17T22:41:19.446541+0000","last_became_peered":"2024-09-17T22:41:19.446541+0000","last_unstale":"2024-09-17T22:43:51.357959+0000","last_undegraded":"2024-09-17T22:43:51.357959+0000","last_fullsized":"2024-09-17T22:43:51.357959+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:12:51.399521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035192800000000001,"stat_sum":{"num_bytes":19,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":5091,"num_read_kb":19688,"num_write":329,"num_write_kb":4179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"130'391","reported_seq":6764,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352819+0000","last_change":"2024-09-17T22:43:19.150923+0000","last_active":"2024-09-17T22:43:51.352819+0000","last_peered":"2024-09-17T22:43:51.352819+0000","last_clean":"2024-09-17T22:43:51.352819+0000","last_became_active":"2024-09-17T22:41:19.449940+0000","last_became_peered":"2024-09-17T22:41:19.449940+0000","last_unstale":"2024-09-17T22:43:51.352819+0000","last_undegraded":"2024-09-17T22:43:51.352819+0000","last_fullsized":"2024-09-17T22:43:51.352819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:57:32.786125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002665995,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6668,"num_read_kb":22143,"num_write":345,"num_write_kb":9035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"129'383","reported_seq":3221,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749001+0000","last_change":"2024-09-17T22:43:19.191965+0000","last_active":"2024-09-17T22:43:51.749001+0000","last_peered":"2024-09-17T22:43:51.749001+0000","last_clean":"2024-09-17T22:43:51.749001+0000","last_became_active":"2024-09-17T22:41:19.446287+0000","last_became_peered":"2024-09-17T22:41:19.446287+0000","last_unstale":"2024-09-17T22:43:51.749001+0000","last_undegraded":"2024-09-17T22:43:51.749001+0000","last_fullsized":"2024-09-17T22:43:51.749001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:23:52.909972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001462812,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3441,"num_read_kb":20083,"num_write":323,"num_write_kb":10930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"131'513","reported_seq":2752,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749198+0000","last_change":"2024-09-17T22:43:19.191786+0000","last_active":"2024-09-17T22:43:51.749198+0000","last_peered":"2024-09-17T22:43:51.749198+0000","last_clean":"2024-09-17T22:43:51.749198+0000","last_became_active":"2024-09-17T22:41:19.445923+0000","last_became_peered":"2024-09-17T22:41:19.445923+0000","last_unstale":"2024-09-17T22:43:51.749198+0000","last_undegraded":"2024-09-17T22:43:51.749198+0000","last_fullsized":"2024-09-17T22:43:51.749198+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:02:33.097995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001414085,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3445,"num_read_kb":15105,"num_write":513,"num_write_kb":8714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"129'562","reported_seq":22396,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.386833+0000","last_change":"2024-09-17T22:43:52.386833+0000","last_active":"2024-09-17T22:43:52.386833+0000","last_peered":"2024-09-17T22:43:52.386833+0000","last_clean":"2024-09-17T22:43:52.386833+0000","last_became_active":"2024-09-17T22:43:52.384987+0000","last_became_peered":"2024-09-17T22:43:52.384987+0000","last_unstale":"2024-09-17T22:43:52.386833+0000","last_undegraded":"2024-09-17T22:43:52.386833+0000","last_fullsized":"2024-09-17T22:43:52.386833+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:08:09.285837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016446880000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":22869,"num_read_kb":64042,"num_write":605,"num_write_kb":12652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"139'437","reported_seq":7603,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381556+0000","last_change":"2024-09-17T22:43:52.381556+0000","last_active":"2024-09-17T22:43:52.381556+0000","last_peered":"2024-09-17T22:43:52.381556+0000","last_clean":"2024-09-17T22:43:52.381556+0000","last_became_active":"2024-09-17T22:43:52.380232+0000","last_became_peered":"2024-09-17T22:43:52.380232+0000","last_unstale":"2024-09-17T22:43:52.381556+0000","last_undegraded":"2024-09-17T22:43:52.381556+0000","last_fullsized":"2024-09-17T22:43:52.381556+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:08:17.722346+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.062977108000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8763,"num_read_kb":20352,"num_write":343,"num_write_kb":4822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2823905,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"129'589","reported_seq":7857,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380494+0000","last_change":"2024-09-17T22:43:19.143328+0000","last_active":"2024-09-17T22:43:52.380494+0000","last_peered":"2024-09-17T22:43:52.380494+0000","last_clean":"2024-09-17T22:43:52.380494+0000","last_became_active":"2024-09-17T22:41:19.451631+0000","last_became_peered":"2024-09-17T22:41:19.451631+0000","last_unstale":"2024-09-17T22:43:52.380494+0000","last_undegraded":"2024-09-17T22:43:52.380494+0000","last_fullsized":"2024-09-17T22:43:52.380494+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:31.809865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090157099999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8321,"num_read_kb":19442,"num_write":736,"num_write_kb":19536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":295,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380849+0000","last_change":"2024-09-17T22:43:32.195958+0000","last_active":"2024-09-17T22:43:52.380849+0000","last_peered":"2024-09-17T22:43:52.380849+0000","last_clean":"2024-09-17T22:43:52.380849+0000","last_became_active":"2024-09-17T22:43:32.195676+0000","last_became_peered":"2024-09-17T22:43:32.195676+0000","last_unstale":"2024-09-17T22:43:52.380849+0000","last_undegraded":"2024-09-17T22:43:52.380849+0000","last_fullsized":"2024-09-17T22:43:52.380849+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:41:52.669174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"131'579","reported_seq":5338,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352815+0000","last_change":"2024-09-17T22:43:19.150601+0000","last_active":"2024-09-17T22:43:51.352815+0000","last_peered":"2024-09-17T22:43:51.352815+0000","last_clean":"2024-09-17T22:43:51.352815+0000","last_became_active":"2024-09-17T22:41:19.448966+0000","last_became_peered":"2024-09-17T22:41:19.448966+0000","last_unstale":"2024-09-17T22:43:51.352815+0000","last_undegraded":"2024-09-17T22:43:51.352815+0000","last_fullsized":"2024-09-17T22:43:51.352815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:20:32.636962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026117269999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6258,"num_read_kb":29920,"num_write":539,"num_write_kb":6243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":292,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352861+0000","last_change":"2024-09-17T22:43:32.195712+0000","last_active":"2024-09-17T22:43:51.352861+0000","last_peered":"2024-09-17T22:43:51.352861+0000","last_clean":"2024-09-17T22:43:51.352861+0000","last_became_active":"2024-09-17T22:43:32.195451+0000","last_became_peered":"2024-09-17T22:43:32.195451+0000","last_unstale":"2024-09-17T22:43:51.352861+0000","last_undegraded":"2024-09-17T22:43:51.352861+0000","last_fullsized":"2024-09-17T22:43:51.352861+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:52.062601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"131'396","reported_seq":1773,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352783+0000","last_change":"2024-09-17T22:43:19.149604+0000","last_active":"2024-09-17T22:43:51.352783+0000","last_peered":"2024-09-17T22:43:51.352783+0000","last_clean":"2024-09-17T22:43:51.352783+0000","last_became_active":"2024-09-17T22:41:19.451637+0000","last_became_peered":"2024-09-17T22:41:19.451637+0000","last_unstale":"2024-09-17T22:43:51.352783+0000","last_undegraded":"2024-09-17T22:43:51.352783+0000","last_fullsized":"2024-09-17T22:43:51.352783+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:16.254887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001593059,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1474,"num_read_kb":9975,"num_write":408,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":293,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380692+0000","last_change":"2024-09-17T22:43:15.154296+0000","last_active":"2024-09-17T22:43:52.380692+0000","last_peered":"2024-09-17T22:43:52.380692+0000","last_clean":"2024-09-17T22:43:52.380692+0000","last_became_active":"2024-09-17T22:43:15.153956+0000","last_became_peered":"2024-09-17T22:43:15.153956+0000","last_unstale":"2024-09-17T22:43:52.380692+0000","last_undegraded":"2024-09-17T22:43:52.380692+0000","last_fullsized":"2024-09-17T22:43:52.380692+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:19.340692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"131'478","reported_seq":1431,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357944+0000","last_change":"2024-09-17T22:43:19.151215+0000","last_active":"2024-09-17T22:43:51.357944+0000","last_peered":"2024-09-17T22:43:51.357944+0000","last_clean":"2024-09-17T22:43:51.357944+0000","last_became_active":"2024-09-17T22:41:19.444893+0000","last_became_peered":"2024-09-17T22:41:19.444893+0000","last_unstale":"2024-09-17T22:43:51.357944+0000","last_undegraded":"2024-09-17T22:43:51.357944+0000","last_fullsized":"2024-09-17T22:43:51.357944+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:34:31.498807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001189308,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1718,"num_read_kb":9540,"num_write":633,"num_write_kb":7900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":286,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749190+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:43:51.749190+0000","last_peered":"2024-09-17T22:43:51.749190+0000","last_clean":"2024-09-17T22:43:51.749190+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:43:51.749190+0000","last_undegraded":"2024-09-17T22:43:51.749190+0000","last_fullsized":"2024-09-17T22:43:51.749190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"130'393","reported_seq":15619,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.379789+0000","last_change":"2024-09-17T22:43:19.187091+0000","last_active":"2024-09-17T22:43:52.379789+0000","last_peered":"2024-09-17T22:43:52.379789+0000","last_clean":"2024-09-17T22:43:52.379789+0000","last_became_active":"2024-09-17T22:41:19.451723+0000","last_became_peered":"2024-09-17T22:41:19.451723+0000","last_unstale":"2024-09-17T22:43:52.379789+0000","last_undegraded":"2024-09-17T22:43:52.379789+0000","last_fullsized":"2024-09-17T22:43:52.379789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:50:14.037236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061483000000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":15368,"num_read_kb":33652,"num_write":428,"num_write_kb":8748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"139'879","reported_seq":12126,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.391763+0000","last_change":"2024-09-17T22:43:52.391763+0000","last_active":"2024-09-17T22:43:52.391763+0000","last_peered":"2024-09-17T22:43:52.391763+0000","last_clean":"2024-09-17T22:43:52.391763+0000","last_became_active":"2024-09-17T22:43:52.388167+0000","last_became_peered":"2024-09-17T22:43:52.388167+0000","last_unstale":"2024-09-17T22:43:52.391763+0000","last_undegraded":"2024-09-17T22:43:52.391763+0000","last_fullsized":"2024-09-17T22:43:52.391763+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:55:17.606117+0000","scrub_duration":0,"objects_trimmed":42,"snaptrim_duration":0.107009245,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":15325,"num_read_kb":33693,"num_write":856,"num_write_kb":8706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":4229520,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":204,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381267+0000","last_change":"2024-09-17T22:43:42.285641+0000","last_active":"2024-09-17T22:43:52.381267+0000","last_peered":"2024-09-17T22:43:52.381267+0000","last_clean":"2024-09-17T22:43:52.381267+0000","last_became_active":"2024-09-17T22:43:42.285307+0000","last_became_peered":"2024-09-17T22:43:42.285307+0000","last_unstale":"2024-09-17T22:43:52.381267+0000","last_undegraded":"2024-09-17T22:43:52.381267+0000","last_fullsized":"2024-09-17T22:43:52.381267+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:39.203508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"130'385","reported_seq":4190,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749152+0000","last_change":"2024-09-17T22:43:19.191554+0000","last_active":"2024-09-17T22:43:51.749152+0000","last_peered":"2024-09-17T22:43:51.749152+0000","last_clean":"2024-09-17T22:43:51.749152+0000","last_became_active":"2024-09-17T22:41:19.444314+0000","last_became_peered":"2024-09-17T22:41:19.444314+0000","last_unstale":"2024-09-17T22:43:51.749152+0000","last_undegraded":"2024-09-17T22:43:51.749152+0000","last_fullsized":"2024-09-17T22:43:51.749152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:07:00.374588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011857370000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4006,"num_read_kb":22354,"num_write":358,"num_write_kb":9353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":286,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.358312+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:43:51.358312+0000","last_peered":"2024-09-17T22:43:51.358312+0000","last_clean":"2024-09-17T22:43:51.358312+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:43:51.358312+0000","last_undegraded":"2024-09-17T22:43:51.358312+0000","last_fullsized":"2024-09-17T22:43:51.358312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"130'481","reported_seq":7455,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.358137+0000","last_change":"2024-09-17T22:43:19.150805+0000","last_active":"2024-09-17T22:43:51.358137+0000","last_peered":"2024-09-17T22:43:51.358137+0000","last_clean":"2024-09-17T22:43:51.358137+0000","last_became_active":"2024-09-17T22:41:19.449791+0000","last_became_peered":"2024-09-17T22:41:19.449791+0000","last_unstale":"2024-09-17T22:43:51.358137+0000","last_undegraded":"2024-09-17T22:43:51.358137+0000","last_fullsized":"2024-09-17T22:43:51.358137+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:48.110050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088469700000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8092,"num_read_kb":9247,"num_write":519,"num_write_kb":4605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":288,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380327+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:43:52.380327+0000","last_peered":"2024-09-17T22:43:52.380327+0000","last_clean":"2024-09-17T22:43:52.380327+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:43:52.380327+0000","last_undegraded":"2024-09-17T22:43:52.380327+0000","last_fullsized":"2024-09-17T22:43:52.380327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"131'756","reported_seq":8739,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352735+0000","last_change":"2024-09-17T22:43:19.148879+0000","last_active":"2024-09-17T22:43:51.352735+0000","last_peered":"2024-09-17T22:43:51.352735+0000","last_clean":"2024-09-17T22:43:51.352735+0000","last_became_active":"2024-09-17T22:41:19.449279+0000","last_became_peered":"2024-09-17T22:41:19.449279+0000","last_unstale":"2024-09-17T22:43:51.352735+0000","last_undegraded":"2024-09-17T22:43:51.352735+0000","last_fullsized":"2024-09-17T22:43:51.352735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:51:59.677897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089414599999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":11291,"num_read_kb":22947,"num_write":956,"num_write_kb":2860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":203,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381466+0000","last_change":"2024-09-17T22:43:42.286959+0000","last_active":"2024-09-17T22:43:52.381466+0000","last_peered":"2024-09-17T22:43:52.381466+0000","last_clean":"2024-09-17T22:43:52.381466+0000","last_became_active":"2024-09-17T22:43:42.286652+0000","last_became_peered":"2024-09-17T22:43:42.286652+0000","last_unstale":"2024-09-17T22:43:52.381466+0000","last_undegraded":"2024-09-17T22:43:52.381466+0000","last_fullsized":"2024-09-17T22:43:52.381466+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:50:39.824946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":438,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357877+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:43:51.357877+0000","last_peered":"2024-09-17T22:43:51.357877+0000","last_clean":"2024-09-17T22:43:51.357877+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:43:51.357877+0000","last_undegraded":"2024-09-17T22:43:51.357877+0000","last_fullsized":"2024-09-17T22:43:51.357877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"129'613","reported_seq":11950,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352720+0000","last_change":"2024-09-17T22:43:19.149769+0000","last_active":"2024-09-17T22:43:51.352720+0000","last_peered":"2024-09-17T22:43:51.352720+0000","last_clean":"2024-09-17T22:43:51.352720+0000","last_became_active":"2024-09-17T22:41:19.448956+0000","last_became_peered":"2024-09-17T22:41:19.448956+0000","last_unstale":"2024-09-17T22:43:51.352720+0000","last_undegraded":"2024-09-17T22:43:51.352720+0000","last_fullsized":"2024-09-17T22:43:51.352720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:28:45.951042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017463789999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13252,"num_read_kb":46911,"num_write":655,"num_write_kb":25175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"129'785","reported_seq":14761,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381062+0000","last_change":"2024-09-17T22:43:19.145329+0000","last_active":"2024-09-17T22:43:52.381062+0000","last_peered":"2024-09-17T22:43:52.381062+0000","last_clean":"2024-09-17T22:43:52.381062+0000","last_became_active":"2024-09-17T22:41:19.449003+0000","last_became_peered":"2024-09-17T22:41:19.449003+0000","last_unstale":"2024-09-17T22:43:52.381062+0000","last_undegraded":"2024-09-17T22:43:52.381062+0000","last_fullsized":"2024-09-17T22:43:52.381062+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:13:00.810670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910964,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":16223,"num_read_kb":43243,"num_write":1040,"num_write_kb":9843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"130'470","reported_seq":8486,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381176+0000","last_change":"2024-09-17T22:43:42.285286+0000","last_active":"2024-09-17T22:43:52.381176+0000","last_peered":"2024-09-17T22:43:52.381176+0000","last_clean":"2024-09-17T22:43:52.381176+0000","last_became_active":"2024-09-17T22:43:42.284094+0000","last_became_peered":"2024-09-17T22:43:42.284094+0000","last_unstale":"2024-09-17T22:43:52.381176+0000","last_undegraded":"2024-09-17T22:43:52.381176+0000","last_fullsized":"2024-09-17T22:43:52.381176+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:36:47.430783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001830772,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":8191,"num_read_kb":41081,"num_write":473,"num_write_kb":9769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"131'423","reported_seq":2735,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.358360+0000","last_change":"2024-09-17T22:43:19.142462+0000","last_active":"2024-09-17T22:43:51.358360+0000","last_peered":"2024-09-17T22:43:51.358360+0000","last_clean":"2024-09-17T22:43:51.358360+0000","last_became_active":"2024-09-17T22:41:19.447215+0000","last_became_peered":"2024-09-17T22:41:19.447215+0000","last_unstale":"2024-09-17T22:43:51.358360+0000","last_undegraded":"2024-09-17T22:43:51.358360+0000","last_fullsized":"2024-09-17T22:43:51.358360+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:37:48.721121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066798900000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3409,"num_read_kb":14480,"num_write":404,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"139'353","reported_seq":21688,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381548+0000","last_change":"2024-09-17T22:43:52.381548+0000","last_active":"2024-09-17T22:43:52.381548+0000","last_peered":"2024-09-17T22:43:52.381548+0000","last_clean":"2024-09-17T22:43:52.381548+0000","last_became_active":"2024-09-17T22:43:52.380211+0000","last_became_peered":"2024-09-17T22:43:52.380211+0000","last_unstale":"2024-09-17T22:43:52.381548+0000","last_undegraded":"2024-09-17T22:43:52.381548+0000","last_fullsized":"2024-09-17T22:43:52.381548+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:43:11.256179+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.064717442,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":21473,"num_read_kb":63694,"num_write":340,"num_write_kb":14940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":1192963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"129'486","reported_seq":2031,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380602+0000","last_change":"2024-09-17T22:43:42.283323+0000","last_active":"2024-09-17T22:43:52.380602+0000","last_peered":"2024-09-17T22:43:52.380602+0000","last_clean":"2024-09-17T22:43:52.380602+0000","last_became_active":"2024-09-17T22:43:42.282252+0000","last_became_peered":"2024-09-17T22:43:42.282252+0000","last_unstale":"2024-09-17T22:43:52.380602+0000","last_undegraded":"2024-09-17T22:43:52.380602+0000","last_fullsized":"2024-09-17T22:43:52.380602+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:31.315273+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.002173033,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2216,"num_read_kb":10204,"num_write":574,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"139'372","reported_seq":8839,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.379984+0000","last_change":"2024-09-17T22:43:35.323373+0000","last_active":"2024-09-17T22:43:52.379984+0000","last_peered":"2024-09-17T22:43:52.379984+0000","last_clean":"2024-09-17T22:43:52.379984+0000","last_became_active":"2024-09-17T22:43:35.224141+0000","last_became_peered":"2024-09-17T22:43:35.224141+0000","last_unstale":"2024-09-17T22:43:52.379984+0000","last_undegraded":"2024-09-17T22:43:52.379984+0000","last_fullsized":"2024-09-17T22:43:52.379984+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:01:02.683062+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.098411079999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8804,"num_read_kb":23106,"num_write":291,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":2514411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"139'353","reported_seq":1860,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.390365+0000","last_change":"2024-09-17T22:43:52.390365+0000","last_active":"2024-09-17T22:43:52.390365+0000","last_peered":"2024-09-17T22:43:52.390365+0000","last_clean":"2024-09-17T22:43:52.390365+0000","last_became_active":"2024-09-17T22:43:52.388960+0000","last_became_peered":"2024-09-17T22:43:52.388960+0000","last_unstale":"2024-09-17T22:43:52.390365+0000","last_undegraded":"2024-09-17T22:43:52.390365+0000","last_fullsized":"2024-09-17T22:43:52.390365+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:13:54.534247+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.086636568999999997,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1634,"num_read_kb":5501,"num_write":380,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1563844,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"129'349","reported_seq":2170,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357762+0000","last_change":"2024-09-17T22:43:19.144272+0000","last_active":"2024-09-17T22:43:51.357762+0000","last_peered":"2024-09-17T22:43:51.357762+0000","last_clean":"2024-09-17T22:43:51.357762+0000","last_became_active":"2024-09-17T22:41:19.447419+0000","last_became_peered":"2024-09-17T22:41:19.447419+0000","last_unstale":"2024-09-17T22:43:51.357762+0000","last_undegraded":"2024-09-17T22:43:51.357762+0000","last_fullsized":"2024-09-17T22:43:51.357762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:48.842752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00132408,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3110,"num_read_kb":8489,"num_write":286,"num_write_kb":3161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"129'616","reported_seq":3681,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380951+0000","last_change":"2024-09-17T22:43:19.144657+0000","last_active":"2024-09-17T22:43:52.380951+0000","last_peered":"2024-09-17T22:43:52.380951+0000","last_clean":"2024-09-17T22:43:52.380951+0000","last_became_active":"2024-09-17T22:41:19.454151+0000","last_became_peered":"2024-09-17T22:41:19.454151+0000","last_unstale":"2024-09-17T22:43:52.380951+0000","last_undegraded":"2024-09-17T22:43:52.380951+0000","last_fullsized":"2024-09-17T22:43:52.380951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:39:51.641693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001718979,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5266,"num_read_kb":13546,"num_write":658,"num_write_kb":3731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"130'910","reported_seq":11717,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357917+0000","last_change":"2024-09-17T22:43:35.218645+0000","last_active":"2024-09-17T22:43:51.357917+0000","last_peered":"2024-09-17T22:43:51.357917+0000","last_clean":"2024-09-17T22:43:51.357917+0000","last_became_active":"2024-09-17T22:43:35.217045+0000","last_became_peered":"2024-09-17T22:43:35.217045+0000","last_unstale":"2024-09-17T22:43:51.357917+0000","last_undegraded":"2024-09-17T22:43:51.357917+0000","last_fullsized":"2024-09-17T22:43:51.357917+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:20.169030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001497735,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14642,"num_read_kb":44074,"num_write":1142,"num_write_kb":22161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"139'502","reported_seq":2630,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381373+0000","last_change":"2024-09-17T22:43:35.301824+0000","last_active":"2024-09-17T22:43:52.381373+0000","last_peered":"2024-09-17T22:43:52.381373+0000","last_clean":"2024-09-17T22:43:52.381373+0000","last_became_active":"2024-09-17T22:43:35.217551+0000","last_became_peered":"2024-09-17T22:43:35.217551+0000","last_unstale":"2024-09-17T22:43:52.381373+0000","last_undegraded":"2024-09-17T22:43:52.381373+0000","last_fullsized":"2024-09-17T22:43:52.381373+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:10.204124+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.082900606000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":2503,"num_read_kb":8800,"num_write":617,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":291987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":492,"num_read":268105,"num_read_kb":802236,"num_write":18141,"num_write_kb":293703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":30918130,"num_keys_recovered":86,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11879,"internal_metadata":0},"log_size":16633,"ondisk_log_size":16633,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542183,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28908,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342932,"statfs":{"total":96636764160,"available":96607162368,"internally_reserved":0,"allocated":1921024,"data_stored":1492409,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":135,"seq":579820584967,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34748,"kb_used_data":1828,"kb_used_omap":21,"kb_used_meta":32874,"kb_avail":94337092,"statfs":{"total":96636764160,"available":96601182208,"internally_reserved":0,"allocated":1871872,"data_stored":907980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22295,"internal_metadata":33663209},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542181,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41320,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330520,"statfs":{"total":96636764160,"available":96594452480,"internally_reserved":0,"allocated":1847296,"data_stored":896064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542181,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28848,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342992,"statfs":{"total":96636764160,"available":96607223808,"internally_reserved":0,"allocated":1863680,"data_stored":902041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542181,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46948,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324892,"statfs":{"total":96636764160,"available":96588689408,"internally_reserved":0,"allocated":1908736,"data_stored":1486451,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28848,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342992,"statfs":{"total":96636764160,"available":96607223808,"internally_reserved":0,"allocated":1859584,"data_stored":902022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55232,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316608,"statfs":{"total":96636764160,"available":96580206592,"internally_reserved":0,"allocated":1875968,"data_stored":907999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51068,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320772,"statfs":{"total":96636764160,"available":96584470528,"internally_reserved":0,"allocated":1871872,"data_stored":907980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11879,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:43:57.436 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T22:43:57.437 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-17T22:43:57.728 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-17T22:43:57.738 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T22:43:57.738 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:43:58.012 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:43:58.013 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:43:58.024 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-17T22:43:56.339636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":500,"num_object_clones":0,"num_object_copies":1000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":500,"num_whiteouts":492,"num_read":268221,"num_read_kb":802336,"num_write":18302,"num_write_kb":296121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":30918130,"num_keys_recovered":86,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16753,"ondisk_log_size":16753,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":315920,"kb_used_data":14668,"kb_used_omap":32,"kb_used_meta":301023,"kb_avail":754658800,"statfs":{"total":773094113280,"available":772770611200,"internally_reserved":0,"allocated":15020032,"data_stored":8402946,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":33450,"internal_metadata":308247894},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-114,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001178"},"pg_stats":[{"pgid":"3.1f","version":"131'860","reported_seq":7310,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749216+0000","last_change":"2024-09-17T22:43:19.191421+0000","last_active":"2024-09-17T22:43:51.749216+0000","last_peered":"2024-09-17T22:43:51.749216+0000","last_clean":"2024-09-17T22:43:51.749216+0000","last_became_active":"2024-09-17T22:41:19.443658+0000","last_became_peered":"2024-09-17T22:41:19.443658+0000","last_unstale":"2024-09-17T22:43:51.749216+0000","last_undegraded":"2024-09-17T22:43:51.749216+0000","last_fullsized":"2024-09-17T22:43:51.749216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:58.423329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001073428,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8816,"num_read_kb":14906,"num_write":1102,"num_write_kb":4548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"130'786","reported_seq":7422,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.748986+0000","last_change":"2024-09-17T22:43:19.190807+0000","last_active":"2024-09-17T22:43:51.748986+0000","last_peered":"2024-09-17T22:43:51.748986+0000","last_clean":"2024-09-17T22:43:51.748986+0000","last_became_active":"2024-09-17T22:41:19.449357+0000","last_became_peered":"2024-09-17T22:41:19.449357+0000","last_unstale":"2024-09-17T22:43:51.748986+0000","last_undegraded":"2024-09-17T22:43:51.748986+0000","last_fullsized":"2024-09-17T22:43:51.748986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:20.300342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051110000000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":8865,"num_read_kb":26537,"num_write":1126,"num_write_kb":10958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"131'435","reported_seq":2380,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357945+0000","last_change":"2024-09-17T22:43:19.144579+0000","last_active":"2024-09-17T22:43:51.357945+0000","last_peered":"2024-09-17T22:43:51.357945+0000","last_clean":"2024-09-17T22:43:51.357945+0000","last_became_active":"2024-09-17T22:42:58.002822+0000","last_became_peered":"2024-09-17T22:42:58.002822+0000","last_unstale":"2024-09-17T22:43:51.357945+0000","last_undegraded":"2024-09-17T22:43:51.357945+0000","last_fullsized":"2024-09-17T22:43:51.357945+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:18:33.730453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022657670000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":17641,"num_read_kb":48444,"num_write":377,"num_write_kb":8987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":7308630,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"131'494","reported_seq":2826,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357881+0000","last_change":"2024-09-17T22:43:19.146655+0000","last_active":"2024-09-17T22:43:51.357881+0000","last_peered":"2024-09-17T22:43:51.357881+0000","last_clean":"2024-09-17T22:43:51.357881+0000","last_became_active":"2024-09-17T22:42:34.310411+0000","last_became_peered":"2024-09-17T22:42:34.310411+0000","last_unstale":"2024-09-17T22:43:51.357881+0000","last_undegraded":"2024-09-17T22:43:51.357881+0000","last_fullsized":"2024-09-17T22:43:51.357881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:34.577953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039390110000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":2615,"num_read_kb":14215,"num_write":599,"num_write_kb":12172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":7352772,"num_keys_recovered":74,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"139'244","reported_seq":8116,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352818+0000","last_change":"2024-09-17T22:43:35.308387+0000","last_active":"2024-09-17T22:43:51.352818+0000","last_peered":"2024-09-17T22:43:51.352818+0000","last_clean":"2024-09-17T22:43:51.352818+0000","last_became_active":"2024-09-17T22:43:35.215952+0000","last_became_peered":"2024-09-17T22:43:35.215952+0000","last_unstale":"2024-09-17T22:43:51.352818+0000","last_undegraded":"2024-09-17T22:43:51.352818+0000","last_fullsized":"2024-09-17T22:43:51.352818+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:45.376055+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.091482261999999995,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7315,"num_read_kb":22822,"num_write":186,"num_write_kb":6215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":3640098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"131'360","reported_seq":5241,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357959+0000","last_change":"2024-09-17T22:43:19.145987+0000","last_active":"2024-09-17T22:43:51.357959+0000","last_peered":"2024-09-17T22:43:51.357959+0000","last_clean":"2024-09-17T22:43:51.357959+0000","last_became_active":"2024-09-17T22:41:19.446541+0000","last_became_peered":"2024-09-17T22:41:19.446541+0000","last_unstale":"2024-09-17T22:43:51.357959+0000","last_undegraded":"2024-09-17T22:43:51.357959+0000","last_fullsized":"2024-09-17T22:43:51.357959+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:12:51.399521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035192800000000001,"stat_sum":{"num_bytes":19,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":5091,"num_read_kb":19688,"num_write":329,"num_write_kb":4179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"130'391","reported_seq":6764,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352819+0000","last_change":"2024-09-17T22:43:19.150923+0000","last_active":"2024-09-17T22:43:51.352819+0000","last_peered":"2024-09-17T22:43:51.352819+0000","last_clean":"2024-09-17T22:43:51.352819+0000","last_became_active":"2024-09-17T22:41:19.449940+0000","last_became_peered":"2024-09-17T22:41:19.449940+0000","last_unstale":"2024-09-17T22:43:51.352819+0000","last_undegraded":"2024-09-17T22:43:51.352819+0000","last_fullsized":"2024-09-17T22:43:51.352819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:57:32.786125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002665995,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6668,"num_read_kb":22143,"num_write":345,"num_write_kb":9035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"129'383","reported_seq":3221,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749001+0000","last_change":"2024-09-17T22:43:19.191965+0000","last_active":"2024-09-17T22:43:51.749001+0000","last_peered":"2024-09-17T22:43:51.749001+0000","last_clean":"2024-09-17T22:43:51.749001+0000","last_became_active":"2024-09-17T22:41:19.446287+0000","last_became_peered":"2024-09-17T22:41:19.446287+0000","last_unstale":"2024-09-17T22:43:51.749001+0000","last_undegraded":"2024-09-17T22:43:51.749001+0000","last_fullsized":"2024-09-17T22:43:51.749001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:23:52.909972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001462812,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3441,"num_read_kb":20083,"num_write":323,"num_write_kb":10930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"131'513","reported_seq":2752,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749198+0000","last_change":"2024-09-17T22:43:19.191786+0000","last_active":"2024-09-17T22:43:51.749198+0000","last_peered":"2024-09-17T22:43:51.749198+0000","last_clean":"2024-09-17T22:43:51.749198+0000","last_became_active":"2024-09-17T22:41:19.445923+0000","last_became_peered":"2024-09-17T22:41:19.445923+0000","last_unstale":"2024-09-17T22:43:51.749198+0000","last_undegraded":"2024-09-17T22:43:51.749198+0000","last_fullsized":"2024-09-17T22:43:51.749198+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:02:33.097995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001414085,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3445,"num_read_kb":15105,"num_write":513,"num_write_kb":8714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"129'562","reported_seq":22396,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.386833+0000","last_change":"2024-09-17T22:43:52.386833+0000","last_active":"2024-09-17T22:43:52.386833+0000","last_peered":"2024-09-17T22:43:52.386833+0000","last_clean":"2024-09-17T22:43:52.386833+0000","last_became_active":"2024-09-17T22:43:52.384987+0000","last_became_peered":"2024-09-17T22:43:52.384987+0000","last_unstale":"2024-09-17T22:43:52.386833+0000","last_undegraded":"2024-09-17T22:43:52.386833+0000","last_fullsized":"2024-09-17T22:43:52.386833+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:08:09.285837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016446880000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":22869,"num_read_kb":64042,"num_write":605,"num_write_kb":12652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"139'437","reported_seq":7603,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381556+0000","last_change":"2024-09-17T22:43:52.381556+0000","last_active":"2024-09-17T22:43:52.381556+0000","last_peered":"2024-09-17T22:43:52.381556+0000","last_clean":"2024-09-17T22:43:52.381556+0000","last_became_active":"2024-09-17T22:43:52.380232+0000","last_became_peered":"2024-09-17T22:43:52.380232+0000","last_unstale":"2024-09-17T22:43:52.381556+0000","last_undegraded":"2024-09-17T22:43:52.381556+0000","last_fullsized":"2024-09-17T22:43:52.381556+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:08:17.722346+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.062977108000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8763,"num_read_kb":20352,"num_write":343,"num_write_kb":4822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2823905,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"129'589","reported_seq":7857,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380494+0000","last_change":"2024-09-17T22:43:19.143328+0000","last_active":"2024-09-17T22:43:52.380494+0000","last_peered":"2024-09-17T22:43:52.380494+0000","last_clean":"2024-09-17T22:43:52.380494+0000","last_became_active":"2024-09-17T22:41:19.451631+0000","last_became_peered":"2024-09-17T22:41:19.451631+0000","last_unstale":"2024-09-17T22:43:52.380494+0000","last_undegraded":"2024-09-17T22:43:52.380494+0000","last_fullsized":"2024-09-17T22:43:52.380494+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:31.809865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090157099999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8321,"num_read_kb":19442,"num_write":736,"num_write_kb":19536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":295,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380849+0000","last_change":"2024-09-17T22:43:32.195958+0000","last_active":"2024-09-17T22:43:52.380849+0000","last_peered":"2024-09-17T22:43:52.380849+0000","last_clean":"2024-09-17T22:43:52.380849+0000","last_became_active":"2024-09-17T22:43:32.195676+0000","last_became_peered":"2024-09-17T22:43:32.195676+0000","last_unstale":"2024-09-17T22:43:52.380849+0000","last_undegraded":"2024-09-17T22:43:52.380849+0000","last_fullsized":"2024-09-17T22:43:52.380849+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:41:52.669174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"131'579","reported_seq":5338,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352815+0000","last_change":"2024-09-17T22:43:19.150601+0000","last_active":"2024-09-17T22:43:51.352815+0000","last_peered":"2024-09-17T22:43:51.352815+0000","last_clean":"2024-09-17T22:43:51.352815+0000","last_became_active":"2024-09-17T22:41:19.448966+0000","last_became_peered":"2024-09-17T22:41:19.448966+0000","last_unstale":"2024-09-17T22:43:51.352815+0000","last_undegraded":"2024-09-17T22:43:51.352815+0000","last_fullsized":"2024-09-17T22:43:51.352815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:20:32.636962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026117269999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6258,"num_read_kb":29920,"num_write":539,"num_write_kb":6243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":292,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352861+0000","last_change":"2024-09-17T22:43:32.195712+0000","last_active":"2024-09-17T22:43:51.352861+0000","last_peered":"2024-09-17T22:43:51.352861+0000","last_clean":"2024-09-17T22:43:51.352861+0000","last_became_active":"2024-09-17T22:43:32.195451+0000","last_became_peered":"2024-09-17T22:43:32.195451+0000","last_unstale":"2024-09-17T22:43:51.352861+0000","last_undegraded":"2024-09-17T22:43:51.352861+0000","last_fullsized":"2024-09-17T22:43:51.352861+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:52.062601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"131'396","reported_seq":1773,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352783+0000","last_change":"2024-09-17T22:43:19.149604+0000","last_active":"2024-09-17T22:43:51.352783+0000","last_peered":"2024-09-17T22:43:51.352783+0000","last_clean":"2024-09-17T22:43:51.352783+0000","last_became_active":"2024-09-17T22:41:19.451637+0000","last_became_peered":"2024-09-17T22:41:19.451637+0000","last_unstale":"2024-09-17T22:43:51.352783+0000","last_undegraded":"2024-09-17T22:43:51.352783+0000","last_fullsized":"2024-09-17T22:43:51.352783+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:16.254887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001593059,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1474,"num_read_kb":9975,"num_write":408,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":293,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380692+0000","last_change":"2024-09-17T22:43:15.154296+0000","last_active":"2024-09-17T22:43:52.380692+0000","last_peered":"2024-09-17T22:43:52.380692+0000","last_clean":"2024-09-17T22:43:52.380692+0000","last_became_active":"2024-09-17T22:43:15.153956+0000","last_became_peered":"2024-09-17T22:43:15.153956+0000","last_unstale":"2024-09-17T22:43:52.380692+0000","last_undegraded":"2024-09-17T22:43:52.380692+0000","last_fullsized":"2024-09-17T22:43:52.380692+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:19.340692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"131'478","reported_seq":1431,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357944+0000","last_change":"2024-09-17T22:43:19.151215+0000","last_active":"2024-09-17T22:43:51.357944+0000","last_peered":"2024-09-17T22:43:51.357944+0000","last_clean":"2024-09-17T22:43:51.357944+0000","last_became_active":"2024-09-17T22:41:19.444893+0000","last_became_peered":"2024-09-17T22:41:19.444893+0000","last_unstale":"2024-09-17T22:43:51.357944+0000","last_undegraded":"2024-09-17T22:43:51.357944+0000","last_fullsized":"2024-09-17T22:43:51.357944+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:34:31.498807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001189308,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1718,"num_read_kb":9540,"num_write":633,"num_write_kb":7900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":286,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749190+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:43:51.749190+0000","last_peered":"2024-09-17T22:43:51.749190+0000","last_clean":"2024-09-17T22:43:51.749190+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:43:51.749190+0000","last_undegraded":"2024-09-17T22:43:51.749190+0000","last_fullsized":"2024-09-17T22:43:51.749190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"130'393","reported_seq":15619,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.379789+0000","last_change":"2024-09-17T22:43:19.187091+0000","last_active":"2024-09-17T22:43:52.379789+0000","last_peered":"2024-09-17T22:43:52.379789+0000","last_clean":"2024-09-17T22:43:52.379789+0000","last_became_active":"2024-09-17T22:41:19.451723+0000","last_became_peered":"2024-09-17T22:41:19.451723+0000","last_unstale":"2024-09-17T22:43:52.379789+0000","last_undegraded":"2024-09-17T22:43:52.379789+0000","last_fullsized":"2024-09-17T22:43:52.379789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:50:14.037236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061483000000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":15368,"num_read_kb":33652,"num_write":428,"num_write_kb":8748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"139'879","reported_seq":12126,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.391763+0000","last_change":"2024-09-17T22:43:52.391763+0000","last_active":"2024-09-17T22:43:52.391763+0000","last_peered":"2024-09-17T22:43:52.391763+0000","last_clean":"2024-09-17T22:43:52.391763+0000","last_became_active":"2024-09-17T22:43:52.388167+0000","last_became_peered":"2024-09-17T22:43:52.388167+0000","last_unstale":"2024-09-17T22:43:52.391763+0000","last_undegraded":"2024-09-17T22:43:52.391763+0000","last_fullsized":"2024-09-17T22:43:52.391763+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:55:17.606117+0000","scrub_duration":0,"objects_trimmed":42,"snaptrim_duration":0.107009245,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":15325,"num_read_kb":33693,"num_write":856,"num_write_kb":8706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":4229520,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":204,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381267+0000","last_change":"2024-09-17T22:43:42.285641+0000","last_active":"2024-09-17T22:43:52.381267+0000","last_peered":"2024-09-17T22:43:52.381267+0000","last_clean":"2024-09-17T22:43:52.381267+0000","last_became_active":"2024-09-17T22:43:42.285307+0000","last_became_peered":"2024-09-17T22:43:42.285307+0000","last_unstale":"2024-09-17T22:43:52.381267+0000","last_undegraded":"2024-09-17T22:43:52.381267+0000","last_fullsized":"2024-09-17T22:43:52.381267+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:39.203508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"130'385","reported_seq":4190,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749152+0000","last_change":"2024-09-17T22:43:19.191554+0000","last_active":"2024-09-17T22:43:51.749152+0000","last_peered":"2024-09-17T22:43:51.749152+0000","last_clean":"2024-09-17T22:43:51.749152+0000","last_became_active":"2024-09-17T22:41:19.444314+0000","last_became_peered":"2024-09-17T22:41:19.444314+0000","last_unstale":"2024-09-17T22:43:51.749152+0000","last_undegraded":"2024-09-17T22:43:51.749152+0000","last_fullsized":"2024-09-17T22:43:51.749152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:07:00.374588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011857370000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4006,"num_read_kb":22354,"num_write":358,"num_write_kb":9353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":286,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.358312+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:43:51.358312+0000","last_peered":"2024-09-17T22:43:51.358312+0000","last_clean":"2024-09-17T22:43:51.358312+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:43:51.358312+0000","last_undegraded":"2024-09-17T22:43:51.358312+0000","last_fullsized":"2024-09-17T22:43:51.358312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"130'481","reported_seq":7455,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.358137+0000","last_change":"2024-09-17T22:43:19.150805+0000","last_active":"2024-09-17T22:43:51.358137+0000","last_peered":"2024-09-17T22:43:51.358137+0000","last_clean":"2024-09-17T22:43:51.358137+0000","last_became_active":"2024-09-17T22:41:19.449791+0000","last_became_peered":"2024-09-17T22:41:19.449791+0000","last_unstale":"2024-09-17T22:43:51.358137+0000","last_undegraded":"2024-09-17T22:43:51.358137+0000","last_fullsized":"2024-09-17T22:43:51.358137+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:48.110050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088469700000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8092,"num_read_kb":9247,"num_write":519,"num_write_kb":4605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":288,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380327+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:43:52.380327+0000","last_peered":"2024-09-17T22:43:52.380327+0000","last_clean":"2024-09-17T22:43:52.380327+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:43:52.380327+0000","last_undegraded":"2024-09-17T22:43:52.380327+0000","last_fullsized":"2024-09-17T22:43:52.380327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"131'756","reported_seq":8739,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352735+0000","last_change":"2024-09-17T22:43:19.148879+0000","last_active":"2024-09-17T22:43:51.352735+0000","last_peered":"2024-09-17T22:43:51.352735+0000","last_clean":"2024-09-17T22:43:51.352735+0000","last_became_active":"2024-09-17T22:41:19.449279+0000","last_became_peered":"2024-09-17T22:41:19.449279+0000","last_unstale":"2024-09-17T22:43:51.352735+0000","last_undegraded":"2024-09-17T22:43:51.352735+0000","last_fullsized":"2024-09-17T22:43:51.352735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:51:59.677897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089414599999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":11291,"num_read_kb":22947,"num_write":956,"num_write_kb":2860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":203,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381466+0000","last_change":"2024-09-17T22:43:42.286959+0000","last_active":"2024-09-17T22:43:52.381466+0000","last_peered":"2024-09-17T22:43:52.381466+0000","last_clean":"2024-09-17T22:43:52.381466+0000","last_became_active":"2024-09-17T22:43:42.286652+0000","last_became_peered":"2024-09-17T22:43:42.286652+0000","last_unstale":"2024-09-17T22:43:52.381466+0000","last_undegraded":"2024-09-17T22:43:52.381466+0000","last_fullsized":"2024-09-17T22:43:52.381466+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:50:39.824946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":438,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357877+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:43:51.357877+0000","last_peered":"2024-09-17T22:43:51.357877+0000","last_clean":"2024-09-17T22:43:51.357877+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:43:51.357877+0000","last_undegraded":"2024-09-17T22:43:51.357877+0000","last_fullsized":"2024-09-17T22:43:51.357877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"129'613","reported_seq":11950,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352720+0000","last_change":"2024-09-17T22:43:19.149769+0000","last_active":"2024-09-17T22:43:51.352720+0000","last_peered":"2024-09-17T22:43:51.352720+0000","last_clean":"2024-09-17T22:43:51.352720+0000","last_became_active":"2024-09-17T22:41:19.448956+0000","last_became_peered":"2024-09-17T22:41:19.448956+0000","last_unstale":"2024-09-17T22:43:51.352720+0000","last_undegraded":"2024-09-17T22:43:51.352720+0000","last_fullsized":"2024-09-17T22:43:51.352720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:28:45.951042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017463789999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13252,"num_read_kb":46911,"num_write":655,"num_write_kb":25175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"129'785","reported_seq":14761,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381062+0000","last_change":"2024-09-17T22:43:19.145329+0000","last_active":"2024-09-17T22:43:52.381062+0000","last_peered":"2024-09-17T22:43:52.381062+0000","last_clean":"2024-09-17T22:43:52.381062+0000","last_became_active":"2024-09-17T22:41:19.449003+0000","last_became_peered":"2024-09-17T22:41:19.449003+0000","last_unstale":"2024-09-17T22:43:52.381062+0000","last_undegraded":"2024-09-17T22:43:52.381062+0000","last_fullsized":"2024-09-17T22:43:52.381062+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:13:00.810670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910964,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":16223,"num_read_kb":43243,"num_write":1040,"num_write_kb":9843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"130'470","reported_seq":8486,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381176+0000","last_change":"2024-09-17T22:43:42.285286+0000","last_active":"2024-09-17T22:43:52.381176+0000","last_peered":"2024-09-17T22:43:52.381176+0000","last_clean":"2024-09-17T22:43:52.381176+0000","last_became_active":"2024-09-17T22:43:42.284094+0000","last_became_peered":"2024-09-17T22:43:42.284094+0000","last_unstale":"2024-09-17T22:43:52.381176+0000","last_undegraded":"2024-09-17T22:43:52.381176+0000","last_fullsized":"2024-09-17T22:43:52.381176+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:36:47.430783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001830772,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":8191,"num_read_kb":41081,"num_write":473,"num_write_kb":9769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"131'423","reported_seq":2735,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.358360+0000","last_change":"2024-09-17T22:43:19.142462+0000","last_active":"2024-09-17T22:43:51.358360+0000","last_peered":"2024-09-17T22:43:51.358360+0000","last_clean":"2024-09-17T22:43:51.358360+0000","last_became_active":"2024-09-17T22:41:19.447215+0000","last_became_peered":"2024-09-17T22:41:19.447215+0000","last_unstale":"2024-09-17T22:43:51.358360+0000","last_undegraded":"2024-09-17T22:43:51.358360+0000","last_fullsized":"2024-09-17T22:43:51.358360+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:37:48.721121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066798900000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3409,"num_read_kb":14480,"num_write":404,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"139'353","reported_seq":21688,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381548+0000","last_change":"2024-09-17T22:43:52.381548+0000","last_active":"2024-09-17T22:43:52.381548+0000","last_peered":"2024-09-17T22:43:52.381548+0000","last_clean":"2024-09-17T22:43:52.381548+0000","last_became_active":"2024-09-17T22:43:52.380211+0000","last_became_peered":"2024-09-17T22:43:52.380211+0000","last_unstale":"2024-09-17T22:43:52.381548+0000","last_undegraded":"2024-09-17T22:43:52.381548+0000","last_fullsized":"2024-09-17T22:43:52.381548+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:43:11.256179+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.064717442,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":21473,"num_read_kb":63694,"num_write":340,"num_write_kb":14940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":1192963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"129'486","reported_seq":2031,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380602+0000","last_change":"2024-09-17T22:43:42.283323+0000","last_active":"2024-09-17T22:43:52.380602+0000","last_peered":"2024-09-17T22:43:52.380602+0000","last_clean":"2024-09-17T22:43:52.380602+0000","last_became_active":"2024-09-17T22:43:42.282252+0000","last_became_peered":"2024-09-17T22:43:42.282252+0000","last_unstale":"2024-09-17T22:43:52.380602+0000","last_undegraded":"2024-09-17T22:43:52.380602+0000","last_fullsized":"2024-09-17T22:43:52.380602+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:31.315273+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.002173033,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2216,"num_read_kb":10204,"num_write":574,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"139'372","reported_seq":8839,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.379984+0000","last_change":"2024-09-17T22:43:35.323373+0000","last_active":"2024-09-17T22:43:52.379984+0000","last_peered":"2024-09-17T22:43:52.379984+0000","last_clean":"2024-09-17T22:43:52.379984+0000","last_became_active":"2024-09-17T22:43:35.224141+0000","last_became_peered":"2024-09-17T22:43:35.224141+0000","last_unstale":"2024-09-17T22:43:52.379984+0000","last_undegraded":"2024-09-17T22:43:52.379984+0000","last_fullsized":"2024-09-17T22:43:52.379984+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:01:02.683062+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.098411079999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8804,"num_read_kb":23106,"num_write":291,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":2514411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"139'353","reported_seq":1860,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.390365+0000","last_change":"2024-09-17T22:43:52.390365+0000","last_active":"2024-09-17T22:43:52.390365+0000","last_peered":"2024-09-17T22:43:52.390365+0000","last_clean":"2024-09-17T22:43:52.390365+0000","last_became_active":"2024-09-17T22:43:52.388960+0000","last_became_peered":"2024-09-17T22:43:52.388960+0000","last_unstale":"2024-09-17T22:43:52.390365+0000","last_undegraded":"2024-09-17T22:43:52.390365+0000","last_fullsized":"2024-09-17T22:43:52.390365+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:13:54.534247+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.086636568999999997,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1634,"num_read_kb":5501,"num_write":380,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1563844,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"129'349","reported_seq":2170,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357762+0000","last_change":"2024-09-17T22:43:19.144272+0000","last_active":"2024-09-17T22:43:51.357762+0000","last_peered":"2024-09-17T22:43:51.357762+0000","last_clean":"2024-09-17T22:43:51.357762+0000","last_became_active":"2024-09-17T22:41:19.447419+0000","last_became_peered":"2024-09-17T22:41:19.447419+0000","last_unstale":"2024-09-17T22:43:51.357762+0000","last_undegraded":"2024-09-17T22:43:51.357762+0000","last_fullsized":"2024-09-17T22:43:51.357762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:48.842752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00132408,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3110,"num_read_kb":8489,"num_write":286,"num_write_kb":3161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"129'616","reported_seq":3681,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380951+0000","last_change":"2024-09-17T22:43:19.144657+0000","last_active":"2024-09-17T22:43:52.380951+0000","last_peered":"2024-09-17T22:43:52.380951+0000","last_clean":"2024-09-17T22:43:52.380951+0000","last_became_active":"2024-09-17T22:41:19.454151+0000","last_became_peered":"2024-09-17T22:41:19.454151+0000","last_unstale":"2024-09-17T22:43:52.380951+0000","last_undegraded":"2024-09-17T22:43:52.380951+0000","last_fullsized":"2024-09-17T22:43:52.380951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:39:51.641693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001718979,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5266,"num_read_kb":13546,"num_write":658,"num_write_kb":3731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"130'910","reported_seq":11717,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357917+0000","last_change":"2024-09-17T22:43:35.218645+0000","last_active":"2024-09-17T22:43:51.357917+0000","last_peered":"2024-09-17T22:43:51.357917+0000","last_clean":"2024-09-17T22:43:51.357917+0000","last_became_active":"2024-09-17T22:43:35.217045+0000","last_became_peered":"2024-09-17T22:43:35.217045+0000","last_unstale":"2024-09-17T22:43:51.357917+0000","last_undegraded":"2024-09-17T22:43:51.357917+0000","last_fullsized":"2024-09-17T22:43:51.357917+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:20.169030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001497735,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14642,"num_read_kb":44074,"num_write":1142,"num_write_kb":22161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"139'502","reported_seq":2630,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381373+0000","last_change":"2024-09-17T22:43:35.301824+0000","last_active":"2024-09-17T22:43:52.381373+0000","last_peered":"2024-09-17T22:43:52.381373+0000","last_clean":"2024-09-17T22:43:52.381373+0000","last_became_active":"2024-09-17T22:43:35.217551+0000","last_became_peered":"2024-09-17T22:43:35.217551+0000","last_unstale":"2024-09-17T22:43:52.381373+0000","last_undegraded":"2024-09-17T22:43:52.381373+0000","last_fullsized":"2024-09-17T22:43:52.381373+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:10.204124+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.082900606000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":2503,"num_read_kb":8800,"num_write":617,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":291987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":492,"num_read":268105,"num_read_kb":802236,"num_write":18141,"num_write_kb":293703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":30918130,"num_keys_recovered":86,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11879,"internal_metadata":0},"log_size":16633,"ondisk_log_size":16633,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542183,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28908,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342932,"statfs":{"total":96636764160,"available":96607162368,"internally_reserved":0,"allocated":1921024,"data_stored":1492409,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":135,"seq":579820584967,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34748,"kb_used_data":1828,"kb_used_omap":21,"kb_used_meta":32874,"kb_avail":94337092,"statfs":{"total":96636764160,"available":96601182208,"internally_reserved":0,"allocated":1871872,"data_stored":907980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22295,"internal_metadata":33663209},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542181,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41320,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330520,"statfs":{"total":96636764160,"available":96594452480,"internally_reserved":0,"allocated":1847296,"data_stored":896064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542181,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28848,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342992,"statfs":{"total":96636764160,"available":96607223808,"internally_reserved":0,"allocated":1863680,"data_stored":902041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542181,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46948,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324892,"statfs":{"total":96636764160,"available":96588689408,"internally_reserved":0,"allocated":1908736,"data_stored":1486451,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28848,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342992,"statfs":{"total":96636764160,"available":96607223808,"internally_reserved":0,"allocated":1859584,"data_stored":902022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55232,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316608,"statfs":{"total":96636764160,"available":96580206592,"internally_reserved":0,"allocated":1875968,"data_stored":907999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51068,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320772,"statfs":{"total":96636764160,"available":96584470528,"internally_reserved":0,"allocated":1871872,"data_stored":907980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11879,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:43:58.026 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:43:58.304 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:43:58.304 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:43:58.316 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-17T22:43:56.339636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":500,"num_object_clones":0,"num_object_copies":1000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":500,"num_whiteouts":492,"num_read":268221,"num_read_kb":802336,"num_write":18302,"num_write_kb":296121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":30918130,"num_keys_recovered":86,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16753,"ondisk_log_size":16753,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":315920,"kb_used_data":14668,"kb_used_omap":32,"kb_used_meta":301023,"kb_avail":754658800,"statfs":{"total":773094113280,"available":772770611200,"internally_reserved":0,"allocated":15020032,"data_stored":8402946,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":33450,"internal_metadata":308247894},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-114,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001178"},"pg_stats":[{"pgid":"3.1f","version":"131'860","reported_seq":7310,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749216+0000","last_change":"2024-09-17T22:43:19.191421+0000","last_active":"2024-09-17T22:43:51.749216+0000","last_peered":"2024-09-17T22:43:51.749216+0000","last_clean":"2024-09-17T22:43:51.749216+0000","last_became_active":"2024-09-17T22:41:19.443658+0000","last_became_peered":"2024-09-17T22:41:19.443658+0000","last_unstale":"2024-09-17T22:43:51.749216+0000","last_undegraded":"2024-09-17T22:43:51.749216+0000","last_fullsized":"2024-09-17T22:43:51.749216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:58.423329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001073428,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8816,"num_read_kb":14906,"num_write":1102,"num_write_kb":4548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"130'786","reported_seq":7422,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.748986+0000","last_change":"2024-09-17T22:43:19.190807+0000","last_active":"2024-09-17T22:43:51.748986+0000","last_peered":"2024-09-17T22:43:51.748986+0000","last_clean":"2024-09-17T22:43:51.748986+0000","last_became_active":"2024-09-17T22:41:19.449357+0000","last_became_peered":"2024-09-17T22:41:19.449357+0000","last_unstale":"2024-09-17T22:43:51.748986+0000","last_undegraded":"2024-09-17T22:43:51.748986+0000","last_fullsized":"2024-09-17T22:43:51.748986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:20.300342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051110000000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":8865,"num_read_kb":26537,"num_write":1126,"num_write_kb":10958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"131'435","reported_seq":2380,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357945+0000","last_change":"2024-09-17T22:43:19.144579+0000","last_active":"2024-09-17T22:43:51.357945+0000","last_peered":"2024-09-17T22:43:51.357945+0000","last_clean":"2024-09-17T22:43:51.357945+0000","last_became_active":"2024-09-17T22:42:58.002822+0000","last_became_peered":"2024-09-17T22:42:58.002822+0000","last_unstale":"2024-09-17T22:43:51.357945+0000","last_undegraded":"2024-09-17T22:43:51.357945+0000","last_fullsized":"2024-09-17T22:43:51.357945+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:18:33.730453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022657670000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":17641,"num_read_kb":48444,"num_write":377,"num_write_kb":8987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":7308630,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"131'494","reported_seq":2826,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357881+0000","last_change":"2024-09-17T22:43:19.146655+0000","last_active":"2024-09-17T22:43:51.357881+0000","last_peered":"2024-09-17T22:43:51.357881+0000","last_clean":"2024-09-17T22:43:51.357881+0000","last_became_active":"2024-09-17T22:42:34.310411+0000","last_became_peered":"2024-09-17T22:42:34.310411+0000","last_unstale":"2024-09-17T22:43:51.357881+0000","last_undegraded":"2024-09-17T22:43:51.357881+0000","last_fullsized":"2024-09-17T22:43:51.357881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:34.577953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039390110000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":2615,"num_read_kb":14215,"num_write":599,"num_write_kb":12172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":7352772,"num_keys_recovered":74,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"139'244","reported_seq":8116,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352818+0000","last_change":"2024-09-17T22:43:35.308387+0000","last_active":"2024-09-17T22:43:51.352818+0000","last_peered":"2024-09-17T22:43:51.352818+0000","last_clean":"2024-09-17T22:43:51.352818+0000","last_became_active":"2024-09-17T22:43:35.215952+0000","last_became_peered":"2024-09-17T22:43:35.215952+0000","last_unstale":"2024-09-17T22:43:51.352818+0000","last_undegraded":"2024-09-17T22:43:51.352818+0000","last_fullsized":"2024-09-17T22:43:51.352818+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:45.376055+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.091482261999999995,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7315,"num_read_kb":22822,"num_write":186,"num_write_kb":6215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":3640098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"131'360","reported_seq":5241,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357959+0000","last_change":"2024-09-17T22:43:19.145987+0000","last_active":"2024-09-17T22:43:51.357959+0000","last_peered":"2024-09-17T22:43:51.357959+0000","last_clean":"2024-09-17T22:43:51.357959+0000","last_became_active":"2024-09-17T22:41:19.446541+0000","last_became_peered":"2024-09-17T22:41:19.446541+0000","last_unstale":"2024-09-17T22:43:51.357959+0000","last_undegraded":"2024-09-17T22:43:51.357959+0000","last_fullsized":"2024-09-17T22:43:51.357959+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:12:51.399521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035192800000000001,"stat_sum":{"num_bytes":19,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":5091,"num_read_kb":19688,"num_write":329,"num_write_kb":4179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"130'391","reported_seq":6764,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352819+0000","last_change":"2024-09-17T22:43:19.150923+0000","last_active":"2024-09-17T22:43:51.352819+0000","last_peered":"2024-09-17T22:43:51.352819+0000","last_clean":"2024-09-17T22:43:51.352819+0000","last_became_active":"2024-09-17T22:41:19.449940+0000","last_became_peered":"2024-09-17T22:41:19.449940+0000","last_unstale":"2024-09-17T22:43:51.352819+0000","last_undegraded":"2024-09-17T22:43:51.352819+0000","last_fullsized":"2024-09-17T22:43:51.352819+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:57:32.786125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002665995,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6668,"num_read_kb":22143,"num_write":345,"num_write_kb":9035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"129'383","reported_seq":3221,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749001+0000","last_change":"2024-09-17T22:43:19.191965+0000","last_active":"2024-09-17T22:43:51.749001+0000","last_peered":"2024-09-17T22:43:51.749001+0000","last_clean":"2024-09-17T22:43:51.749001+0000","last_became_active":"2024-09-17T22:41:19.446287+0000","last_became_peered":"2024-09-17T22:41:19.446287+0000","last_unstale":"2024-09-17T22:43:51.749001+0000","last_undegraded":"2024-09-17T22:43:51.749001+0000","last_fullsized":"2024-09-17T22:43:51.749001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:23:52.909972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001462812,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3441,"num_read_kb":20083,"num_write":323,"num_write_kb":10930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"131'513","reported_seq":2752,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749198+0000","last_change":"2024-09-17T22:43:19.191786+0000","last_active":"2024-09-17T22:43:51.749198+0000","last_peered":"2024-09-17T22:43:51.749198+0000","last_clean":"2024-09-17T22:43:51.749198+0000","last_became_active":"2024-09-17T22:41:19.445923+0000","last_became_peered":"2024-09-17T22:41:19.445923+0000","last_unstale":"2024-09-17T22:43:51.749198+0000","last_undegraded":"2024-09-17T22:43:51.749198+0000","last_fullsized":"2024-09-17T22:43:51.749198+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:02:33.097995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001414085,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3445,"num_read_kb":15105,"num_write":513,"num_write_kb":8714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"129'562","reported_seq":22396,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.386833+0000","last_change":"2024-09-17T22:43:52.386833+0000","last_active":"2024-09-17T22:43:52.386833+0000","last_peered":"2024-09-17T22:43:52.386833+0000","last_clean":"2024-09-17T22:43:52.386833+0000","last_became_active":"2024-09-17T22:43:52.384987+0000","last_became_peered":"2024-09-17T22:43:52.384987+0000","last_unstale":"2024-09-17T22:43:52.386833+0000","last_undegraded":"2024-09-17T22:43:52.386833+0000","last_fullsized":"2024-09-17T22:43:52.386833+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:08:09.285837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016446880000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":22869,"num_read_kb":64042,"num_write":605,"num_write_kb":12652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"139'437","reported_seq":7603,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381556+0000","last_change":"2024-09-17T22:43:52.381556+0000","last_active":"2024-09-17T22:43:52.381556+0000","last_peered":"2024-09-17T22:43:52.381556+0000","last_clean":"2024-09-17T22:43:52.381556+0000","last_became_active":"2024-09-17T22:43:52.380232+0000","last_became_peered":"2024-09-17T22:43:52.380232+0000","last_unstale":"2024-09-17T22:43:52.381556+0000","last_undegraded":"2024-09-17T22:43:52.381556+0000","last_fullsized":"2024-09-17T22:43:52.381556+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:08:17.722346+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.062977108000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8763,"num_read_kb":20352,"num_write":343,"num_write_kb":4822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2823905,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"129'589","reported_seq":7857,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380494+0000","last_change":"2024-09-17T22:43:19.143328+0000","last_active":"2024-09-17T22:43:52.380494+0000","last_peered":"2024-09-17T22:43:52.380494+0000","last_clean":"2024-09-17T22:43:52.380494+0000","last_became_active":"2024-09-17T22:41:19.451631+0000","last_became_peered":"2024-09-17T22:41:19.451631+0000","last_unstale":"2024-09-17T22:43:52.380494+0000","last_undegraded":"2024-09-17T22:43:52.380494+0000","last_fullsized":"2024-09-17T22:43:52.380494+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:31.809865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090157099999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8321,"num_read_kb":19442,"num_write":736,"num_write_kb":19536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":295,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380849+0000","last_change":"2024-09-17T22:43:32.195958+0000","last_active":"2024-09-17T22:43:52.380849+0000","last_peered":"2024-09-17T22:43:52.380849+0000","last_clean":"2024-09-17T22:43:52.380849+0000","last_became_active":"2024-09-17T22:43:32.195676+0000","last_became_peered":"2024-09-17T22:43:32.195676+0000","last_unstale":"2024-09-17T22:43:52.380849+0000","last_undegraded":"2024-09-17T22:43:52.380849+0000","last_fullsized":"2024-09-17T22:43:52.380849+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:41:52.669174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"131'579","reported_seq":5338,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352815+0000","last_change":"2024-09-17T22:43:19.150601+0000","last_active":"2024-09-17T22:43:51.352815+0000","last_peered":"2024-09-17T22:43:51.352815+0000","last_clean":"2024-09-17T22:43:51.352815+0000","last_became_active":"2024-09-17T22:41:19.448966+0000","last_became_peered":"2024-09-17T22:41:19.448966+0000","last_unstale":"2024-09-17T22:43:51.352815+0000","last_undegraded":"2024-09-17T22:43:51.352815+0000","last_fullsized":"2024-09-17T22:43:51.352815+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:20:32.636962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026117269999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6258,"num_read_kb":29920,"num_write":539,"num_write_kb":6243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":292,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352861+0000","last_change":"2024-09-17T22:43:32.195712+0000","last_active":"2024-09-17T22:43:51.352861+0000","last_peered":"2024-09-17T22:43:51.352861+0000","last_clean":"2024-09-17T22:43:51.352861+0000","last_became_active":"2024-09-17T22:43:32.195451+0000","last_became_peered":"2024-09-17T22:43:32.195451+0000","last_unstale":"2024-09-17T22:43:51.352861+0000","last_undegraded":"2024-09-17T22:43:51.352861+0000","last_fullsized":"2024-09-17T22:43:51.352861+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:52.062601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"131'396","reported_seq":1773,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352783+0000","last_change":"2024-09-17T22:43:19.149604+0000","last_active":"2024-09-17T22:43:51.352783+0000","last_peered":"2024-09-17T22:43:51.352783+0000","last_clean":"2024-09-17T22:43:51.352783+0000","last_became_active":"2024-09-17T22:41:19.451637+0000","last_became_peered":"2024-09-17T22:41:19.451637+0000","last_unstale":"2024-09-17T22:43:51.352783+0000","last_undegraded":"2024-09-17T22:43:51.352783+0000","last_fullsized":"2024-09-17T22:43:51.352783+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:16.254887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001593059,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1474,"num_read_kb":9975,"num_write":408,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":293,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380692+0000","last_change":"2024-09-17T22:43:15.154296+0000","last_active":"2024-09-17T22:43:52.380692+0000","last_peered":"2024-09-17T22:43:52.380692+0000","last_clean":"2024-09-17T22:43:52.380692+0000","last_became_active":"2024-09-17T22:43:15.153956+0000","last_became_peered":"2024-09-17T22:43:15.153956+0000","last_unstale":"2024-09-17T22:43:52.380692+0000","last_undegraded":"2024-09-17T22:43:52.380692+0000","last_fullsized":"2024-09-17T22:43:52.380692+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:19.340692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"131'478","reported_seq":1431,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357944+0000","last_change":"2024-09-17T22:43:19.151215+0000","last_active":"2024-09-17T22:43:51.357944+0000","last_peered":"2024-09-17T22:43:51.357944+0000","last_clean":"2024-09-17T22:43:51.357944+0000","last_became_active":"2024-09-17T22:41:19.444893+0000","last_became_peered":"2024-09-17T22:41:19.444893+0000","last_unstale":"2024-09-17T22:43:51.357944+0000","last_undegraded":"2024-09-17T22:43:51.357944+0000","last_fullsized":"2024-09-17T22:43:51.357944+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:34:31.498807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001189308,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1718,"num_read_kb":9540,"num_write":633,"num_write_kb":7900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":286,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749190+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:43:51.749190+0000","last_peered":"2024-09-17T22:43:51.749190+0000","last_clean":"2024-09-17T22:43:51.749190+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:43:51.749190+0000","last_undegraded":"2024-09-17T22:43:51.749190+0000","last_fullsized":"2024-09-17T22:43:51.749190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"130'393","reported_seq":15619,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.379789+0000","last_change":"2024-09-17T22:43:19.187091+0000","last_active":"2024-09-17T22:43:52.379789+0000","last_peered":"2024-09-17T22:43:52.379789+0000","last_clean":"2024-09-17T22:43:52.379789+0000","last_became_active":"2024-09-17T22:41:19.451723+0000","last_became_peered":"2024-09-17T22:41:19.451723+0000","last_unstale":"2024-09-17T22:43:52.379789+0000","last_undegraded":"2024-09-17T22:43:52.379789+0000","last_fullsized":"2024-09-17T22:43:52.379789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:50:14.037236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061483000000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":15368,"num_read_kb":33652,"num_write":428,"num_write_kb":8748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"139'879","reported_seq":12126,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.391763+0000","last_change":"2024-09-17T22:43:52.391763+0000","last_active":"2024-09-17T22:43:52.391763+0000","last_peered":"2024-09-17T22:43:52.391763+0000","last_clean":"2024-09-17T22:43:52.391763+0000","last_became_active":"2024-09-17T22:43:52.388167+0000","last_became_peered":"2024-09-17T22:43:52.388167+0000","last_unstale":"2024-09-17T22:43:52.391763+0000","last_undegraded":"2024-09-17T22:43:52.391763+0000","last_fullsized":"2024-09-17T22:43:52.391763+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:55:17.606117+0000","scrub_duration":0,"objects_trimmed":42,"snaptrim_duration":0.107009245,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":15325,"num_read_kb":33693,"num_write":856,"num_write_kb":8706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":4229520,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":204,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381267+0000","last_change":"2024-09-17T22:43:42.285641+0000","last_active":"2024-09-17T22:43:52.381267+0000","last_peered":"2024-09-17T22:43:52.381267+0000","last_clean":"2024-09-17T22:43:52.381267+0000","last_became_active":"2024-09-17T22:43:42.285307+0000","last_became_peered":"2024-09-17T22:43:42.285307+0000","last_unstale":"2024-09-17T22:43:52.381267+0000","last_undegraded":"2024-09-17T22:43:52.381267+0000","last_fullsized":"2024-09-17T22:43:52.381267+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:39.203508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"130'385","reported_seq":4190,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.749152+0000","last_change":"2024-09-17T22:43:19.191554+0000","last_active":"2024-09-17T22:43:51.749152+0000","last_peered":"2024-09-17T22:43:51.749152+0000","last_clean":"2024-09-17T22:43:51.749152+0000","last_became_active":"2024-09-17T22:41:19.444314+0000","last_became_peered":"2024-09-17T22:41:19.444314+0000","last_unstale":"2024-09-17T22:43:51.749152+0000","last_undegraded":"2024-09-17T22:43:51.749152+0000","last_fullsized":"2024-09-17T22:43:51.749152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:07:00.374588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011857370000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4006,"num_read_kb":22354,"num_write":358,"num_write_kb":9353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":286,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.358312+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:43:51.358312+0000","last_peered":"2024-09-17T22:43:51.358312+0000","last_clean":"2024-09-17T22:43:51.358312+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:43:51.358312+0000","last_undegraded":"2024-09-17T22:43:51.358312+0000","last_fullsized":"2024-09-17T22:43:51.358312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"130'481","reported_seq":7455,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.358137+0000","last_change":"2024-09-17T22:43:19.150805+0000","last_active":"2024-09-17T22:43:51.358137+0000","last_peered":"2024-09-17T22:43:51.358137+0000","last_clean":"2024-09-17T22:43:51.358137+0000","last_became_active":"2024-09-17T22:41:19.449791+0000","last_became_peered":"2024-09-17T22:41:19.449791+0000","last_unstale":"2024-09-17T22:43:51.358137+0000","last_undegraded":"2024-09-17T22:43:51.358137+0000","last_fullsized":"2024-09-17T22:43:51.358137+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:48.110050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088469700000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8092,"num_read_kb":9247,"num_write":519,"num_write_kb":4605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":288,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380327+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:43:52.380327+0000","last_peered":"2024-09-17T22:43:52.380327+0000","last_clean":"2024-09-17T22:43:52.380327+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:43:52.380327+0000","last_undegraded":"2024-09-17T22:43:52.380327+0000","last_fullsized":"2024-09-17T22:43:52.380327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"131'756","reported_seq":8739,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352735+0000","last_change":"2024-09-17T22:43:19.148879+0000","last_active":"2024-09-17T22:43:51.352735+0000","last_peered":"2024-09-17T22:43:51.352735+0000","last_clean":"2024-09-17T22:43:51.352735+0000","last_became_active":"2024-09-17T22:41:19.449279+0000","last_became_peered":"2024-09-17T22:41:19.449279+0000","last_unstale":"2024-09-17T22:43:51.352735+0000","last_undegraded":"2024-09-17T22:43:51.352735+0000","last_fullsized":"2024-09-17T22:43:51.352735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:51:59.677897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089414599999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":11291,"num_read_kb":22947,"num_write":956,"num_write_kb":2860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":203,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381466+0000","last_change":"2024-09-17T22:43:42.286959+0000","last_active":"2024-09-17T22:43:52.381466+0000","last_peered":"2024-09-17T22:43:52.381466+0000","last_clean":"2024-09-17T22:43:52.381466+0000","last_became_active":"2024-09-17T22:43:42.286652+0000","last_became_peered":"2024-09-17T22:43:42.286652+0000","last_unstale":"2024-09-17T22:43:52.381466+0000","last_undegraded":"2024-09-17T22:43:52.381466+0000","last_fullsized":"2024-09-17T22:43:52.381466+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:50:39.824946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":438,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357877+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:43:51.357877+0000","last_peered":"2024-09-17T22:43:51.357877+0000","last_clean":"2024-09-17T22:43:51.357877+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:43:51.357877+0000","last_undegraded":"2024-09-17T22:43:51.357877+0000","last_fullsized":"2024-09-17T22:43:51.357877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"129'613","reported_seq":11950,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.352720+0000","last_change":"2024-09-17T22:43:19.149769+0000","last_active":"2024-09-17T22:43:51.352720+0000","last_peered":"2024-09-17T22:43:51.352720+0000","last_clean":"2024-09-17T22:43:51.352720+0000","last_became_active":"2024-09-17T22:41:19.448956+0000","last_became_peered":"2024-09-17T22:41:19.448956+0000","last_unstale":"2024-09-17T22:43:51.352720+0000","last_undegraded":"2024-09-17T22:43:51.352720+0000","last_fullsized":"2024-09-17T22:43:51.352720+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:28:45.951042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017463789999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13252,"num_read_kb":46911,"num_write":655,"num_write_kb":25175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"129'785","reported_seq":14761,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381062+0000","last_change":"2024-09-17T22:43:19.145329+0000","last_active":"2024-09-17T22:43:52.381062+0000","last_peered":"2024-09-17T22:43:52.381062+0000","last_clean":"2024-09-17T22:43:52.381062+0000","last_became_active":"2024-09-17T22:41:19.449003+0000","last_became_peered":"2024-09-17T22:41:19.449003+0000","last_unstale":"2024-09-17T22:43:52.381062+0000","last_undegraded":"2024-09-17T22:43:52.381062+0000","last_fullsized":"2024-09-17T22:43:52.381062+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:13:00.810670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910964,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":16223,"num_read_kb":43243,"num_write":1040,"num_write_kb":9843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"130'470","reported_seq":8486,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381176+0000","last_change":"2024-09-17T22:43:42.285286+0000","last_active":"2024-09-17T22:43:52.381176+0000","last_peered":"2024-09-17T22:43:52.381176+0000","last_clean":"2024-09-17T22:43:52.381176+0000","last_became_active":"2024-09-17T22:43:42.284094+0000","last_became_peered":"2024-09-17T22:43:42.284094+0000","last_unstale":"2024-09-17T22:43:52.381176+0000","last_undegraded":"2024-09-17T22:43:52.381176+0000","last_fullsized":"2024-09-17T22:43:52.381176+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:36:47.430783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001830772,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":8191,"num_read_kb":41081,"num_write":473,"num_write_kb":9769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"131'423","reported_seq":2735,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.358360+0000","last_change":"2024-09-17T22:43:19.142462+0000","last_active":"2024-09-17T22:43:51.358360+0000","last_peered":"2024-09-17T22:43:51.358360+0000","last_clean":"2024-09-17T22:43:51.358360+0000","last_became_active":"2024-09-17T22:41:19.447215+0000","last_became_peered":"2024-09-17T22:41:19.447215+0000","last_unstale":"2024-09-17T22:43:51.358360+0000","last_undegraded":"2024-09-17T22:43:51.358360+0000","last_fullsized":"2024-09-17T22:43:51.358360+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:37:48.721121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066798900000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3409,"num_read_kb":14480,"num_write":404,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"139'353","reported_seq":21688,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381548+0000","last_change":"2024-09-17T22:43:52.381548+0000","last_active":"2024-09-17T22:43:52.381548+0000","last_peered":"2024-09-17T22:43:52.381548+0000","last_clean":"2024-09-17T22:43:52.381548+0000","last_became_active":"2024-09-17T22:43:52.380211+0000","last_became_peered":"2024-09-17T22:43:52.380211+0000","last_unstale":"2024-09-17T22:43:52.381548+0000","last_undegraded":"2024-09-17T22:43:52.381548+0000","last_fullsized":"2024-09-17T22:43:52.381548+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:43:11.256179+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.064717442,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":21473,"num_read_kb":63694,"num_write":340,"num_write_kb":14940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":1192963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"129'486","reported_seq":2031,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380602+0000","last_change":"2024-09-17T22:43:42.283323+0000","last_active":"2024-09-17T22:43:52.380602+0000","last_peered":"2024-09-17T22:43:52.380602+0000","last_clean":"2024-09-17T22:43:52.380602+0000","last_became_active":"2024-09-17T22:43:42.282252+0000","last_became_peered":"2024-09-17T22:43:42.282252+0000","last_unstale":"2024-09-17T22:43:52.380602+0000","last_undegraded":"2024-09-17T22:43:52.380602+0000","last_fullsized":"2024-09-17T22:43:52.380602+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:31.315273+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.002173033,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2216,"num_read_kb":10204,"num_write":574,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"139'372","reported_seq":8839,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.379984+0000","last_change":"2024-09-17T22:43:35.323373+0000","last_active":"2024-09-17T22:43:52.379984+0000","last_peered":"2024-09-17T22:43:52.379984+0000","last_clean":"2024-09-17T22:43:52.379984+0000","last_became_active":"2024-09-17T22:43:35.224141+0000","last_became_peered":"2024-09-17T22:43:35.224141+0000","last_unstale":"2024-09-17T22:43:52.379984+0000","last_undegraded":"2024-09-17T22:43:52.379984+0000","last_fullsized":"2024-09-17T22:43:52.379984+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:01:02.683062+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.098411079999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8804,"num_read_kb":23106,"num_write":291,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":2514411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"139'353","reported_seq":1860,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.390365+0000","last_change":"2024-09-17T22:43:52.390365+0000","last_active":"2024-09-17T22:43:52.390365+0000","last_peered":"2024-09-17T22:43:52.390365+0000","last_clean":"2024-09-17T22:43:52.390365+0000","last_became_active":"2024-09-17T22:43:52.388960+0000","last_became_peered":"2024-09-17T22:43:52.388960+0000","last_unstale":"2024-09-17T22:43:52.390365+0000","last_undegraded":"2024-09-17T22:43:52.390365+0000","last_fullsized":"2024-09-17T22:43:52.390365+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:13:54.534247+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.086636568999999997,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1634,"num_read_kb":5501,"num_write":380,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1563844,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"129'349","reported_seq":2170,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357762+0000","last_change":"2024-09-17T22:43:19.144272+0000","last_active":"2024-09-17T22:43:51.357762+0000","last_peered":"2024-09-17T22:43:51.357762+0000","last_clean":"2024-09-17T22:43:51.357762+0000","last_became_active":"2024-09-17T22:41:19.447419+0000","last_became_peered":"2024-09-17T22:41:19.447419+0000","last_unstale":"2024-09-17T22:43:51.357762+0000","last_undegraded":"2024-09-17T22:43:51.357762+0000","last_fullsized":"2024-09-17T22:43:51.357762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:48.842752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00132408,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3110,"num_read_kb":8489,"num_write":286,"num_write_kb":3161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"129'616","reported_seq":3681,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380951+0000","last_change":"2024-09-17T22:43:19.144657+0000","last_active":"2024-09-17T22:43:52.380951+0000","last_peered":"2024-09-17T22:43:52.380951+0000","last_clean":"2024-09-17T22:43:52.380951+0000","last_became_active":"2024-09-17T22:41:19.454151+0000","last_became_peered":"2024-09-17T22:41:19.454151+0000","last_unstale":"2024-09-17T22:43:52.380951+0000","last_undegraded":"2024-09-17T22:43:52.380951+0000","last_fullsized":"2024-09-17T22:43:52.380951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:39:51.641693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001718979,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5266,"num_read_kb":13546,"num_write":658,"num_write_kb":3731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"130'910","reported_seq":11717,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-17T22:43:51.357917+0000","last_change":"2024-09-17T22:43:35.218645+0000","last_active":"2024-09-17T22:43:51.357917+0000","last_peered":"2024-09-17T22:43:51.357917+0000","last_clean":"2024-09-17T22:43:51.357917+0000","last_became_active":"2024-09-17T22:43:35.217045+0000","last_became_peered":"2024-09-17T22:43:35.217045+0000","last_unstale":"2024-09-17T22:43:51.357917+0000","last_undegraded":"2024-09-17T22:43:51.357917+0000","last_fullsized":"2024-09-17T22:43:51.357917+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:20.169030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001497735,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14642,"num_read_kb":44074,"num_write":1142,"num_write_kb":22161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"139'502","reported_seq":2630,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381373+0000","last_change":"2024-09-17T22:43:35.301824+0000","last_active":"2024-09-17T22:43:52.381373+0000","last_peered":"2024-09-17T22:43:52.381373+0000","last_clean":"2024-09-17T22:43:52.381373+0000","last_became_active":"2024-09-17T22:43:35.217551+0000","last_became_peered":"2024-09-17T22:43:35.217551+0000","last_unstale":"2024-09-17T22:43:52.381373+0000","last_undegraded":"2024-09-17T22:43:52.381373+0000","last_fullsized":"2024-09-17T22:43:52.381373+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:10.204124+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.082900606000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":2503,"num_read_kb":8800,"num_write":617,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":291987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":492,"num_read":268105,"num_read_kb":802236,"num_write":18141,"num_write_kb":293703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":30918130,"num_keys_recovered":86,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11879,"internal_metadata":0},"log_size":16633,"ondisk_log_size":16633,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542183,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28908,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342932,"statfs":{"total":96636764160,"available":96607162368,"internally_reserved":0,"allocated":1921024,"data_stored":1492409,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":135,"seq":579820584967,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34748,"kb_used_data":1828,"kb_used_omap":21,"kb_used_meta":32874,"kb_avail":94337092,"statfs":{"total":96636764160,"available":96601182208,"internally_reserved":0,"allocated":1871872,"data_stored":907980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22295,"internal_metadata":33663209},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542181,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41320,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330520,"statfs":{"total":96636764160,"available":96594452480,"internally_reserved":0,"allocated":1847296,"data_stored":896064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542181,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28848,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342992,"statfs":{"total":96636764160,"available":96607223808,"internally_reserved":0,"allocated":1863680,"data_stored":902041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542181,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46948,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324892,"statfs":{"total":96636764160,"available":96588689408,"internally_reserved":0,"allocated":1908736,"data_stored":1486451,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28848,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342992,"statfs":{"total":96636764160,"available":96607223808,"internally_reserved":0,"allocated":1859584,"data_stored":902022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55232,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316608,"statfs":{"total":96636764160,"available":96580206592,"internally_reserved":0,"allocated":1875968,"data_stored":907999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51068,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320772,"statfs":{"total":96636764160,"available":96584470528,"internally_reserved":0,"allocated":1871872,"data_stored":907980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11879,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:43:58.318 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T22:43:58.318 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:43:58.596 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:43:58.597 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:43:58.608 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2024-09-17T22:43:58.340223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":500,"num_object_clones":0,"num_object_copies":1000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":500,"num_whiteouts":492,"num_read":268221,"num_read_kb":802336,"num_write":18302,"num_write_kb":296121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":30918130,"num_keys_recovered":86,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16753,"ondisk_log_size":16753,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":316008,"kb_used_data":14752,"kb_used_omap":32,"kb_used_meta":301023,"kb_avail":754658712,"statfs":{"total":773094113280,"available":772770521088,"internally_reserved":0,"allocated":15106048,"data_stored":8444652,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":33450,"internal_metadata":308247894},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-114,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.994969"},"pg_stats":[{"pgid":"3.1f","version":"131'860","reported_seq":7312,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375133+0000","last_change":"2024-09-17T22:43:19.191421+0000","last_active":"2024-09-17T22:43:52.375133+0000","last_peered":"2024-09-17T22:43:52.375133+0000","last_clean":"2024-09-17T22:43:52.375133+0000","last_became_active":"2024-09-17T22:41:19.443658+0000","last_became_peered":"2024-09-17T22:41:19.443658+0000","last_unstale":"2024-09-17T22:43:52.375133+0000","last_undegraded":"2024-09-17T22:43:52.375133+0000","last_fullsized":"2024-09-17T22:43:52.375133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:58.423329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001073428,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8816,"num_read_kb":14906,"num_write":1102,"num_write_kb":4548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"130'786","reported_seq":7424,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375065+0000","last_change":"2024-09-17T22:43:19.190807+0000","last_active":"2024-09-17T22:43:52.375065+0000","last_peered":"2024-09-17T22:43:52.375065+0000","last_clean":"2024-09-17T22:43:52.375065+0000","last_became_active":"2024-09-17T22:41:19.449357+0000","last_became_peered":"2024-09-17T22:41:19.449357+0000","last_unstale":"2024-09-17T22:43:52.375065+0000","last_undegraded":"2024-09-17T22:43:52.375065+0000","last_fullsized":"2024-09-17T22:43:52.375065+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:20.300342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051110000000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":8865,"num_read_kb":26537,"num_write":1126,"num_write_kb":10958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"131'435","reported_seq":2382,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.382642+0000","last_change":"2024-09-17T22:43:19.144579+0000","last_active":"2024-09-17T22:43:52.382642+0000","last_peered":"2024-09-17T22:43:52.382642+0000","last_clean":"2024-09-17T22:43:52.382642+0000","last_became_active":"2024-09-17T22:42:58.002822+0000","last_became_peered":"2024-09-17T22:42:58.002822+0000","last_unstale":"2024-09-17T22:43:52.382642+0000","last_undegraded":"2024-09-17T22:43:52.382642+0000","last_fullsized":"2024-09-17T22:43:52.382642+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:18:33.730453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022657670000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":17641,"num_read_kb":48444,"num_write":377,"num_write_kb":8987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":7308630,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"131'494","reported_seq":2828,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.382520+0000","last_change":"2024-09-17T22:43:19.146655+0000","last_active":"2024-09-17T22:43:52.382520+0000","last_peered":"2024-09-17T22:43:52.382520+0000","last_clean":"2024-09-17T22:43:52.382520+0000","last_became_active":"2024-09-17T22:42:34.310411+0000","last_became_peered":"2024-09-17T22:42:34.310411+0000","last_unstale":"2024-09-17T22:43:52.382520+0000","last_undegraded":"2024-09-17T22:43:52.382520+0000","last_fullsized":"2024-09-17T22:43:52.382520+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:34.577953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039390110000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":2615,"num_read_kb":14215,"num_write":599,"num_write_kb":12172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":7352772,"num_keys_recovered":74,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"139'244","reported_seq":8118,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375352+0000","last_change":"2024-09-17T22:43:35.308387+0000","last_active":"2024-09-17T22:43:52.375352+0000","last_peered":"2024-09-17T22:43:52.375352+0000","last_clean":"2024-09-17T22:43:52.375352+0000","last_became_active":"2024-09-17T22:43:35.215952+0000","last_became_peered":"2024-09-17T22:43:35.215952+0000","last_unstale":"2024-09-17T22:43:52.375352+0000","last_undegraded":"2024-09-17T22:43:52.375352+0000","last_fullsized":"2024-09-17T22:43:52.375352+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:45.376055+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.091482261999999995,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7315,"num_read_kb":22822,"num_write":186,"num_write_kb":6215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":3640098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"131'360","reported_seq":5243,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.383089+0000","last_change":"2024-09-17T22:43:19.145987+0000","last_active":"2024-09-17T22:43:52.383089+0000","last_peered":"2024-09-17T22:43:52.383089+0000","last_clean":"2024-09-17T22:43:52.383089+0000","last_became_active":"2024-09-17T22:41:19.446541+0000","last_became_peered":"2024-09-17T22:41:19.446541+0000","last_unstale":"2024-09-17T22:43:52.383089+0000","last_undegraded":"2024-09-17T22:43:52.383089+0000","last_fullsized":"2024-09-17T22:43:52.383089+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:12:51.399521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035192800000000001,"stat_sum":{"num_bytes":19,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":5091,"num_read_kb":19688,"num_write":329,"num_write_kb":4179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"130'391","reported_seq":6766,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375206+0000","last_change":"2024-09-17T22:43:19.150923+0000","last_active":"2024-09-17T22:43:52.375206+0000","last_peered":"2024-09-17T22:43:52.375206+0000","last_clean":"2024-09-17T22:43:52.375206+0000","last_became_active":"2024-09-17T22:41:19.449940+0000","last_became_peered":"2024-09-17T22:41:19.449940+0000","last_unstale":"2024-09-17T22:43:52.375206+0000","last_undegraded":"2024-09-17T22:43:52.375206+0000","last_fullsized":"2024-09-17T22:43:52.375206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:57:32.786125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002665995,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6668,"num_read_kb":22143,"num_write":345,"num_write_kb":9035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"129'383","reported_seq":3223,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375004+0000","last_change":"2024-09-17T22:43:19.191965+0000","last_active":"2024-09-17T22:43:52.375004+0000","last_peered":"2024-09-17T22:43:52.375004+0000","last_clean":"2024-09-17T22:43:52.375004+0000","last_became_active":"2024-09-17T22:41:19.446287+0000","last_became_peered":"2024-09-17T22:41:19.446287+0000","last_unstale":"2024-09-17T22:43:52.375004+0000","last_undegraded":"2024-09-17T22:43:52.375004+0000","last_fullsized":"2024-09-17T22:43:52.375004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:23:52.909972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001462812,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3441,"num_read_kb":20083,"num_write":323,"num_write_kb":10930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"131'513","reported_seq":2754,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375231+0000","last_change":"2024-09-17T22:43:19.191786+0000","last_active":"2024-09-17T22:43:52.375231+0000","last_peered":"2024-09-17T22:43:52.375231+0000","last_clean":"2024-09-17T22:43:52.375231+0000","last_became_active":"2024-09-17T22:41:19.445923+0000","last_became_peered":"2024-09-17T22:41:19.445923+0000","last_unstale":"2024-09-17T22:43:52.375231+0000","last_undegraded":"2024-09-17T22:43:52.375231+0000","last_fullsized":"2024-09-17T22:43:52.375231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:02:33.097995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001414085,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3445,"num_read_kb":15105,"num_write":513,"num_write_kb":8714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"129'562","reported_seq":22396,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.386833+0000","last_change":"2024-09-17T22:43:52.386833+0000","last_active":"2024-09-17T22:43:52.386833+0000","last_peered":"2024-09-17T22:43:52.386833+0000","last_clean":"2024-09-17T22:43:52.386833+0000","last_became_active":"2024-09-17T22:43:52.384987+0000","last_became_peered":"2024-09-17T22:43:52.384987+0000","last_unstale":"2024-09-17T22:43:52.386833+0000","last_undegraded":"2024-09-17T22:43:52.386833+0000","last_fullsized":"2024-09-17T22:43:52.386833+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:08:09.285837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016446880000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":22869,"num_read_kb":64042,"num_write":605,"num_write_kb":12652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"139'437","reported_seq":7603,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381556+0000","last_change":"2024-09-17T22:43:52.381556+0000","last_active":"2024-09-17T22:43:52.381556+0000","last_peered":"2024-09-17T22:43:52.381556+0000","last_clean":"2024-09-17T22:43:52.381556+0000","last_became_active":"2024-09-17T22:43:52.380232+0000","last_became_peered":"2024-09-17T22:43:52.380232+0000","last_unstale":"2024-09-17T22:43:52.381556+0000","last_undegraded":"2024-09-17T22:43:52.381556+0000","last_fullsized":"2024-09-17T22:43:52.381556+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:08:17.722346+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.062977108000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8763,"num_read_kb":20352,"num_write":343,"num_write_kb":4822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2823905,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"129'589","reported_seq":7857,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380494+0000","last_change":"2024-09-17T22:43:19.143328+0000","last_active":"2024-09-17T22:43:52.380494+0000","last_peered":"2024-09-17T22:43:52.380494+0000","last_clean":"2024-09-17T22:43:52.380494+0000","last_became_active":"2024-09-17T22:41:19.451631+0000","last_became_peered":"2024-09-17T22:41:19.451631+0000","last_unstale":"2024-09-17T22:43:52.380494+0000","last_undegraded":"2024-09-17T22:43:52.380494+0000","last_fullsized":"2024-09-17T22:43:52.380494+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:31.809865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090157099999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8321,"num_read_kb":19442,"num_write":736,"num_write_kb":19536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":295,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380849+0000","last_change":"2024-09-17T22:43:32.195958+0000","last_active":"2024-09-17T22:43:52.380849+0000","last_peered":"2024-09-17T22:43:52.380849+0000","last_clean":"2024-09-17T22:43:52.380849+0000","last_became_active":"2024-09-17T22:43:32.195676+0000","last_became_peered":"2024-09-17T22:43:32.195676+0000","last_unstale":"2024-09-17T22:43:52.380849+0000","last_undegraded":"2024-09-17T22:43:52.380849+0000","last_fullsized":"2024-09-17T22:43:52.380849+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:41:52.669174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"131'579","reported_seq":5340,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375188+0000","last_change":"2024-09-17T22:43:19.150601+0000","last_active":"2024-09-17T22:43:52.375188+0000","last_peered":"2024-09-17T22:43:52.375188+0000","last_clean":"2024-09-17T22:43:52.375188+0000","last_became_active":"2024-09-17T22:41:19.448966+0000","last_became_peered":"2024-09-17T22:41:19.448966+0000","last_unstale":"2024-09-17T22:43:52.375188+0000","last_undegraded":"2024-09-17T22:43:52.375188+0000","last_fullsized":"2024-09-17T22:43:52.375188+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:20:32.636962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026117269999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6258,"num_read_kb":29920,"num_write":539,"num_write_kb":6243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":294,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375302+0000","last_change":"2024-09-17T22:43:32.195712+0000","last_active":"2024-09-17T22:43:52.375302+0000","last_peered":"2024-09-17T22:43:52.375302+0000","last_clean":"2024-09-17T22:43:52.375302+0000","last_became_active":"2024-09-17T22:43:32.195451+0000","last_became_peered":"2024-09-17T22:43:32.195451+0000","last_unstale":"2024-09-17T22:43:52.375302+0000","last_undegraded":"2024-09-17T22:43:52.375302+0000","last_fullsized":"2024-09-17T22:43:52.375302+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:52.062601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"131'396","reported_seq":1775,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375303+0000","last_change":"2024-09-17T22:43:19.149604+0000","last_active":"2024-09-17T22:43:52.375303+0000","last_peered":"2024-09-17T22:43:52.375303+0000","last_clean":"2024-09-17T22:43:52.375303+0000","last_became_active":"2024-09-17T22:41:19.451637+0000","last_became_peered":"2024-09-17T22:41:19.451637+0000","last_unstale":"2024-09-17T22:43:52.375303+0000","last_undegraded":"2024-09-17T22:43:52.375303+0000","last_fullsized":"2024-09-17T22:43:52.375303+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:16.254887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001593059,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1474,"num_read_kb":9975,"num_write":408,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":293,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380692+0000","last_change":"2024-09-17T22:43:15.154296+0000","last_active":"2024-09-17T22:43:52.380692+0000","last_peered":"2024-09-17T22:43:52.380692+0000","last_clean":"2024-09-17T22:43:52.380692+0000","last_became_active":"2024-09-17T22:43:15.153956+0000","last_became_peered":"2024-09-17T22:43:15.153956+0000","last_unstale":"2024-09-17T22:43:52.380692+0000","last_undegraded":"2024-09-17T22:43:52.380692+0000","last_fullsized":"2024-09-17T22:43:52.380692+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:19.340692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"131'478","reported_seq":1433,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.378790+0000","last_change":"2024-09-17T22:43:19.151215+0000","last_active":"2024-09-17T22:43:52.378790+0000","last_peered":"2024-09-17T22:43:52.378790+0000","last_clean":"2024-09-17T22:43:52.378790+0000","last_became_active":"2024-09-17T22:41:19.444893+0000","last_became_peered":"2024-09-17T22:41:19.444893+0000","last_unstale":"2024-09-17T22:43:52.378790+0000","last_undegraded":"2024-09-17T22:43:52.378790+0000","last_fullsized":"2024-09-17T22:43:52.378790+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:34:31.498807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001189308,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1718,"num_read_kb":9540,"num_write":633,"num_write_kb":7900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":288,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375044+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:43:52.375044+0000","last_peered":"2024-09-17T22:43:52.375044+0000","last_clean":"2024-09-17T22:43:52.375044+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:43:52.375044+0000","last_undegraded":"2024-09-17T22:43:52.375044+0000","last_fullsized":"2024-09-17T22:43:52.375044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"130'393","reported_seq":15619,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.379789+0000","last_change":"2024-09-17T22:43:19.187091+0000","last_active":"2024-09-17T22:43:52.379789+0000","last_peered":"2024-09-17T22:43:52.379789+0000","last_clean":"2024-09-17T22:43:52.379789+0000","last_became_active":"2024-09-17T22:41:19.451723+0000","last_became_peered":"2024-09-17T22:41:19.451723+0000","last_unstale":"2024-09-17T22:43:52.379789+0000","last_undegraded":"2024-09-17T22:43:52.379789+0000","last_fullsized":"2024-09-17T22:43:52.379789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:50:14.037236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061483000000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":15368,"num_read_kb":33652,"num_write":428,"num_write_kb":8748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"139'879","reported_seq":12126,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.391763+0000","last_change":"2024-09-17T22:43:52.391763+0000","last_active":"2024-09-17T22:43:52.391763+0000","last_peered":"2024-09-17T22:43:52.391763+0000","last_clean":"2024-09-17T22:43:52.391763+0000","last_became_active":"2024-09-17T22:43:52.388167+0000","last_became_peered":"2024-09-17T22:43:52.388167+0000","last_unstale":"2024-09-17T22:43:52.391763+0000","last_undegraded":"2024-09-17T22:43:52.391763+0000","last_fullsized":"2024-09-17T22:43:52.391763+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:55:17.606117+0000","scrub_duration":0,"objects_trimmed":42,"snaptrim_duration":0.107009245,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":15325,"num_read_kb":33693,"num_write":856,"num_write_kb":8706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":4229520,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":204,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381267+0000","last_change":"2024-09-17T22:43:42.285641+0000","last_active":"2024-09-17T22:43:52.381267+0000","last_peered":"2024-09-17T22:43:52.381267+0000","last_clean":"2024-09-17T22:43:52.381267+0000","last_became_active":"2024-09-17T22:43:42.285307+0000","last_became_peered":"2024-09-17T22:43:42.285307+0000","last_unstale":"2024-09-17T22:43:52.381267+0000","last_undegraded":"2024-09-17T22:43:52.381267+0000","last_fullsized":"2024-09-17T22:43:52.381267+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:39.203508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"130'385","reported_seq":4192,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.374835+0000","last_change":"2024-09-17T22:43:19.191554+0000","last_active":"2024-09-17T22:43:52.374835+0000","last_peered":"2024-09-17T22:43:52.374835+0000","last_clean":"2024-09-17T22:43:52.374835+0000","last_became_active":"2024-09-17T22:41:19.444314+0000","last_became_peered":"2024-09-17T22:41:19.444314+0000","last_unstale":"2024-09-17T22:43:52.374835+0000","last_undegraded":"2024-09-17T22:43:52.374835+0000","last_fullsized":"2024-09-17T22:43:52.374835+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:07:00.374588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011857370000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4006,"num_read_kb":22354,"num_write":358,"num_write_kb":9353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":288,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.378473+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:43:52.378473+0000","last_peered":"2024-09-17T22:43:52.378473+0000","last_clean":"2024-09-17T22:43:52.378473+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:43:52.378473+0000","last_undegraded":"2024-09-17T22:43:52.378473+0000","last_fullsized":"2024-09-17T22:43:52.378473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"130'481","reported_seq":7457,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.378586+0000","last_change":"2024-09-17T22:43:19.150805+0000","last_active":"2024-09-17T22:43:52.378586+0000","last_peered":"2024-09-17T22:43:52.378586+0000","last_clean":"2024-09-17T22:43:52.378586+0000","last_became_active":"2024-09-17T22:41:19.449791+0000","last_became_peered":"2024-09-17T22:41:19.449791+0000","last_unstale":"2024-09-17T22:43:52.378586+0000","last_undegraded":"2024-09-17T22:43:52.378586+0000","last_fullsized":"2024-09-17T22:43:52.378586+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:48.110050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088469700000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8092,"num_read_kb":9247,"num_write":519,"num_write_kb":4605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":288,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380327+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:43:52.380327+0000","last_peered":"2024-09-17T22:43:52.380327+0000","last_clean":"2024-09-17T22:43:52.380327+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:43:52.380327+0000","last_undegraded":"2024-09-17T22:43:52.380327+0000","last_fullsized":"2024-09-17T22:43:52.380327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"131'756","reported_seq":8741,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375213+0000","last_change":"2024-09-17T22:43:19.148879+0000","last_active":"2024-09-17T22:43:52.375213+0000","last_peered":"2024-09-17T22:43:52.375213+0000","last_clean":"2024-09-17T22:43:52.375213+0000","last_became_active":"2024-09-17T22:41:19.449279+0000","last_became_peered":"2024-09-17T22:41:19.449279+0000","last_unstale":"2024-09-17T22:43:52.375213+0000","last_undegraded":"2024-09-17T22:43:52.375213+0000","last_fullsized":"2024-09-17T22:43:52.375213+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:51:59.677897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089414599999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":11291,"num_read_kb":22947,"num_write":956,"num_write_kb":2860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":203,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381466+0000","last_change":"2024-09-17T22:43:42.286959+0000","last_active":"2024-09-17T22:43:52.381466+0000","last_peered":"2024-09-17T22:43:52.381466+0000","last_clean":"2024-09-17T22:43:52.381466+0000","last_became_active":"2024-09-17T22:43:42.286652+0000","last_became_peered":"2024-09-17T22:43:42.286652+0000","last_unstale":"2024-09-17T22:43:52.381466+0000","last_undegraded":"2024-09-17T22:43:52.381466+0000","last_fullsized":"2024-09-17T22:43:52.381466+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:50:39.824946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":440,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.382907+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:43:52.382907+0000","last_peered":"2024-09-17T22:43:52.382907+0000","last_clean":"2024-09-17T22:43:52.382907+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:43:52.382907+0000","last_undegraded":"2024-09-17T22:43:52.382907+0000","last_fullsized":"2024-09-17T22:43:52.382907+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"129'613","reported_seq":11952,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375136+0000","last_change":"2024-09-17T22:43:19.149769+0000","last_active":"2024-09-17T22:43:52.375136+0000","last_peered":"2024-09-17T22:43:52.375136+0000","last_clean":"2024-09-17T22:43:52.375136+0000","last_became_active":"2024-09-17T22:41:19.448956+0000","last_became_peered":"2024-09-17T22:41:19.448956+0000","last_unstale":"2024-09-17T22:43:52.375136+0000","last_undegraded":"2024-09-17T22:43:52.375136+0000","last_fullsized":"2024-09-17T22:43:52.375136+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:28:45.951042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017463789999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13252,"num_read_kb":46911,"num_write":655,"num_write_kb":25175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"129'785","reported_seq":14761,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381062+0000","last_change":"2024-09-17T22:43:19.145329+0000","last_active":"2024-09-17T22:43:52.381062+0000","last_peered":"2024-09-17T22:43:52.381062+0000","last_clean":"2024-09-17T22:43:52.381062+0000","last_became_active":"2024-09-17T22:41:19.449003+0000","last_became_peered":"2024-09-17T22:41:19.449003+0000","last_unstale":"2024-09-17T22:43:52.381062+0000","last_undegraded":"2024-09-17T22:43:52.381062+0000","last_fullsized":"2024-09-17T22:43:52.381062+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:13:00.810670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910964,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":16223,"num_read_kb":43243,"num_write":1040,"num_write_kb":9843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"130'470","reported_seq":8486,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381176+0000","last_change":"2024-09-17T22:43:42.285286+0000","last_active":"2024-09-17T22:43:52.381176+0000","last_peered":"2024-09-17T22:43:52.381176+0000","last_clean":"2024-09-17T22:43:52.381176+0000","last_became_active":"2024-09-17T22:43:42.284094+0000","last_became_peered":"2024-09-17T22:43:42.284094+0000","last_unstale":"2024-09-17T22:43:52.381176+0000","last_undegraded":"2024-09-17T22:43:52.381176+0000","last_fullsized":"2024-09-17T22:43:52.381176+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:36:47.430783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001830772,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":8191,"num_read_kb":41081,"num_write":473,"num_write_kb":9769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"131'423","reported_seq":2737,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.378529+0000","last_change":"2024-09-17T22:43:19.142462+0000","last_active":"2024-09-17T22:43:52.378529+0000","last_peered":"2024-09-17T22:43:52.378529+0000","last_clean":"2024-09-17T22:43:52.378529+0000","last_became_active":"2024-09-17T22:41:19.447215+0000","last_became_peered":"2024-09-17T22:41:19.447215+0000","last_unstale":"2024-09-17T22:43:52.378529+0000","last_undegraded":"2024-09-17T22:43:52.378529+0000","last_fullsized":"2024-09-17T22:43:52.378529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:37:48.721121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066798900000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3409,"num_read_kb":14480,"num_write":404,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"139'353","reported_seq":21688,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381548+0000","last_change":"2024-09-17T22:43:52.381548+0000","last_active":"2024-09-17T22:43:52.381548+0000","last_peered":"2024-09-17T22:43:52.381548+0000","last_clean":"2024-09-17T22:43:52.381548+0000","last_became_active":"2024-09-17T22:43:52.380211+0000","last_became_peered":"2024-09-17T22:43:52.380211+0000","last_unstale":"2024-09-17T22:43:52.381548+0000","last_undegraded":"2024-09-17T22:43:52.381548+0000","last_fullsized":"2024-09-17T22:43:52.381548+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:43:11.256179+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.064717442,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":21473,"num_read_kb":63694,"num_write":340,"num_write_kb":14940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":1192963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"129'486","reported_seq":2031,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380602+0000","last_change":"2024-09-17T22:43:42.283323+0000","last_active":"2024-09-17T22:43:52.380602+0000","last_peered":"2024-09-17T22:43:52.380602+0000","last_clean":"2024-09-17T22:43:52.380602+0000","last_became_active":"2024-09-17T22:43:42.282252+0000","last_became_peered":"2024-09-17T22:43:42.282252+0000","last_unstale":"2024-09-17T22:43:52.380602+0000","last_undegraded":"2024-09-17T22:43:52.380602+0000","last_fullsized":"2024-09-17T22:43:52.380602+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:31.315273+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.002173033,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2216,"num_read_kb":10204,"num_write":574,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"139'372","reported_seq":8839,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.379984+0000","last_change":"2024-09-17T22:43:35.323373+0000","last_active":"2024-09-17T22:43:52.379984+0000","last_peered":"2024-09-17T22:43:52.379984+0000","last_clean":"2024-09-17T22:43:52.379984+0000","last_became_active":"2024-09-17T22:43:35.224141+0000","last_became_peered":"2024-09-17T22:43:35.224141+0000","last_unstale":"2024-09-17T22:43:52.379984+0000","last_undegraded":"2024-09-17T22:43:52.379984+0000","last_fullsized":"2024-09-17T22:43:52.379984+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:01:02.683062+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.098411079999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8804,"num_read_kb":23106,"num_write":291,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":2514411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"139'353","reported_seq":1860,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.390365+0000","last_change":"2024-09-17T22:43:52.390365+0000","last_active":"2024-09-17T22:43:52.390365+0000","last_peered":"2024-09-17T22:43:52.390365+0000","last_clean":"2024-09-17T22:43:52.390365+0000","last_became_active":"2024-09-17T22:43:52.388960+0000","last_became_peered":"2024-09-17T22:43:52.388960+0000","last_unstale":"2024-09-17T22:43:52.390365+0000","last_undegraded":"2024-09-17T22:43:52.390365+0000","last_fullsized":"2024-09-17T22:43:52.390365+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:13:54.534247+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.086636568999999997,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1634,"num_read_kb":5501,"num_write":380,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1563844,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"129'349","reported_seq":2172,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.382811+0000","last_change":"2024-09-17T22:43:19.144272+0000","last_active":"2024-09-17T22:43:52.382811+0000","last_peered":"2024-09-17T22:43:52.382811+0000","last_clean":"2024-09-17T22:43:52.382811+0000","last_became_active":"2024-09-17T22:41:19.447419+0000","last_became_peered":"2024-09-17T22:41:19.447419+0000","last_unstale":"2024-09-17T22:43:52.382811+0000","last_undegraded":"2024-09-17T22:43:52.382811+0000","last_fullsized":"2024-09-17T22:43:52.382811+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:48.842752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00132408,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3110,"num_read_kb":8489,"num_write":286,"num_write_kb":3161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"129'616","reported_seq":3681,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380951+0000","last_change":"2024-09-17T22:43:19.144657+0000","last_active":"2024-09-17T22:43:52.380951+0000","last_peered":"2024-09-17T22:43:52.380951+0000","last_clean":"2024-09-17T22:43:52.380951+0000","last_became_active":"2024-09-17T22:41:19.454151+0000","last_became_peered":"2024-09-17T22:41:19.454151+0000","last_unstale":"2024-09-17T22:43:52.380951+0000","last_undegraded":"2024-09-17T22:43:52.380951+0000","last_fullsized":"2024-09-17T22:43:52.380951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:39:51.641693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001718979,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5266,"num_read_kb":13546,"num_write":658,"num_write_kb":3731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"130'910","reported_seq":11719,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.378726+0000","last_change":"2024-09-17T22:43:35.218645+0000","last_active":"2024-09-17T22:43:52.378726+0000","last_peered":"2024-09-17T22:43:52.378726+0000","last_clean":"2024-09-17T22:43:52.378726+0000","last_became_active":"2024-09-17T22:43:35.217045+0000","last_became_peered":"2024-09-17T22:43:35.217045+0000","last_unstale":"2024-09-17T22:43:52.378726+0000","last_undegraded":"2024-09-17T22:43:52.378726+0000","last_fullsized":"2024-09-17T22:43:52.378726+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:20.169030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001497735,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14642,"num_read_kb":44074,"num_write":1142,"num_write_kb":22161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"139'502","reported_seq":2630,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381373+0000","last_change":"2024-09-17T22:43:35.301824+0000","last_active":"2024-09-17T22:43:52.381373+0000","last_peered":"2024-09-17T22:43:52.381373+0000","last_clean":"2024-09-17T22:43:52.381373+0000","last_became_active":"2024-09-17T22:43:35.217551+0000","last_became_peered":"2024-09-17T22:43:35.217551+0000","last_unstale":"2024-09-17T22:43:52.381373+0000","last_undegraded":"2024-09-17T22:43:52.381373+0000","last_fullsized":"2024-09-17T22:43:52.381373+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:10.204124+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.082900606000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":2503,"num_read_kb":8800,"num_write":617,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":291987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":492,"num_read":268105,"num_read_kb":802236,"num_write":18141,"num_write_kb":293703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":30918130,"num_keys_recovered":86,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11879,"internal_metadata":0},"log_size":16633,"ondisk_log_size":16633,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28924,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342916,"statfs":{"total":96636764160,"available":96607145984,"internally_reserved":0,"allocated":1933312,"data_stored":1498367,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":135,"seq":579820584967,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34748,"kb_used_data":1828,"kb_used_omap":21,"kb_used_meta":32874,"kb_avail":94337092,"statfs":{"total":96636764160,"available":96601182208,"internally_reserved":0,"allocated":1871872,"data_stored":907980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22295,"internal_metadata":33663209},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542182,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41340,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330500,"statfs":{"total":96636764160,"available":96594432000,"internally_reserved":0,"allocated":1871872,"data_stored":907980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542182,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28864,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342976,"statfs":{"total":96636764160,"available":96607207424,"internally_reserved":0,"allocated":1875968,"data_stored":907999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542182,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46968,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324872,"statfs":{"total":96636764160,"available":96588668928,"internally_reserved":0,"allocated":1933312,"data_stored":1498367,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542182,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28864,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342976,"statfs":{"total":96636764160,"available":96607207424,"internally_reserved":0,"allocated":1871872,"data_stored":907980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55232,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316608,"statfs":{"total":96636764160,"available":96580206592,"internally_reserved":0,"allocated":1875968,"data_stored":907999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51068,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320772,"statfs":{"total":96636764160,"available":96584470528,"internally_reserved":0,"allocated":1871872,"data_stored":907980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11879,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:43:58.610 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T22:43:58.885 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:43:58.886 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":156,"fsid":"551c5ed8-e790-485a-bc95-a9578a29275c","created":"2024-09-17T22:40:50.189230+0000","modified":"2024-09-17T22:43:52.355227+0000","last_up_change":"2024-09-17T22:43:31.180198+0000","last_in_change":"2024-09-17T22:40:52.260526+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T22:41:02.338356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T22:41:08.097861+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T22:41:17.501542+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":40,"snap_epoch":131,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3bd5c11f-e7c2-4968-839a-2799c00d93a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":50,"up_from":135,"up_thru":155,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":4198232340},{"type":"v1","addr":"172.21.15.115:6825","nonce":4198232340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":4198232340},{"type":"v1","addr":"172.21.15.115:6827","nonce":4198232340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":4198232340},{"type":"v1","addr":"172.21.15.115:6831","nonce":4198232340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":4198232340},{"type":"v1","addr":"172.21.15.115:6829","nonce":4198232340}]},"public_addr":"172.21.15.115:6825/4198232340","cluster_addr":"172.21.15.115:6827/4198232340","heartbeat_back_addr":"172.21.15.115:6831/4198232340","heartbeat_front_addr":"172.21.15.115:6829/4198232340","state":["exists","up"]},{"osd":1,"uuid":"02918203-cf2e-49fc-b994-743b5d81b4a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6817","nonce":1941780390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6819","nonce":1941780390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6823","nonce":1941780390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":1941780390},{"type":"v1","addr":"172.21.15.115:6821","nonce":1941780390}]},"public_addr":"172.21.15.115:6817/1941780390","cluster_addr":"172.21.15.115:6819/1941780390","heartbeat_back_addr":"172.21.15.115:6823/1941780390","heartbeat_front_addr":"172.21.15.115:6821/1941780390","state":["exists","up"]},{"osd":2,"uuid":"22ee34ef-522e-4121-85b7-38bbafeca5de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6809","nonce":689014698}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6811","nonce":689014698}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6815","nonce":689014698}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":689014698},{"type":"v1","addr":"172.21.15.115:6813","nonce":689014698}]},"public_addr":"172.21.15.115:6809/689014698","cluster_addr":"172.21.15.115:6811/689014698","heartbeat_back_addr":"172.21.15.115:6815/689014698","heartbeat_front_addr":"172.21.15.115:6813/689014698","state":["exists","up"]},{"osd":3,"uuid":"5d464921-5696-43c5-b89d-430a54ddefb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6801","nonce":453192890}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6803","nonce":453192890}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6807","nonce":453192890}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":453192890},{"type":"v1","addr":"172.21.15.115:6805","nonce":453192890}]},"public_addr":"172.21.15.115:6801/453192890","cluster_addr":"172.21.15.115:6803/453192890","heartbeat_back_addr":"172.21.15.115:6807/453192890","heartbeat_front_addr":"172.21.15.115:6805/453192890","state":["exists","up"]},{"osd":4,"uuid":"4cb6973a-63ae-4f48-af94-37610aee3c06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6801","nonce":380388860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6803","nonce":380388860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6807","nonce":380388860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":380388860},{"type":"v1","addr":"172.21.15.129:6805","nonce":380388860}]},"public_addr":"172.21.15.129:6801/380388860","cluster_addr":"172.21.15.129:6803/380388860","heartbeat_back_addr":"172.21.15.129:6807/380388860","heartbeat_front_addr":"172.21.15.129:6805/380388860","state":["exists","up"]},{"osd":5,"uuid":"ef9fe11b-b719-48f4-9c6c-e1b1b355acdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6809","nonce":943488949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6811","nonce":943488949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6815","nonce":943488949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":943488949},{"type":"v1","addr":"172.21.15.129:6813","nonce":943488949}]},"public_addr":"172.21.15.129:6809/943488949","cluster_addr":"172.21.15.129:6811/943488949","heartbeat_back_addr":"172.21.15.129:6815/943488949","heartbeat_front_addr":"172.21.15.129:6813/943488949","state":["exists","up"]},{"osd":6,"uuid":"e4045aec-22c1-401b-bfbf-64f115bc32af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6817","nonce":1873501077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6819","nonce":1873501077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6823","nonce":1873501077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":1873501077},{"type":"v1","addr":"172.21.15.129:6821","nonce":1873501077}]},"public_addr":"172.21.15.129:6817/1873501077","cluster_addr":"172.21.15.129:6819/1873501077","heartbeat_back_addr":"172.21.15.129:6823/1873501077","heartbeat_front_addr":"172.21.15.129:6821/1873501077","state":["exists","up"]},{"osd":7,"uuid":"4a05a5d7-8281-4572-a9d0-6095ee5e168b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6825","nonce":3609357666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6827","nonce":3609357666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6831","nonce":3609357666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3609357666},{"type":"v1","addr":"172.21.15.129:6829","nonce":3609357666}]},"public_addr":"172.21.15.129:6825/3609357666","cluster_addr":"172.21.15.129:6827/3609357666","heartbeat_back_addr":"172.21.15.129:6831/3609357666","heartbeat_front_addr":"172.21.15.129:6829/3609357666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:59.351758+0000","dead_epoch":51},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:59.058915+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.922383+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.802379+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.881681+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.616649+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:58.589555+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T22:40:59.047783+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.1c","osds":[1,3]},{"pgid":"3.1d","osds":[1,6]}],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":4,"to":7}]},{"pgid":"3.12","mappings":[{"from":3,"to":0}]},{"pgid":"3.16","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T22:43:59.900 INFO:tasks.ceph:Scrubbing osd.0 2024-09-17T22:43:59.900 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-17T22:44:00.036 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:44:00.036 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-17T22:44:00.036 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:44:00.047 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-17T22:44:00.322 INFO:teuthology.orchestra.run.smithi115.stderr:instructed osd(s) 0 to deep-scrub 2024-09-17T22:44:00.335 INFO:tasks.ceph:Scrubbing osd.1 2024-09-17T22:44:00.335 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-17T22:44:00.461 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:44:00.461 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T22:44:00.461 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:44:00.472 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-17T22:44:00.747 INFO:teuthology.orchestra.run.smithi115.stderr:instructed osd(s) 1 to deep-scrub 2024-09-17T22:44:00.761 INFO:tasks.ceph:Scrubbing osd.2 2024-09-17T22:44:00.761 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-17T22:44:00.882 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:44:00.883 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T22:44:00.883 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:44:00.894 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-17T22:44:01.169 INFO:teuthology.orchestra.run.smithi115.stderr:instructed osd(s) 2 to deep-scrub 2024-09-17T22:44:01.182 INFO:tasks.ceph:Scrubbing osd.3 2024-09-17T22:44:01.182 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-17T22:44:01.303 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:44:01.304 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T22:44:01.304 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:44:01.314 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-17T22:44:01.593 INFO:teuthology.orchestra.run.smithi115.stderr:instructed osd(s) 3 to deep-scrub 2024-09-17T22:44:01.605 INFO:tasks.ceph:Scrubbing osd.4 2024-09-17T22:44:01.606 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-17T22:44:01.730 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:44:01.730 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T22:44:01.730 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:44:01.740 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-17T22:44:02.016 INFO:teuthology.orchestra.run.smithi115.stderr:instructed osd(s) 4 to deep-scrub 2024-09-17T22:44:02.029 INFO:tasks.ceph:Scrubbing osd.5 2024-09-17T22:44:02.029 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-17T22:44:02.152 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:44:02.152 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T22:44:02.152 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:44:02.163 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-17T22:44:02.441 INFO:teuthology.orchestra.run.smithi115.stderr:instructed osd(s) 5 to deep-scrub 2024-09-17T22:44:02.455 INFO:tasks.ceph:Scrubbing osd.6 2024-09-17T22:44:02.455 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-17T22:44:02.582 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:44:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T22:44:02.583 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:44:02.593 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-17T22:44:02.875 INFO:teuthology.orchestra.run.smithi115.stderr:instructed osd(s) 6 to deep-scrub 2024-09-17T22:44:02.889 INFO:tasks.ceph:Scrubbing osd.7 2024-09-17T22:44:02.889 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-17T22:44:03.012 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-17T22:44:03.012 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T22:44:03.012 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-17T22:44:03.022 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-17T22:44:03.305 INFO:teuthology.orchestra.run.smithi115.stderr:instructed osd(s) 7 to deep-scrub 2024-09-17T22:44:03.318 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:44:03.599 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:44:03.599 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:44:03.611 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-17T22:44:02.341070+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":500,"num_object_clones":0,"num_object_copies":1000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":500,"num_whiteouts":492,"num_read":268221,"num_read_kb":802336,"num_write":18302,"num_write_kb":296121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":30918130,"num_keys_recovered":86,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16753,"ondisk_log_size":16753,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":316000,"kb_used_data":14752,"kb_used_omap":32,"kb_used_meta":301023,"kb_avail":754658720,"statfs":{"total":773094113280,"available":772770529280,"internally_reserved":0,"allocated":15106048,"data_stored":8444652,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":33450,"internal_metadata":308247894},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002169"},"pg_stats":[{"pgid":"3.1f","version":"131'860","reported_seq":7312,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375133+0000","last_change":"2024-09-17T22:43:19.191421+0000","last_active":"2024-09-17T22:43:52.375133+0000","last_peered":"2024-09-17T22:43:52.375133+0000","last_clean":"2024-09-17T22:43:52.375133+0000","last_became_active":"2024-09-17T22:41:19.443658+0000","last_became_peered":"2024-09-17T22:41:19.443658+0000","last_unstale":"2024-09-17T22:43:52.375133+0000","last_undegraded":"2024-09-17T22:43:52.375133+0000","last_fullsized":"2024-09-17T22:43:52.375133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:11:58.423329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001073428,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8816,"num_read_kb":14906,"num_write":1102,"num_write_kb":4548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"130'786","reported_seq":7424,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375065+0000","last_change":"2024-09-17T22:43:19.190807+0000","last_active":"2024-09-17T22:43:52.375065+0000","last_peered":"2024-09-17T22:43:52.375065+0000","last_clean":"2024-09-17T22:43:52.375065+0000","last_became_active":"2024-09-17T22:41:19.449357+0000","last_became_peered":"2024-09-17T22:41:19.449357+0000","last_unstale":"2024-09-17T22:43:52.375065+0000","last_undegraded":"2024-09-17T22:43:52.375065+0000","last_fullsized":"2024-09-17T22:43:52.375065+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:50:20.300342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051110000000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":8865,"num_read_kb":26537,"num_write":1126,"num_write_kb":10958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"131'435","reported_seq":2382,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.382642+0000","last_change":"2024-09-17T22:43:19.144579+0000","last_active":"2024-09-17T22:43:52.382642+0000","last_peered":"2024-09-17T22:43:52.382642+0000","last_clean":"2024-09-17T22:43:52.382642+0000","last_became_active":"2024-09-17T22:42:58.002822+0000","last_became_peered":"2024-09-17T22:42:58.002822+0000","last_unstale":"2024-09-17T22:43:52.382642+0000","last_undegraded":"2024-09-17T22:43:52.382642+0000","last_fullsized":"2024-09-17T22:43:52.382642+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:18:33.730453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022657670000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":17641,"num_read_kb":48444,"num_write":377,"num_write_kb":8987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":7308630,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"131'494","reported_seq":2828,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.382520+0000","last_change":"2024-09-17T22:43:19.146655+0000","last_active":"2024-09-17T22:43:52.382520+0000","last_peered":"2024-09-17T22:43:52.382520+0000","last_clean":"2024-09-17T22:43:52.382520+0000","last_became_active":"2024-09-17T22:42:34.310411+0000","last_became_peered":"2024-09-17T22:42:34.310411+0000","last_unstale":"2024-09-17T22:43:52.382520+0000","last_undegraded":"2024-09-17T22:43:52.382520+0000","last_fullsized":"2024-09-17T22:43:52.382520+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:34.577953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039390110000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":2615,"num_read_kb":14215,"num_write":599,"num_write_kb":12172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":7352772,"num_keys_recovered":74,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"139'244","reported_seq":8118,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375352+0000","last_change":"2024-09-17T22:43:35.308387+0000","last_active":"2024-09-17T22:43:52.375352+0000","last_peered":"2024-09-17T22:43:52.375352+0000","last_clean":"2024-09-17T22:43:52.375352+0000","last_became_active":"2024-09-17T22:43:35.215952+0000","last_became_peered":"2024-09-17T22:43:35.215952+0000","last_unstale":"2024-09-17T22:43:52.375352+0000","last_undegraded":"2024-09-17T22:43:52.375352+0000","last_fullsized":"2024-09-17T22:43:52.375352+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:45.376055+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.091482261999999995,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7315,"num_read_kb":22822,"num_write":186,"num_write_kb":6215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":3640098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"131'360","reported_seq":5243,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.383089+0000","last_change":"2024-09-17T22:43:19.145987+0000","last_active":"2024-09-17T22:43:52.383089+0000","last_peered":"2024-09-17T22:43:52.383089+0000","last_clean":"2024-09-17T22:43:52.383089+0000","last_became_active":"2024-09-17T22:41:19.446541+0000","last_became_peered":"2024-09-17T22:41:19.446541+0000","last_unstale":"2024-09-17T22:43:52.383089+0000","last_undegraded":"2024-09-17T22:43:52.383089+0000","last_fullsized":"2024-09-17T22:43:52.383089+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:12:51.399521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035192800000000001,"stat_sum":{"num_bytes":19,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":5091,"num_read_kb":19688,"num_write":329,"num_write_kb":4179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"130'391","reported_seq":6766,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375206+0000","last_change":"2024-09-17T22:43:19.150923+0000","last_active":"2024-09-17T22:43:52.375206+0000","last_peered":"2024-09-17T22:43:52.375206+0000","last_clean":"2024-09-17T22:43:52.375206+0000","last_became_active":"2024-09-17T22:41:19.449940+0000","last_became_peered":"2024-09-17T22:41:19.449940+0000","last_unstale":"2024-09-17T22:43:52.375206+0000","last_undegraded":"2024-09-17T22:43:52.375206+0000","last_fullsized":"2024-09-17T22:43:52.375206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:57:32.786125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002665995,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6668,"num_read_kb":22143,"num_write":345,"num_write_kb":9035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"129'383","reported_seq":3223,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375004+0000","last_change":"2024-09-17T22:43:19.191965+0000","last_active":"2024-09-17T22:43:52.375004+0000","last_peered":"2024-09-17T22:43:52.375004+0000","last_clean":"2024-09-17T22:43:52.375004+0000","last_became_active":"2024-09-17T22:41:19.446287+0000","last_became_peered":"2024-09-17T22:41:19.446287+0000","last_unstale":"2024-09-17T22:43:52.375004+0000","last_undegraded":"2024-09-17T22:43:52.375004+0000","last_fullsized":"2024-09-17T22:43:52.375004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:23:52.909972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001462812,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3441,"num_read_kb":20083,"num_write":323,"num_write_kb":10930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"131'513","reported_seq":2754,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375231+0000","last_change":"2024-09-17T22:43:19.191786+0000","last_active":"2024-09-17T22:43:52.375231+0000","last_peered":"2024-09-17T22:43:52.375231+0000","last_clean":"2024-09-17T22:43:52.375231+0000","last_became_active":"2024-09-17T22:41:19.445923+0000","last_became_peered":"2024-09-17T22:41:19.445923+0000","last_unstale":"2024-09-17T22:43:52.375231+0000","last_undegraded":"2024-09-17T22:43:52.375231+0000","last_fullsized":"2024-09-17T22:43:52.375231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:02:33.097995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001414085,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3445,"num_read_kb":15105,"num_write":513,"num_write_kb":8714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"129'562","reported_seq":22396,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.386833+0000","last_change":"2024-09-17T22:43:52.386833+0000","last_active":"2024-09-17T22:43:52.386833+0000","last_peered":"2024-09-17T22:43:52.386833+0000","last_clean":"2024-09-17T22:43:52.386833+0000","last_became_active":"2024-09-17T22:43:52.384987+0000","last_became_peered":"2024-09-17T22:43:52.384987+0000","last_unstale":"2024-09-17T22:43:52.386833+0000","last_undegraded":"2024-09-17T22:43:52.386833+0000","last_fullsized":"2024-09-17T22:43:52.386833+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:08:09.285837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016446880000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":22869,"num_read_kb":64042,"num_write":605,"num_write_kb":12652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"139'437","reported_seq":7603,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381556+0000","last_change":"2024-09-17T22:43:52.381556+0000","last_active":"2024-09-17T22:43:52.381556+0000","last_peered":"2024-09-17T22:43:52.381556+0000","last_clean":"2024-09-17T22:43:52.381556+0000","last_became_active":"2024-09-17T22:43:52.380232+0000","last_became_peered":"2024-09-17T22:43:52.380232+0000","last_unstale":"2024-09-17T22:43:52.381556+0000","last_undegraded":"2024-09-17T22:43:52.381556+0000","last_fullsized":"2024-09-17T22:43:52.381556+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:08:17.722346+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.062977108000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8763,"num_read_kb":20352,"num_write":343,"num_write_kb":4822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2823905,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"129'589","reported_seq":7857,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380494+0000","last_change":"2024-09-17T22:43:19.143328+0000","last_active":"2024-09-17T22:43:52.380494+0000","last_peered":"2024-09-17T22:43:52.380494+0000","last_clean":"2024-09-17T22:43:52.380494+0000","last_became_active":"2024-09-17T22:41:19.451631+0000","last_became_peered":"2024-09-17T22:41:19.451631+0000","last_unstale":"2024-09-17T22:43:52.380494+0000","last_undegraded":"2024-09-17T22:43:52.380494+0000","last_fullsized":"2024-09-17T22:43:52.380494+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:31.809865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090157099999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8321,"num_read_kb":19442,"num_write":736,"num_write_kb":19536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":295,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380849+0000","last_change":"2024-09-17T22:43:32.195958+0000","last_active":"2024-09-17T22:43:52.380849+0000","last_peered":"2024-09-17T22:43:52.380849+0000","last_clean":"2024-09-17T22:43:52.380849+0000","last_became_active":"2024-09-17T22:43:32.195676+0000","last_became_peered":"2024-09-17T22:43:32.195676+0000","last_unstale":"2024-09-17T22:43:52.380849+0000","last_undegraded":"2024-09-17T22:43:52.380849+0000","last_fullsized":"2024-09-17T22:43:52.380849+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:41:52.669174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"131'579","reported_seq":5340,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375188+0000","last_change":"2024-09-17T22:43:19.150601+0000","last_active":"2024-09-17T22:43:52.375188+0000","last_peered":"2024-09-17T22:43:52.375188+0000","last_clean":"2024-09-17T22:43:52.375188+0000","last_became_active":"2024-09-17T22:41:19.448966+0000","last_became_peered":"2024-09-17T22:41:19.448966+0000","last_unstale":"2024-09-17T22:43:52.375188+0000","last_undegraded":"2024-09-17T22:43:52.375188+0000","last_fullsized":"2024-09-17T22:43:52.375188+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:20:32.636962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026117269999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6258,"num_read_kb":29920,"num_write":539,"num_write_kb":6243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":294,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375302+0000","last_change":"2024-09-17T22:43:32.195712+0000","last_active":"2024-09-17T22:43:52.375302+0000","last_peered":"2024-09-17T22:43:52.375302+0000","last_clean":"2024-09-17T22:43:52.375302+0000","last_became_active":"2024-09-17T22:43:32.195451+0000","last_became_peered":"2024-09-17T22:43:32.195451+0000","last_unstale":"2024-09-17T22:43:52.375302+0000","last_undegraded":"2024-09-17T22:43:52.375302+0000","last_fullsized":"2024-09-17T22:43:52.375302+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:52.062601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"131'396","reported_seq":1775,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375303+0000","last_change":"2024-09-17T22:43:19.149604+0000","last_active":"2024-09-17T22:43:52.375303+0000","last_peered":"2024-09-17T22:43:52.375303+0000","last_clean":"2024-09-17T22:43:52.375303+0000","last_became_active":"2024-09-17T22:41:19.451637+0000","last_became_peered":"2024-09-17T22:41:19.451637+0000","last_unstale":"2024-09-17T22:43:52.375303+0000","last_undegraded":"2024-09-17T22:43:52.375303+0000","last_fullsized":"2024-09-17T22:43:52.375303+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:16.254887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001593059,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1474,"num_read_kb":9975,"num_write":408,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":293,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380692+0000","last_change":"2024-09-17T22:43:15.154296+0000","last_active":"2024-09-17T22:43:52.380692+0000","last_peered":"2024-09-17T22:43:52.380692+0000","last_clean":"2024-09-17T22:43:52.380692+0000","last_became_active":"2024-09-17T22:43:15.153956+0000","last_became_peered":"2024-09-17T22:43:15.153956+0000","last_unstale":"2024-09-17T22:43:52.380692+0000","last_undegraded":"2024-09-17T22:43:52.380692+0000","last_fullsized":"2024-09-17T22:43:52.380692+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:49:19.340692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"131'478","reported_seq":1433,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.378790+0000","last_change":"2024-09-17T22:43:19.151215+0000","last_active":"2024-09-17T22:43:52.378790+0000","last_peered":"2024-09-17T22:43:52.378790+0000","last_clean":"2024-09-17T22:43:52.378790+0000","last_became_active":"2024-09-17T22:41:19.444893+0000","last_became_peered":"2024-09-17T22:41:19.444893+0000","last_unstale":"2024-09-17T22:43:52.378790+0000","last_undegraded":"2024-09-17T22:43:52.378790+0000","last_fullsized":"2024-09-17T22:43:52.378790+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:34:31.498807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001189308,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1718,"num_read_kb":9540,"num_write":633,"num_write_kb":7900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":288,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375044+0000","last_change":"2024-09-17T22:41:11.510563+0000","last_active":"2024-09-17T22:43:52.375044+0000","last_peered":"2024-09-17T22:43:52.375044+0000","last_clean":"2024-09-17T22:43:52.375044+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:43:52.375044+0000","last_undegraded":"2024-09-17T22:43:52.375044+0000","last_fullsized":"2024-09-17T22:43:52.375044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:01:03.124897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"130'393","reported_seq":15619,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.379789+0000","last_change":"2024-09-17T22:43:19.187091+0000","last_active":"2024-09-17T22:43:52.379789+0000","last_peered":"2024-09-17T22:43:52.379789+0000","last_clean":"2024-09-17T22:43:52.379789+0000","last_became_active":"2024-09-17T22:41:19.451723+0000","last_became_peered":"2024-09-17T22:41:19.451723+0000","last_unstale":"2024-09-17T22:43:52.379789+0000","last_undegraded":"2024-09-17T22:43:52.379789+0000","last_fullsized":"2024-09-17T22:43:52.379789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:50:14.037236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061483000000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":15368,"num_read_kb":33652,"num_write":428,"num_write_kb":8748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"139'879","reported_seq":12126,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.391763+0000","last_change":"2024-09-17T22:43:52.391763+0000","last_active":"2024-09-17T22:43:52.391763+0000","last_peered":"2024-09-17T22:43:52.391763+0000","last_clean":"2024-09-17T22:43:52.391763+0000","last_became_active":"2024-09-17T22:43:52.388167+0000","last_became_peered":"2024-09-17T22:43:52.388167+0000","last_unstale":"2024-09-17T22:43:52.391763+0000","last_undegraded":"2024-09-17T22:43:52.391763+0000","last_fullsized":"2024-09-17T22:43:52.391763+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:55:17.606117+0000","scrub_duration":0,"objects_trimmed":42,"snaptrim_duration":0.107009245,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":15325,"num_read_kb":33693,"num_write":856,"num_write_kb":8706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":4229520,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":204,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381267+0000","last_change":"2024-09-17T22:43:42.285641+0000","last_active":"2024-09-17T22:43:52.381267+0000","last_peered":"2024-09-17T22:43:52.381267+0000","last_clean":"2024-09-17T22:43:52.381267+0000","last_became_active":"2024-09-17T22:43:42.285307+0000","last_became_peered":"2024-09-17T22:43:42.285307+0000","last_unstale":"2024-09-17T22:43:52.381267+0000","last_undegraded":"2024-09-17T22:43:52.381267+0000","last_fullsized":"2024-09-17T22:43:52.381267+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:39.203508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"130'385","reported_seq":4192,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.374835+0000","last_change":"2024-09-17T22:43:19.191554+0000","last_active":"2024-09-17T22:43:52.374835+0000","last_peered":"2024-09-17T22:43:52.374835+0000","last_clean":"2024-09-17T22:43:52.374835+0000","last_became_active":"2024-09-17T22:41:19.444314+0000","last_became_peered":"2024-09-17T22:41:19.444314+0000","last_unstale":"2024-09-17T22:43:52.374835+0000","last_undegraded":"2024-09-17T22:43:52.374835+0000","last_fullsized":"2024-09-17T22:43:52.374835+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:07:00.374588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011857370000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4006,"num_read_kb":22354,"num_write":358,"num_write_kb":9353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":288,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.378473+0000","last_change":"2024-09-17T22:41:11.619738+0000","last_active":"2024-09-17T22:43:52.378473+0000","last_peered":"2024-09-17T22:43:52.378473+0000","last_clean":"2024-09-17T22:43:52.378473+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:43:52.378473+0000","last_undegraded":"2024-09-17T22:43:52.378473+0000","last_fullsized":"2024-09-17T22:43:52.378473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:42.594463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"130'481","reported_seq":7457,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.378586+0000","last_change":"2024-09-17T22:43:19.150805+0000","last_active":"2024-09-17T22:43:52.378586+0000","last_peered":"2024-09-17T22:43:52.378586+0000","last_clean":"2024-09-17T22:43:52.378586+0000","last_became_active":"2024-09-17T22:41:19.449791+0000","last_became_peered":"2024-09-17T22:41:19.449791+0000","last_unstale":"2024-09-17T22:43:52.378586+0000","last_undegraded":"2024-09-17T22:43:52.378586+0000","last_fullsized":"2024-09-17T22:43:52.378586+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:49:48.110050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088469700000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8092,"num_read_kb":9247,"num_write":519,"num_write_kb":4605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":288,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380327+0000","last_change":"2024-09-17T22:41:11.397413+0000","last_active":"2024-09-17T22:43:52.380327+0000","last_peered":"2024-09-17T22:43:52.380327+0000","last_clean":"2024-09-17T22:43:52.380327+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:43:52.380327+0000","last_undegraded":"2024-09-17T22:43:52.380327+0000","last_fullsized":"2024-09-17T22:43:52.380327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:39.880395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"131'756","reported_seq":8741,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375213+0000","last_change":"2024-09-17T22:43:19.148879+0000","last_active":"2024-09-17T22:43:52.375213+0000","last_peered":"2024-09-17T22:43:52.375213+0000","last_clean":"2024-09-17T22:43:52.375213+0000","last_became_active":"2024-09-17T22:41:19.449279+0000","last_became_peered":"2024-09-17T22:41:19.449279+0000","last_unstale":"2024-09-17T22:43:52.375213+0000","last_undegraded":"2024-09-17T22:43:52.375213+0000","last_fullsized":"2024-09-17T22:43:52.375213+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:51:59.677897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089414599999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":11291,"num_read_kb":22947,"num_write":956,"num_write_kb":2860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":203,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381466+0000","last_change":"2024-09-17T22:43:42.286959+0000","last_active":"2024-09-17T22:43:52.381466+0000","last_peered":"2024-09-17T22:43:52.381466+0000","last_clean":"2024-09-17T22:43:52.381466+0000","last_became_active":"2024-09-17T22:43:42.286652+0000","last_became_peered":"2024-09-17T22:43:42.286652+0000","last_unstale":"2024-09-17T22:43:52.381466+0000","last_undegraded":"2024-09-17T22:43:52.381466+0000","last_fullsized":"2024-09-17T22:43:52.381466+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:08.370723+0000","last_clean_scrub_stamp":"2024-09-17T22:41:08.370723+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:50:39.824946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":440,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.382907+0000","last_change":"2024-09-17T22:41:03.607227+0000","last_active":"2024-09-17T22:43:52.382907+0000","last_peered":"2024-09-17T22:43:52.382907+0000","last_clean":"2024-09-17T22:43:52.382907+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:43:52.382907+0000","last_undegraded":"2024-09-17T22:43:52.382907+0000","last_fullsized":"2024-09-17T22:43:52.382907+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:02.338825+0000","last_clean_scrub_stamp":"2024-09-17T22:41:02.338825+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:58:30.194323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"129'613","reported_seq":11952,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.375136+0000","last_change":"2024-09-17T22:43:19.149769+0000","last_active":"2024-09-17T22:43:52.375136+0000","last_peered":"2024-09-17T22:43:52.375136+0000","last_clean":"2024-09-17T22:43:52.375136+0000","last_became_active":"2024-09-17T22:41:19.448956+0000","last_became_peered":"2024-09-17T22:41:19.448956+0000","last_unstale":"2024-09-17T22:43:52.375136+0000","last_undegraded":"2024-09-17T22:43:52.375136+0000","last_fullsized":"2024-09-17T22:43:52.375136+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:28:45.951042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017463789999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13252,"num_read_kb":46911,"num_write":655,"num_write_kb":25175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"129'785","reported_seq":14761,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381062+0000","last_change":"2024-09-17T22:43:19.145329+0000","last_active":"2024-09-17T22:43:52.381062+0000","last_peered":"2024-09-17T22:43:52.381062+0000","last_clean":"2024-09-17T22:43:52.381062+0000","last_became_active":"2024-09-17T22:41:19.449003+0000","last_became_peered":"2024-09-17T22:41:19.449003+0000","last_unstale":"2024-09-17T22:43:52.381062+0000","last_undegraded":"2024-09-17T22:43:52.381062+0000","last_fullsized":"2024-09-17T22:43:52.381062+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:13:00.810670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001910964,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":16223,"num_read_kb":43243,"num_write":1040,"num_write_kb":9843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"130'470","reported_seq":8486,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381176+0000","last_change":"2024-09-17T22:43:42.285286+0000","last_active":"2024-09-17T22:43:52.381176+0000","last_peered":"2024-09-17T22:43:52.381176+0000","last_clean":"2024-09-17T22:43:52.381176+0000","last_became_active":"2024-09-17T22:43:42.284094+0000","last_became_peered":"2024-09-17T22:43:42.284094+0000","last_unstale":"2024-09-17T22:43:52.381176+0000","last_undegraded":"2024-09-17T22:43:52.381176+0000","last_fullsized":"2024-09-17T22:43:52.381176+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:36:47.430783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001830772,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":8191,"num_read_kb":41081,"num_write":473,"num_write_kb":9769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"131'423","reported_seq":2737,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.378529+0000","last_change":"2024-09-17T22:43:19.142462+0000","last_active":"2024-09-17T22:43:52.378529+0000","last_peered":"2024-09-17T22:43:52.378529+0000","last_clean":"2024-09-17T22:43:52.378529+0000","last_became_active":"2024-09-17T22:41:19.447215+0000","last_became_peered":"2024-09-17T22:41:19.447215+0000","last_unstale":"2024-09-17T22:43:52.378529+0000","last_undegraded":"2024-09-17T22:43:52.378529+0000","last_fullsized":"2024-09-17T22:43:52.378529+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:37:48.721121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066798900000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3409,"num_read_kb":14480,"num_write":404,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"139'353","reported_seq":21688,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381548+0000","last_change":"2024-09-17T22:43:52.381548+0000","last_active":"2024-09-17T22:43:52.381548+0000","last_peered":"2024-09-17T22:43:52.381548+0000","last_clean":"2024-09-17T22:43:52.381548+0000","last_became_active":"2024-09-17T22:43:52.380211+0000","last_became_peered":"2024-09-17T22:43:52.380211+0000","last_unstale":"2024-09-17T22:43:52.381548+0000","last_undegraded":"2024-09-17T22:43:52.381548+0000","last_fullsized":"2024-09-17T22:43:52.381548+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:43:11.256179+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.064717442,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":21473,"num_read_kb":63694,"num_write":340,"num_write_kb":14940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":1192963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"129'486","reported_seq":2031,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380602+0000","last_change":"2024-09-17T22:43:42.283323+0000","last_active":"2024-09-17T22:43:52.380602+0000","last_peered":"2024-09-17T22:43:52.380602+0000","last_clean":"2024-09-17T22:43:52.380602+0000","last_became_active":"2024-09-17T22:43:42.282252+0000","last_became_peered":"2024-09-17T22:43:42.282252+0000","last_unstale":"2024-09-17T22:43:52.380602+0000","last_undegraded":"2024-09-17T22:43:52.380602+0000","last_fullsized":"2024-09-17T22:43:52.380602+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:31.315273+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.002173033,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2216,"num_read_kb":10204,"num_write":574,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"139'372","reported_seq":8839,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.379984+0000","last_change":"2024-09-17T22:43:35.323373+0000","last_active":"2024-09-17T22:43:52.379984+0000","last_peered":"2024-09-17T22:43:52.379984+0000","last_clean":"2024-09-17T22:43:52.379984+0000","last_became_active":"2024-09-17T22:43:35.224141+0000","last_became_peered":"2024-09-17T22:43:35.224141+0000","last_unstale":"2024-09-17T22:43:52.379984+0000","last_undegraded":"2024-09-17T22:43:52.379984+0000","last_fullsized":"2024-09-17T22:43:52.379984+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:01:02.683062+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.098411079999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8804,"num_read_kb":23106,"num_write":291,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":2514411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"139'353","reported_seq":1860,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.390365+0000","last_change":"2024-09-17T22:43:52.390365+0000","last_active":"2024-09-17T22:43:52.390365+0000","last_peered":"2024-09-17T22:43:52.390365+0000","last_clean":"2024-09-17T22:43:52.390365+0000","last_became_active":"2024-09-17T22:43:52.388960+0000","last_became_peered":"2024-09-17T22:43:52.388960+0000","last_unstale":"2024-09-17T22:43:52.390365+0000","last_undegraded":"2024-09-17T22:43:52.390365+0000","last_fullsized":"2024-09-17T22:43:52.390365+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:13:54.534247+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.086636568999999997,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1634,"num_read_kb":5501,"num_write":380,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1563844,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"129'349","reported_seq":2172,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.382811+0000","last_change":"2024-09-17T22:43:19.144272+0000","last_active":"2024-09-17T22:43:52.382811+0000","last_peered":"2024-09-17T22:43:52.382811+0000","last_clean":"2024-09-17T22:43:52.382811+0000","last_became_active":"2024-09-17T22:41:19.447419+0000","last_became_peered":"2024-09-17T22:41:19.447419+0000","last_unstale":"2024-09-17T22:43:52.382811+0000","last_undegraded":"2024-09-17T22:43:52.382811+0000","last_fullsized":"2024-09-17T22:43:52.382811+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:30:48.842752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00132408,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3110,"num_read_kb":8489,"num_write":286,"num_write_kb":3161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"129'616","reported_seq":3681,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.380951+0000","last_change":"2024-09-17T22:43:19.144657+0000","last_active":"2024-09-17T22:43:52.380951+0000","last_peered":"2024-09-17T22:43:52.380951+0000","last_clean":"2024-09-17T22:43:52.380951+0000","last_became_active":"2024-09-17T22:41:19.454151+0000","last_became_peered":"2024-09-17T22:41:19.454151+0000","last_unstale":"2024-09-17T22:43:52.380951+0000","last_undegraded":"2024-09-17T22:43:52.380951+0000","last_fullsized":"2024-09-17T22:43:52.380951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:39:51.641693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001718979,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5266,"num_read_kb":13546,"num_write":658,"num_write_kb":3731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"130'910","reported_seq":11719,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.378726+0000","last_change":"2024-09-17T22:43:35.218645+0000","last_active":"2024-09-17T22:43:52.378726+0000","last_peered":"2024-09-17T22:43:52.378726+0000","last_clean":"2024-09-17T22:43:52.378726+0000","last_became_active":"2024-09-17T22:43:35.217045+0000","last_became_peered":"2024-09-17T22:43:35.217045+0000","last_unstale":"2024-09-17T22:43:52.378726+0000","last_undegraded":"2024-09-17T22:43:52.378726+0000","last_fullsized":"2024-09-17T22:43:52.378726+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:12:20.169030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001497735,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14642,"num_read_kb":44074,"num_write":1142,"num_write_kb":22161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"139'502","reported_seq":2630,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:43:52.381373+0000","last_change":"2024-09-17T22:43:35.301824+0000","last_active":"2024-09-17T22:43:52.381373+0000","last_peered":"2024-09-17T22:43:52.381373+0000","last_clean":"2024-09-17T22:43:52.381373+0000","last_became_active":"2024-09-17T22:43:35.217551+0000","last_became_peered":"2024-09-17T22:43:35.217551+0000","last_unstale":"2024-09-17T22:43:52.381373+0000","last_undegraded":"2024-09-17T22:43:52.381373+0000","last_fullsized":"2024-09-17T22:43:52.381373+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:41:18.423563+0000","last_clean_scrub_stamp":"2024-09-17T22:41:18.423563+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:10.204124+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.082900606000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":2503,"num_read_kb":8800,"num_write":617,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":291987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":492,"num_read":268105,"num_read_kb":802236,"num_write":18141,"num_write_kb":293703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":30918130,"num_keys_recovered":86,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11879,"internal_metadata":0},"log_size":16633,"ondisk_log_size":16633,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28924,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342916,"statfs":{"total":96636764160,"available":96607145984,"internally_reserved":0,"allocated":1933312,"data_stored":1498367,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":135,"seq":579820584968,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34748,"kb_used_data":1828,"kb_used_omap":21,"kb_used_meta":32874,"kb_avail":94337092,"statfs":{"total":96636764160,"available":96601182208,"internally_reserved":0,"allocated":1871872,"data_stored":907980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22295,"internal_metadata":33663209},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542183,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41332,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330508,"statfs":{"total":96636764160,"available":96594440192,"internally_reserved":0,"allocated":1871872,"data_stored":907980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542183,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28864,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342976,"statfs":{"total":96636764160,"available":96607207424,"internally_reserved":0,"allocated":1875968,"data_stored":907999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542183,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46968,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324872,"statfs":{"total":96636764160,"available":96588668928,"internally_reserved":0,"allocated":1933312,"data_stored":1498367,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542183,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28864,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342976,"statfs":{"total":96636764160,"available":96607207424,"internally_reserved":0,"allocated":1871872,"data_stored":907980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55232,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316608,"statfs":{"total":96636764160,"available":96580206592,"internally_reserved":0,"allocated":1875968,"data_stored":907999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51068,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320772,"statfs":{"total":96636764160,"available":96584470528,"internally_reserved":0,"allocated":1871872,"data_stored":907980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11879,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:44:03.616 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.616 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.616 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.616 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.616 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.616 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.617 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.617 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.617 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.617 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.617 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.617 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.617 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-17T22:41:08.370723+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=8, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.617 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.617 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-17T22:41:08.370723+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=8, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.618 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.618 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-17T22:41:08.370723+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=8, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.618 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.618 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-17T22:41:08.370723+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=8, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.618 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.618 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.618 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-17T22:41:08.370723+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=8, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.618 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.618 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-17T22:41:08.370723+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=8, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.618 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.618 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-17T22:41:08.370723+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=8, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.619 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.619 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-17T22:41:08.370723+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=8, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.619 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-17T22:41:02.338825+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=2, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.619 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.619 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.619 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.619 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.619 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.619 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.619 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.620 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.620 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.620 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.620 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.620 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-17T22:41:18.423563+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=41, tm_sec=18, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=22, tm_min=43, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T22:44:03.620 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-17T22:44:23.621 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T22:44:23.912 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-17T22:44:23.912 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-17T22:44:23.924 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-09-17T22:44:22.345912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":500,"num_object_clones":0,"num_object_copies":1000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":500,"num_whiteouts":492,"num_read":268221,"num_read_kb":802336,"num_write":18302,"num_write_kb":296121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":30918130,"num_keys_recovered":86,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17253,"ondisk_log_size":17253,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":320096,"kb_used_data":14752,"kb_used_omap":32,"kb_used_meta":305183,"kb_avail":754654624,"statfs":{"total":773094113280,"available":772766334976,"internally_reserved":0,"allocated":15106048,"data_stored":8444652,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":33450,"internal_metadata":312507734},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002757"},"pg_stats":[{"pgid":"3.1f","version":"156'873","reported_seq":7334,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:03.704501+0000","last_change":"2024-09-17T22:44:03.704552+0000","last_active":"2024-09-17T22:44:03.704501+0000","last_peered":"2024-09-17T22:44:03.704501+0000","last_clean":"2024-09-17T22:44:03.704501+0000","last_became_active":"2024-09-17T22:41:19.443658+0000","last_became_peered":"2024-09-17T22:41:19.443658+0000","last_unstale":"2024-09-17T22:44:03.704501+0000","last_undegraded":"2024-09-17T22:44:03.704501+0000","last_fullsized":"2024-09-17T22:44:03.704501+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'873","last_scrub_stamp":"2024-09-17T22:44:03.704396+0000","last_deep_scrub":"156'873","last_deep_scrub_stamp":"2024-09-17T22:44:03.704396+0000","last_clean_scrub_stamp":"2024-09-17T22:44:03.704396+0000","objects_scrubbed":13,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:05:03.574422+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.001073428,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8816,"num_read_kb":14906,"num_write":1102,"num_write_kb":4548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"156'806","reported_seq":7453,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:07.588403+0000","last_change":"2024-09-17T22:44:07.588483+0000","last_active":"2024-09-17T22:44:07.588403+0000","last_peered":"2024-09-17T22:44:07.588403+0000","last_clean":"2024-09-17T22:44:07.588403+0000","last_became_active":"2024-09-17T22:41:19.449357+0000","last_became_peered":"2024-09-17T22:41:19.449357+0000","last_unstale":"2024-09-17T22:44:07.588403+0000","last_undegraded":"2024-09-17T22:44:07.588403+0000","last_fullsized":"2024-09-17T22:44:07.588403+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'806","last_scrub_stamp":"2024-09-17T22:44:07.588291+0000","last_deep_scrub":"156'806","last_deep_scrub_stamp":"2024-09-17T22:44:07.588291+0000","last_clean_scrub_stamp":"2024-09-17T22:44:07.588291+0000","objects_scrubbed":20,"log_size":806,"log_dups_size":0,"ondisk_log_size":806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:27:37.673501+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0.00051110000000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":8865,"num_read_kb":26537,"num_write":1126,"num_write_kb":10958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"156'447","reported_seq":2403,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:05.513164+0000","last_change":"2024-09-17T22:44:05.513220+0000","last_active":"2024-09-17T22:44:05.513164+0000","last_peered":"2024-09-17T22:44:05.513164+0000","last_clean":"2024-09-17T22:44:05.513164+0000","last_became_active":"2024-09-17T22:42:58.002822+0000","last_became_peered":"2024-09-17T22:42:58.002822+0000","last_unstale":"2024-09-17T22:44:05.513164+0000","last_undegraded":"2024-09-17T22:44:05.513164+0000","last_fullsized":"2024-09-17T22:44:05.513164+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'447","last_scrub_stamp":"2024-09-17T22:44:05.513075+0000","last_deep_scrub":"156'447","last_deep_scrub_stamp":"2024-09-17T22:44:05.513075+0000","last_clean_scrub_stamp":"2024-09-17T22:44:05.513075+0000","objects_scrubbed":12,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:58:05.356619+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0022657670000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":17641,"num_read_kb":48444,"num_write":377,"num_write_kb":8987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":7308630,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"156'503","reported_seq":2846,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:04.488297+0000","last_change":"2024-09-17T22:44:04.488337+0000","last_active":"2024-09-17T22:44:04.488297+0000","last_peered":"2024-09-17T22:44:04.488297+0000","last_clean":"2024-09-17T22:44:04.488297+0000","last_became_active":"2024-09-17T22:42:34.310411+0000","last_became_peered":"2024-09-17T22:42:34.310411+0000","last_unstale":"2024-09-17T22:44:04.488297+0000","last_undegraded":"2024-09-17T22:44:04.488297+0000","last_fullsized":"2024-09-17T22:44:04.488297+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'503","last_scrub_stamp":"2024-09-17T22:44:04.488221+0000","last_deep_scrub":"156'503","last_deep_scrub_stamp":"2024-09-17T22:44:04.488221+0000","last_clean_scrub_stamp":"2024-09-17T22:44:04.488221+0000","objects_scrubbed":9,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:47:12.381506+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0039390110000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":2615,"num_read_kb":14215,"num_write":599,"num_write_kb":12172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":7352772,"num_keys_recovered":74,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"156'256","reported_seq":8139,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:08.709013+0000","last_change":"2024-09-17T22:44:08.709058+0000","last_active":"2024-09-17T22:44:08.709013+0000","last_peered":"2024-09-17T22:44:08.709013+0000","last_clean":"2024-09-17T22:44:08.709013+0000","last_became_active":"2024-09-17T22:43:35.215952+0000","last_became_peered":"2024-09-17T22:43:35.215952+0000","last_unstale":"2024-09-17T22:44:08.709013+0000","last_undegraded":"2024-09-17T22:44:08.709013+0000","last_fullsized":"2024-09-17T22:44:08.709013+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'256","last_scrub_stamp":"2024-09-17T22:44:08.708925+0000","last_deep_scrub":"156'256","last_deep_scrub_stamp":"2024-09-17T22:44:08.708925+0000","last_clean_scrub_stamp":"2024-09-17T22:44:08.708925+0000","objects_scrubbed":12,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:37:34.459218+0000","scrub_duration":9,"objects_trimmed":28,"snaptrim_duration":0.091482261999999995,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":7315,"num_read_kb":22822,"num_write":186,"num_write_kb":6215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":3640098,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"156'388","reported_seq":5281,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:02.524131+0000","last_change":"2024-09-17T22:44:02.524155+0000","last_active":"2024-09-17T22:44:02.524131+0000","last_peered":"2024-09-17T22:44:02.524131+0000","last_clean":"2024-09-17T22:44:02.524131+0000","last_became_active":"2024-09-17T22:41:19.446541+0000","last_became_peered":"2024-09-17T22:41:19.446541+0000","last_unstale":"2024-09-17T22:44:02.524131+0000","last_undegraded":"2024-09-17T22:44:02.524131+0000","last_fullsized":"2024-09-17T22:44:02.524131+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'388","last_scrub_stamp":"2024-09-17T22:44:02.524040+0000","last_deep_scrub":"156'388","last_deep_scrub_stamp":"2024-09-17T22:44:02.524040+0000","last_clean_scrub_stamp":"2024-09-17T22:44:02.524040+0000","objects_scrubbed":28,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:13:37.230132+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.0035192800000000001,"stat_sum":{"num_bytes":19,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":27,"num_read":5091,"num_read_kb":19688,"num_write":329,"num_write_kb":4179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"156'407","reported_seq":6791,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:05.610576+0000","last_change":"2024-09-17T22:44:05.610617+0000","last_active":"2024-09-17T22:44:05.610576+0000","last_peered":"2024-09-17T22:44:05.610576+0000","last_clean":"2024-09-17T22:44:05.610576+0000","last_became_active":"2024-09-17T22:41:19.449940+0000","last_became_peered":"2024-09-17T22:41:19.449940+0000","last_unstale":"2024-09-17T22:44:05.610576+0000","last_undegraded":"2024-09-17T22:44:05.610576+0000","last_fullsized":"2024-09-17T22:44:05.610576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'407","last_scrub_stamp":"2024-09-17T22:44:05.610489+0000","last_deep_scrub":"156'407","last_deep_scrub_stamp":"2024-09-17T22:44:05.610489+0000","last_clean_scrub_stamp":"2024-09-17T22:44:05.610489+0000","objects_scrubbed":16,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:44:02.013675+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.002665995,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6668,"num_read_kb":22143,"num_write":345,"num_write_kb":9035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"156'399","reported_seq":3248,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:06.572615+0000","last_change":"2024-09-17T22:44:06.572670+0000","last_active":"2024-09-17T22:44:06.572615+0000","last_peered":"2024-09-17T22:44:06.572615+0000","last_clean":"2024-09-17T22:44:06.572615+0000","last_became_active":"2024-09-17T22:41:19.446287+0000","last_became_peered":"2024-09-17T22:41:19.446287+0000","last_unstale":"2024-09-17T22:44:06.572615+0000","last_undegraded":"2024-09-17T22:44:06.572615+0000","last_fullsized":"2024-09-17T22:44:06.572615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'399","last_scrub_stamp":"2024-09-17T22:44:06.572511+0000","last_deep_scrub":"156'399","last_deep_scrub_stamp":"2024-09-17T22:44:06.572511+0000","last_clean_scrub_stamp":"2024-09-17T22:44:06.572511+0000","objects_scrubbed":16,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:19:58.725591+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001462812,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3441,"num_read_kb":20083,"num_write":323,"num_write_kb":10930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"156'528","reported_seq":2778,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:05.620734+0000","last_change":"2024-09-17T22:44:05.620772+0000","last_active":"2024-09-17T22:44:05.620734+0000","last_peered":"2024-09-17T22:44:05.620734+0000","last_clean":"2024-09-17T22:44:05.620734+0000","last_became_active":"2024-09-17T22:41:19.445923+0000","last_became_peered":"2024-09-17T22:41:19.445923+0000","last_unstale":"2024-09-17T22:44:05.620734+0000","last_undegraded":"2024-09-17T22:44:05.620734+0000","last_fullsized":"2024-09-17T22:44:05.620734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'528","last_scrub_stamp":"2024-09-17T22:44:05.620644+0000","last_deep_scrub":"156'528","last_deep_scrub_stamp":"2024-09-17T22:44:05.620644+0000","last_clean_scrub_stamp":"2024-09-17T22:44:05.620644+0000","objects_scrubbed":15,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:53:08.832207+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.001414085,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3445,"num_read_kb":15105,"num_write":513,"num_write_kb":8714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"156'577","reported_seq":22420,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:02.517716+0000","last_change":"2024-09-17T22:44:02.517784+0000","last_active":"2024-09-17T22:44:02.517716+0000","last_peered":"2024-09-17T22:44:02.517716+0000","last_clean":"2024-09-17T22:44:02.517716+0000","last_became_active":"2024-09-17T22:43:52.384987+0000","last_became_peered":"2024-09-17T22:43:52.384987+0000","last_unstale":"2024-09-17T22:44:02.517716+0000","last_undegraded":"2024-09-17T22:44:02.517716+0000","last_fullsized":"2024-09-17T22:44:02.517716+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'577","last_scrub_stamp":"2024-09-17T22:44:02.517605+0000","last_deep_scrub":"156'577","last_deep_scrub_stamp":"2024-09-17T22:44:02.517605+0000","last_clean_scrub_stamp":"2024-09-17T22:44:02.517605+0000","objects_scrubbed":15,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:36:01.212375+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0016446880000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":22869,"num_read_kb":64042,"num_write":605,"num_write_kb":12652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.15","version":"156'454","reported_seq":7629,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:01.497867+0000","last_change":"2024-09-17T22:44:01.497940+0000","last_active":"2024-09-17T22:44:01.497867+0000","last_peered":"2024-09-17T22:44:01.497867+0000","last_clean":"2024-09-17T22:44:01.497867+0000","last_became_active":"2024-09-17T22:43:52.380232+0000","last_became_peered":"2024-09-17T22:43:52.380232+0000","last_unstale":"2024-09-17T22:44:01.497867+0000","last_undegraded":"2024-09-17T22:44:01.497867+0000","last_fullsized":"2024-09-17T22:44:01.497867+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'454","last_scrub_stamp":"2024-09-17T22:44:01.497759+0000","last_deep_scrub":"156'454","last_deep_scrub_stamp":"2024-09-17T22:44:01.497759+0000","last_clean_scrub_stamp":"2024-09-17T22:44:01.497759+0000","objects_scrubbed":17,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:43:41.717321+0000","scrub_duration":10,"objects_trimmed":22,"snaptrim_duration":0.062977108000000004,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8763,"num_read_kb":20352,"num_write":343,"num_write_kb":4822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2823905,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"156'599","reported_seq":7876,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:04.230750+0000","last_change":"2024-09-17T22:44:04.230782+0000","last_active":"2024-09-17T22:44:04.230750+0000","last_peered":"2024-09-17T22:44:04.230750+0000","last_clean":"2024-09-17T22:44:04.230750+0000","last_became_active":"2024-09-17T22:41:19.451631+0000","last_became_peered":"2024-09-17T22:41:19.451631+0000","last_unstale":"2024-09-17T22:44:04.230750+0000","last_undegraded":"2024-09-17T22:44:04.230750+0000","last_fullsized":"2024-09-17T22:44:04.230750+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'599","last_scrub_stamp":"2024-09-17T22:44:04.230679+0000","last_deep_scrub":"156'599","last_deep_scrub_stamp":"2024-09-17T22:44:04.230679+0000","last_clean_scrub_stamp":"2024-09-17T22:44:04.230679+0000","objects_scrubbed":10,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:49:12.089138+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00090157099999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8321,"num_read_kb":19442,"num_write":736,"num_write_kb":19536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":304,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:06.195357+0000","last_change":"2024-09-17T22:44:06.195383+0000","last_active":"2024-09-17T22:44:06.195357+0000","last_peered":"2024-09-17T22:44:06.195357+0000","last_clean":"2024-09-17T22:44:06.195357+0000","last_became_active":"2024-09-17T22:43:32.195676+0000","last_became_peered":"2024-09-17T22:43:32.195676+0000","last_unstale":"2024-09-17T22:44:06.195357+0000","last_undegraded":"2024-09-17T22:44:06.195357+0000","last_fullsized":"2024-09-17T22:44:06.195357+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:44:06.195265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:44:06.195265+0000","last_clean_scrub_stamp":"2024-09-17T22:44:06.195265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:53:13.540597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082453599999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"156'589","reported_seq":5359,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:02.542298+0000","last_change":"2024-09-17T22:44:02.542368+0000","last_active":"2024-09-17T22:44:02.542298+0000","last_peered":"2024-09-17T22:44:02.542298+0000","last_clean":"2024-09-17T22:44:02.542298+0000","last_became_active":"2024-09-17T22:41:19.448966+0000","last_became_peered":"2024-09-17T22:41:19.448966+0000","last_unstale":"2024-09-17T22:44:02.542298+0000","last_undegraded":"2024-09-17T22:44:02.542298+0000","last_fullsized":"2024-09-17T22:44:02.542298+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'589","last_scrub_stamp":"2024-09-17T22:44:02.542204+0000","last_deep_scrub":"156'589","last_deep_scrub_stamp":"2024-09-17T22:44:02.542204+0000","last_clean_scrub_stamp":"2024-09-17T22:44:02.542204+0000","objects_scrubbed":10,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:49:56.710616+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0026117269999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":6258,"num_read_kb":29920,"num_write":539,"num_write_kb":6243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":303,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:07.659919+0000","last_change":"2024-09-17T22:44:07.659948+0000","last_active":"2024-09-17T22:44:07.659919+0000","last_peered":"2024-09-17T22:44:07.659919+0000","last_clean":"2024-09-17T22:44:07.659919+0000","last_became_active":"2024-09-17T22:43:32.195451+0000","last_became_peered":"2024-09-17T22:43:32.195451+0000","last_unstale":"2024-09-17T22:44:07.659919+0000","last_undegraded":"2024-09-17T22:44:07.659919+0000","last_fullsized":"2024-09-17T22:44:07.659919+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:44:07.659820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:44:07.659820+0000","last_clean_scrub_stamp":"2024-09-17T22:44:07.659820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:18:41.222389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037315200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"156'412","reported_seq":1800,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:06.658923+0000","last_change":"2024-09-17T22:44:06.658978+0000","last_active":"2024-09-17T22:44:06.658923+0000","last_peered":"2024-09-17T22:44:06.658923+0000","last_clean":"2024-09-17T22:44:06.658923+0000","last_became_active":"2024-09-17T22:41:19.451637+0000","last_became_peered":"2024-09-17T22:41:19.451637+0000","last_unstale":"2024-09-17T22:44:06.658923+0000","last_undegraded":"2024-09-17T22:44:06.658923+0000","last_fullsized":"2024-09-17T22:44:06.658923+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'412","last_scrub_stamp":"2024-09-17T22:44:06.658813+0000","last_deep_scrub":"156'412","last_deep_scrub_stamp":"2024-09-17T22:44:06.658813+0000","last_clean_scrub_stamp":"2024-09-17T22:44:06.658813+0000","objects_scrubbed":16,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:23:36.309395+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.001593059,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":1474,"num_read_kb":9975,"num_write":408,"num_write_kb":7024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":302,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:07.204400+0000","last_change":"2024-09-17T22:44:07.204424+0000","last_active":"2024-09-17T22:44:07.204400+0000","last_peered":"2024-09-17T22:44:07.204400+0000","last_clean":"2024-09-17T22:44:07.204400+0000","last_became_active":"2024-09-17T22:43:15.153956+0000","last_became_peered":"2024-09-17T22:43:15.153956+0000","last_unstale":"2024-09-17T22:44:07.204400+0000","last_undegraded":"2024-09-17T22:44:07.204400+0000","last_fullsized":"2024-09-17T22:44:07.204400+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:44:07.204264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:44:07.204264+0000","last_clean_scrub_stamp":"2024-09-17T22:44:07.204264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:31.060663+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00075989299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"156'493","reported_seq":1457,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:03.731313+0000","last_change":"2024-09-17T22:44:03.731375+0000","last_active":"2024-09-17T22:44:03.731313+0000","last_peered":"2024-09-17T22:44:03.731313+0000","last_clean":"2024-09-17T22:44:03.731313+0000","last_became_active":"2024-09-17T22:41:19.444893+0000","last_became_peered":"2024-09-17T22:41:19.444893+0000","last_unstale":"2024-09-17T22:44:03.731313+0000","last_undegraded":"2024-09-17T22:44:03.731313+0000","last_fullsized":"2024-09-17T22:44:03.731313+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'493","last_scrub_stamp":"2024-09-17T22:44:03.731208+0000","last_deep_scrub":"156'493","last_deep_scrub_stamp":"2024-09-17T22:44:03.731208+0000","last_clean_scrub_stamp":"2024-09-17T22:44:03.731208+0000","objects_scrubbed":15,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:38:13.272730+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001189308,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":1718,"num_read_kb":9540,"num_write":633,"num_write_kb":7900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":297,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:02.693443+0000","last_change":"2024-09-17T22:44:02.693482+0000","last_active":"2024-09-17T22:44:02.693443+0000","last_peered":"2024-09-17T22:44:02.693443+0000","last_clean":"2024-09-17T22:44:02.693443+0000","last_became_active":"2024-09-17T22:41:09.651118+0000","last_became_peered":"2024-09-17T22:41:09.651118+0000","last_unstale":"2024-09-17T22:44:02.693443+0000","last_undegraded":"2024-09-17T22:44:02.693443+0000","last_fullsized":"2024-09-17T22:44:02.693443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:44:02.693306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:44:02.693306+0000","last_clean_scrub_stamp":"2024-09-17T22:44:02.693306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:57:13.539332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046615900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"156'402","reported_seq":15637,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:01.764132+0000","last_change":"2024-09-17T22:44:01.764173+0000","last_active":"2024-09-17T22:44:01.764132+0000","last_peered":"2024-09-17T22:44:01.764132+0000","last_clean":"2024-09-17T22:44:01.764132+0000","last_became_active":"2024-09-17T22:41:19.451723+0000","last_became_peered":"2024-09-17T22:41:19.451723+0000","last_unstale":"2024-09-17T22:44:01.764132+0000","last_undegraded":"2024-09-17T22:44:01.764132+0000","last_fullsized":"2024-09-17T22:44:01.764132+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'402","last_scrub_stamp":"2024-09-17T22:44:01.764061+0000","last_deep_scrub":"156'402","last_deep_scrub_stamp":"2024-09-17T22:44:01.764061+0000","last_clean_scrub_stamp":"2024-09-17T22:44:01.764061+0000","objects_scrubbed":9,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:26:01.739906+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00061483000000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":15368,"num_read_kb":33652,"num_write":428,"num_write_kb":8748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"156'905","reported_seq":12162,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:04.595987+0000","last_change":"2024-09-17T22:44:04.596015+0000","last_active":"2024-09-17T22:44:04.595987+0000","last_peered":"2024-09-17T22:44:04.595987+0000","last_clean":"2024-09-17T22:44:04.595987+0000","last_became_active":"2024-09-17T22:43:52.388167+0000","last_became_peered":"2024-09-17T22:43:52.388167+0000","last_unstale":"2024-09-17T22:44:04.595987+0000","last_undegraded":"2024-09-17T22:44:04.595987+0000","last_fullsized":"2024-09-17T22:44:04.595987+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'905","last_scrub_stamp":"2024-09-17T22:44:04.595915+0000","last_deep_scrub":"156'905","last_deep_scrub_stamp":"2024-09-17T22:44:04.595915+0000","last_clean_scrub_stamp":"2024-09-17T22:44:04.595915+0000","objects_scrubbed":26,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:35:34.802066+0000","scrub_duration":24,"objects_trimmed":42,"snaptrim_duration":0.107009245,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":15325,"num_read_kb":33693,"num_write":856,"num_write_kb":8706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":4229520,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"156'3","reported_seq":214,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:11.270598+0000","last_change":"2024-09-17T22:44:11.270646+0000","last_active":"2024-09-17T22:44:11.270598+0000","last_peered":"2024-09-17T22:44:11.270598+0000","last_clean":"2024-09-17T22:44:11.270598+0000","last_became_active":"2024-09-17T22:43:42.285307+0000","last_became_peered":"2024-09-17T22:43:42.285307+0000","last_unstale":"2024-09-17T22:44:11.270598+0000","last_undegraded":"2024-09-17T22:44:11.270598+0000","last_fullsized":"2024-09-17T22:44:11.270598+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'3","last_scrub_stamp":"2024-09-17T22:44:11.270477+0000","last_deep_scrub":"156'3","last_deep_scrub_stamp":"2024-09-17T22:44:11.270477+0000","last_clean_scrub_stamp":"2024-09-17T22:44:11.270477+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:20:03.161155+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00052758799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"156'402","reported_seq":4218,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:04.671386+0000","last_change":"2024-09-17T22:44:04.671450+0000","last_active":"2024-09-17T22:44:04.671386+0000","last_peered":"2024-09-17T22:44:04.671386+0000","last_clean":"2024-09-17T22:44:04.671386+0000","last_became_active":"2024-09-17T22:41:19.444314+0000","last_became_peered":"2024-09-17T22:41:19.444314+0000","last_unstale":"2024-09-17T22:44:04.671386+0000","last_undegraded":"2024-09-17T22:44:04.671386+0000","last_fullsized":"2024-09-17T22:44:04.671386+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'402","last_scrub_stamp":"2024-09-17T22:44:04.671281+0000","last_deep_scrub":"156'402","last_deep_scrub_stamp":"2024-09-17T22:44:04.671281+0000","last_clean_scrub_stamp":"2024-09-17T22:44:04.671281+0000","objects_scrubbed":17,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:01:41.483151+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0011857370000000001,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4006,"num_read_kb":22354,"num_write":358,"num_write_kb":9353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":297,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:03.363060+0000","last_change":"2024-09-17T22:44:03.363100+0000","last_active":"2024-09-17T22:44:03.363060+0000","last_peered":"2024-09-17T22:44:03.363060+0000","last_clean":"2024-09-17T22:44:03.363060+0000","last_became_active":"2024-09-17T22:41:09.389227+0000","last_became_peered":"2024-09-17T22:41:09.389227+0000","last_unstale":"2024-09-17T22:44:03.363060+0000","last_undegraded":"2024-09-17T22:44:03.363060+0000","last_fullsized":"2024-09-17T22:44:03.363060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:44:03.362961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:44:03.362961+0000","last_clean_scrub_stamp":"2024-09-17T22:44:03.362961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:25:36.691468+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.000372066,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"156'493","reported_seq":7478,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:04.705126+0000","last_change":"2024-09-17T22:44:04.705165+0000","last_active":"2024-09-17T22:44:04.705126+0000","last_peered":"2024-09-17T22:44:04.705126+0000","last_clean":"2024-09-17T22:44:04.705126+0000","last_became_active":"2024-09-17T22:41:19.449791+0000","last_became_peered":"2024-09-17T22:41:19.449791+0000","last_unstale":"2024-09-17T22:44:04.705126+0000","last_undegraded":"2024-09-17T22:44:04.705126+0000","last_fullsized":"2024-09-17T22:44:04.705126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'493","last_scrub_stamp":"2024-09-17T22:44:04.705046+0000","last_deep_scrub":"156'493","last_deep_scrub_stamp":"2024-09-17T22:44:04.705046+0000","last_clean_scrub_stamp":"2024-09-17T22:44:04.705046+0000","objects_scrubbed":12,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:29.374611+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.00088469700000000004,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8092,"num_read_kb":9247,"num_write":519,"num_write_kb":4605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":297,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:02.230440+0000","last_change":"2024-09-17T22:44:02.230486+0000","last_active":"2024-09-17T22:44:02.230440+0000","last_peered":"2024-09-17T22:44:02.230440+0000","last_clean":"2024-09-17T22:44:02.230440+0000","last_became_active":"2024-09-17T22:41:09.388584+0000","last_became_peered":"2024-09-17T22:41:09.388584+0000","last_unstale":"2024-09-17T22:44:02.230440+0000","last_undegraded":"2024-09-17T22:44:02.230440+0000","last_fullsized":"2024-09-17T22:44:02.230440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T22:44:02.230333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T22:44:02.230333+0000","last_clean_scrub_stamp":"2024-09-17T22:44:02.230333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:23:15.960578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046193900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"156'770","reported_seq":8764,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:04.585111+0000","last_change":"2024-09-17T22:44:04.585168+0000","last_active":"2024-09-17T22:44:04.585111+0000","last_peered":"2024-09-17T22:44:04.585111+0000","last_clean":"2024-09-17T22:44:04.585111+0000","last_became_active":"2024-09-17T22:41:19.449279+0000","last_became_peered":"2024-09-17T22:41:19.449279+0000","last_unstale":"2024-09-17T22:44:04.585111+0000","last_undegraded":"2024-09-17T22:44:04.585111+0000","last_fullsized":"2024-09-17T22:44:04.585111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'770","last_scrub_stamp":"2024-09-17T22:44:04.585022+0000","last_deep_scrub":"156'770","last_deep_scrub_stamp":"2024-09-17T22:44:04.585022+0000","last_clean_scrub_stamp":"2024-09-17T22:44:04.585022+0000","objects_scrubbed":14,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:53:47.034176+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.00089414599999999996,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":11291,"num_read_kb":22947,"num_write":956,"num_write_kb":2860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"156'2","reported_seq":213,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:12.282506+0000","last_change":"2024-09-17T22:44:12.282537+0000","last_active":"2024-09-17T22:44:12.282506+0000","last_peered":"2024-09-17T22:44:12.282506+0000","last_clean":"2024-09-17T22:44:12.282506+0000","last_became_active":"2024-09-17T22:43:42.286652+0000","last_became_peered":"2024-09-17T22:43:42.286652+0000","last_unstale":"2024-09-17T22:44:12.282506+0000","last_undegraded":"2024-09-17T22:44:12.282506+0000","last_fullsized":"2024-09-17T22:44:12.282506+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'2","last_scrub_stamp":"2024-09-17T22:44:12.282406+0000","last_deep_scrub":"156'2","last_deep_scrub_stamp":"2024-09-17T22:44:12.282406+0000","last_clean_scrub_stamp":"2024-09-17T22:44:12.282406+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:22:06.294853+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0011040080000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"156'119","reported_seq":451,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:01.497582+0000","last_change":"2024-09-17T22:44:01.497611+0000","last_active":"2024-09-17T22:44:01.497582+0000","last_peered":"2024-09-17T22:44:01.497582+0000","last_clean":"2024-09-17T22:44:01.497582+0000","last_became_active":"2024-09-17T22:41:03.606079+0000","last_became_peered":"2024-09-17T22:41:03.606079+0000","last_unstale":"2024-09-17T22:44:01.497582+0000","last_undegraded":"2024-09-17T22:44:01.497582+0000","last_fullsized":"2024-09-17T22:44:01.497582+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'119","last_scrub_stamp":"2024-09-17T22:44:01.497516+0000","last_deep_scrub":"156'119","last_deep_scrub_stamp":"2024-09-17T22:44:01.497516+0000","last_clean_scrub_stamp":"2024-09-17T22:44:01.497516+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:33:07.344356+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"156'629","reported_seq":11977,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:03.539160+0000","last_change":"2024-09-17T22:44:03.539205+0000","last_active":"2024-09-17T22:44:03.539160+0000","last_peered":"2024-09-17T22:44:03.539160+0000","last_clean":"2024-09-17T22:44:03.539160+0000","last_became_active":"2024-09-17T22:41:19.448956+0000","last_became_peered":"2024-09-17T22:41:19.448956+0000","last_unstale":"2024-09-17T22:44:03.539160+0000","last_undegraded":"2024-09-17T22:44:03.539160+0000","last_fullsized":"2024-09-17T22:44:03.539160+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'629","last_scrub_stamp":"2024-09-17T22:44:03.539069+0000","last_deep_scrub":"156'629","last_deep_scrub_stamp":"2024-09-17T22:44:03.539069+0000","last_clean_scrub_stamp":"2024-09-17T22:44:03.539069+0000","objects_scrubbed":16,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:47:01.842985+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0017463789999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":13252,"num_read_kb":46911,"num_write":655,"num_write_kb":25175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"156'793","reported_seq":14778,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:03.226943+0000","last_change":"2024-09-17T22:44:03.226999+0000","last_active":"2024-09-17T22:44:03.226943+0000","last_peered":"2024-09-17T22:44:03.226943+0000","last_clean":"2024-09-17T22:44:03.226943+0000","last_became_active":"2024-09-17T22:41:19.449003+0000","last_became_peered":"2024-09-17T22:41:19.449003+0000","last_unstale":"2024-09-17T22:44:03.226943+0000","last_undegraded":"2024-09-17T22:44:03.226943+0000","last_fullsized":"2024-09-17T22:44:03.226943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'793","last_scrub_stamp":"2024-09-17T22:44:03.226834+0000","last_deep_scrub":"156'793","last_deep_scrub_stamp":"2024-09-17T22:44:03.226834+0000","last_clean_scrub_stamp":"2024-09-17T22:44:03.226834+0000","objects_scrubbed":8,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:27:42.549277+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.001910964,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":16223,"num_read_kb":43243,"num_write":1040,"num_write_kb":9843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"156'486","reported_seq":8511,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:10.250607+0000","last_change":"2024-09-17T22:44:10.250655+0000","last_active":"2024-09-17T22:44:10.250607+0000","last_peered":"2024-09-17T22:44:10.250607+0000","last_clean":"2024-09-17T22:44:10.250607+0000","last_became_active":"2024-09-17T22:43:42.284094+0000","last_became_peered":"2024-09-17T22:43:42.284094+0000","last_unstale":"2024-09-17T22:44:10.250607+0000","last_undegraded":"2024-09-17T22:44:10.250607+0000","last_fullsized":"2024-09-17T22:44:10.250607+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'486","last_scrub_stamp":"2024-09-17T22:44:10.250519+0000","last_deep_scrub":"156'486","last_deep_scrub_stamp":"2024-09-17T22:44:10.250519+0000","last_clean_scrub_stamp":"2024-09-17T22:44:10.250519+0000","objects_scrubbed":16,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:19:41.426420+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.001830772,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":8191,"num_read_kb":41081,"num_write":473,"num_write_kb":9769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"156'437","reported_seq":2760,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:04.418742+0000","last_change":"2024-09-17T22:44:04.418792+0000","last_active":"2024-09-17T22:44:04.418742+0000","last_peered":"2024-09-17T22:44:04.418742+0000","last_clean":"2024-09-17T22:44:04.418742+0000","last_became_active":"2024-09-17T22:41:19.447215+0000","last_became_peered":"2024-09-17T22:41:19.447215+0000","last_unstale":"2024-09-17T22:44:04.418742+0000","last_undegraded":"2024-09-17T22:44:04.418742+0000","last_fullsized":"2024-09-17T22:44:04.418742+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'437","last_scrub_stamp":"2024-09-17T22:44:04.418647+0000","last_deep_scrub":"156'437","last_deep_scrub_stamp":"2024-09-17T22:44:04.418647+0000","last_clean_scrub_stamp":"2024-09-17T22:44:04.418647+0000","objects_scrubbed":14,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:04:03.889661+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.00066798900000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3409,"num_read_kb":14480,"num_write":404,"num_write_kb":6277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"156'373","reported_seq":21717,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:00.452524+0000","last_change":"2024-09-17T22:44:00.452571+0000","last_active":"2024-09-17T22:44:00.452524+0000","last_peered":"2024-09-17T22:44:00.452524+0000","last_clean":"2024-09-17T22:44:00.452524+0000","last_became_active":"2024-09-17T22:43:52.380211+0000","last_became_peered":"2024-09-17T22:43:52.380211+0000","last_unstale":"2024-09-17T22:44:00.452524+0000","last_undegraded":"2024-09-17T22:44:00.452524+0000","last_fullsized":"2024-09-17T22:44:00.452524+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'373","last_scrub_stamp":"2024-09-17T22:44:00.452445+0000","last_deep_scrub":"156'373","last_deep_scrub_stamp":"2024-09-17T22:44:00.452445+0000","last_clean_scrub_stamp":"2024-09-17T22:44:00.452445+0000","objects_scrubbed":20,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:06:11.731009+0000","scrub_duration":11,"objects_trimmed":22,"snaptrim_duration":0.064717442,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":21473,"num_read_kb":63694,"num_write":340,"num_write_kb":14940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":1192963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"156'500","reported_seq":2054,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:09.236629+0000","last_change":"2024-09-17T22:44:09.236685+0000","last_active":"2024-09-17T22:44:09.236629+0000","last_peered":"2024-09-17T22:44:09.236629+0000","last_clean":"2024-09-17T22:44:09.236629+0000","last_became_active":"2024-09-17T22:43:42.282252+0000","last_became_peered":"2024-09-17T22:43:42.282252+0000","last_unstale":"2024-09-17T22:44:09.236629+0000","last_undegraded":"2024-09-17T22:44:09.236629+0000","last_fullsized":"2024-09-17T22:44:09.236629+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'500","last_scrub_stamp":"2024-09-17T22:44:09.236544+0000","last_deep_scrub":"156'500","last_deep_scrub_stamp":"2024-09-17T22:44:09.236544+0000","last_clean_scrub_stamp":"2024-09-17T22:44:09.236544+0000","objects_scrubbed":14,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:04.568678+0000","scrub_duration":12,"objects_trimmed":8,"snaptrim_duration":0.002173033,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2216,"num_read_kb":10204,"num_write":574,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"156'389","reported_seq":8865,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:02.743760+0000","last_change":"2024-09-17T22:44:02.743837+0000","last_active":"2024-09-17T22:44:02.743760+0000","last_peered":"2024-09-17T22:44:02.743760+0000","last_clean":"2024-09-17T22:44:02.743760+0000","last_became_active":"2024-09-17T22:43:35.224141+0000","last_became_peered":"2024-09-17T22:43:35.224141+0000","last_unstale":"2024-09-17T22:44:02.743760+0000","last_undegraded":"2024-09-17T22:44:02.743760+0000","last_fullsized":"2024-09-17T22:44:02.743760+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'389","last_scrub_stamp":"2024-09-17T22:44:02.743682+0000","last_deep_scrub":"156'389","last_deep_scrub_stamp":"2024-09-17T22:44:02.743682+0000","last_clean_scrub_stamp":"2024-09-17T22:44:02.743682+0000","objects_scrubbed":17,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:35:17.867281+0000","scrub_duration":13,"objects_trimmed":30,"snaptrim_duration":0.098411079999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8804,"num_read_kb":23106,"num_write":291,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":2514411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"156'374","reported_seq":1890,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:03.549597+0000","last_change":"2024-09-17T22:44:03.549682+0000","last_active":"2024-09-17T22:44:03.549597+0000","last_peered":"2024-09-17T22:44:03.549597+0000","last_clean":"2024-09-17T22:44:03.549597+0000","last_became_active":"2024-09-17T22:43:52.388960+0000","last_became_peered":"2024-09-17T22:43:52.388960+0000","last_unstale":"2024-09-17T22:44:03.549597+0000","last_undegraded":"2024-09-17T22:44:03.549597+0000","last_fullsized":"2024-09-17T22:44:03.549597+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'374","last_scrub_stamp":"2024-09-17T22:44:03.549507+0000","last_deep_scrub":"156'374","last_deep_scrub_stamp":"2024-09-17T22:44:03.549507+0000","last_clean_scrub_stamp":"2024-09-17T22:44:03.549507+0000","objects_scrubbed":21,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:52:14.718652+0000","scrub_duration":12,"objects_trimmed":30,"snaptrim_duration":0.086636568999999997,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":1634,"num_read_kb":5501,"num_write":380,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1563844,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"156'367","reported_seq":2199,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:03.482762+0000","last_change":"2024-09-17T22:44:03.482801+0000","last_active":"2024-09-17T22:44:03.482762+0000","last_peered":"2024-09-17T22:44:03.482762+0000","last_clean":"2024-09-17T22:44:03.482762+0000","last_became_active":"2024-09-17T22:41:19.447419+0000","last_became_peered":"2024-09-17T22:41:19.447419+0000","last_unstale":"2024-09-17T22:44:03.482762+0000","last_undegraded":"2024-09-17T22:44:03.482762+0000","last_fullsized":"2024-09-17T22:44:03.482762+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'367","last_scrub_stamp":"2024-09-17T22:44:03.482692+0000","last_deep_scrub":"156'367","last_deep_scrub_stamp":"2024-09-17T22:44:03.482692+0000","last_clean_scrub_stamp":"2024-09-17T22:44:03.482692+0000","objects_scrubbed":18,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:26:25.705066+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.00132408,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":3110,"num_read_kb":8489,"num_write":286,"num_write_kb":3161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"156'631","reported_seq":3705,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:05.234278+0000","last_change":"2024-09-17T22:44:05.234325+0000","last_active":"2024-09-17T22:44:05.234278+0000","last_peered":"2024-09-17T22:44:05.234278+0000","last_clean":"2024-09-17T22:44:05.234278+0000","last_became_active":"2024-09-17T22:41:19.454151+0000","last_became_peered":"2024-09-17T22:41:19.454151+0000","last_unstale":"2024-09-17T22:44:05.234278+0000","last_undegraded":"2024-09-17T22:44:05.234278+0000","last_fullsized":"2024-09-17T22:44:05.234278+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'631","last_scrub_stamp":"2024-09-17T22:44:05.234179+0000","last_deep_scrub":"156'631","last_deep_scrub_stamp":"2024-09-17T22:44:05.234179+0000","last_clean_scrub_stamp":"2024-09-17T22:44:05.234179+0000","objects_scrubbed":15,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:54.743737+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.001718979,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":5266,"num_read_kb":13546,"num_write":658,"num_write_kb":3731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"156'922","reported_seq":11740,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:05.721220+0000","last_change":"2024-09-17T22:44:05.721270+0000","last_active":"2024-09-17T22:44:05.721220+0000","last_peered":"2024-09-17T22:44:05.721220+0000","last_clean":"2024-09-17T22:44:05.721220+0000","last_became_active":"2024-09-17T22:43:35.217045+0000","last_became_peered":"2024-09-17T22:43:35.217045+0000","last_unstale":"2024-09-17T22:44:05.721220+0000","last_undegraded":"2024-09-17T22:44:05.721220+0000","last_fullsized":"2024-09-17T22:44:05.721220+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'922","last_scrub_stamp":"2024-09-17T22:44:05.721135+0000","last_deep_scrub":"156'922","last_deep_scrub_stamp":"2024-09-17T22:44:05.721135+0000","last_clean_scrub_stamp":"2024-09-17T22:44:05.721135+0000","objects_scrubbed":12,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:31:27.169142+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.001497735,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14642,"num_read_kb":44074,"num_write":1142,"num_write_kb":22161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"156'525","reported_seq":2662,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-17T22:44:08.232183+0000","last_change":"2024-09-17T22:44:08.232252+0000","last_active":"2024-09-17T22:44:08.232183+0000","last_peered":"2024-09-17T22:44:08.232183+0000","last_clean":"2024-09-17T22:44:08.232183+0000","last_became_active":"2024-09-17T22:43:35.217551+0000","last_became_peered":"2024-09-17T22:43:35.217551+0000","last_unstale":"2024-09-17T22:44:08.232183+0000","last_undegraded":"2024-09-17T22:44:08.232183+0000","last_fullsized":"2024-09-17T22:44:08.232183+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'525","last_scrub_stamp":"2024-09-17T22:44:08.232097+0000","last_deep_scrub":"156'525","last_deep_scrub_stamp":"2024-09-17T22:44:08.232097+0000","last_clean_scrub_stamp":"2024-09-17T22:44:08.232097+0000","objects_scrubbed":23,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:37:18.930535+0000","scrub_duration":18,"objects_trimmed":20,"snaptrim_duration":0.082900606000000002,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":2503,"num_read_kb":8800,"num_write":617,"num_write_kb":8384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":291987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":496,"num_object_clones":0,"num_object_copies":992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":492,"num_read":268105,"num_read_kb":802236,"num_write":18141,"num_write_kb":293703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":30918130,"num_keys_recovered":86,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11879,"internal_metadata":0},"log_size":17129,"ondisk_log_size":17129,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9956,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28912,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342928,"statfs":{"total":96636764160,"available":96607158272,"internally_reserved":0,"allocated":1933312,"data_stored":1498367,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":135,"seq":579820584972,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34740,"kb_used_data":1828,"kb_used_omap":21,"kb_used_meta":32874,"kb_avail":94337100,"statfs":{"total":96636764160,"available":96601190400,"internally_reserved":0,"allocated":1871872,"data_stored":907980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22295,"internal_metadata":33663209},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542187,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41332,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330508,"statfs":{"total":96636764160,"available":96594440192,"internally_reserved":0,"allocated":1871872,"data_stored":907980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542187,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28856,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342984,"statfs":{"total":96636764160,"available":96607215616,"internally_reserved":0,"allocated":1875968,"data_stored":907999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51120,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320720,"statfs":{"total":96636764160,"available":96584417280,"internally_reserved":0,"allocated":1933312,"data_stored":1498367,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542187,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28852,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342988,"statfs":{"total":96636764160,"available":96607219712,"internally_reserved":0,"allocated":1871872,"data_stored":907980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55224,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316616,"statfs":{"total":96636764160,"available":96580214784,"internally_reserved":0,"allocated":1875968,"data_stored":907999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51060,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320780,"statfs":{"total":96636764160,"available":96584478720,"internally_reserved":0,"allocated":1871872,"data_stored":907980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4978,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11879,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T22:44:23.927 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-17T22:44:24.217 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-17T22:44:24.217 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-17T22:44:24.218 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T22:44:24.218 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T22:44:24.250 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T22:44:24.251 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T22:44:24.251 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T22:44:24.331 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T22:44:24.331 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T22:44:24.331 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T22:44:24.397 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T22:44:24.398 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-17T22:44:24.398 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T22:44:24.470 INFO:tasks.ceph.osd.3:Stopped 2024-09-17T22:44:24.470 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-17T22:44:24.470 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T22:44:24.502 INFO:tasks.ceph.osd.4:Stopped 2024-09-17T22:44:24.502 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-17T22:44:24.502 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T22:44:24.577 INFO:tasks.ceph.osd.5:Stopped 2024-09-17T22:44:24.578 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-17T22:44:24.578 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T22:44:24.649 INFO:tasks.ceph.osd.6:Stopped 2024-09-17T22:44:24.649 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-17T22:44:24.649 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T22:44:24.721 INFO:tasks.ceph.osd.7:Stopped 2024-09-17T22:44:24.722 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-17T22:44:24.722 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-17T22:44:24.722 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T22:44:24.746 INFO:tasks.ceph.mgr.y:Stopped 2024-09-17T22:44:24.747 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-17T22:44:24.747 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T22:44:24.792 INFO:tasks.ceph.mgr.x:Stopped 2024-09-17T22:44:24.792 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-17T22:44:24.792 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T22:44:24.793 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T22:44:24.804 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T22:44:24.804 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T22:44:24.804 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T22:44:24.855 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T22:44:24.855 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T22:44:24.855 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T22:44:24.867 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T22:44:24.868 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-17T22:44:24.868 DEBUG:teuthology.orchestra.run.smithi115:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-17T22:44:24.933 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-17T22:44:24.933 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-17T22:44:25.156 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-17T22:44:25.156 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-17T22:44:25.238 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-17T22:44:25.238 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-17T22:44:25.329 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi115.front.sepia.ceph.com 2024-09-17T22:44:25.329 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-17T22:44:25.379 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T22:44:25.380 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-17T22:44:25.524 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T22:44:25.524 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-17T22:44:25.580 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T22:44:25.581 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-17T22:44:25.628 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T22:44:25.628 DEBUG:teuthology.orchestra.run.smithi129:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-17T22:44:25.678 INFO:tasks.ceph:Archiving mon data... 2024-09-17T22:44:25.679 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909540/data/mon.a.tgz 2024-09-17T22:44:25.679 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-09-17T22:44:25.695 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.LX0GkLpA0C 2024-09-17T22:44:25.695 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.LX0GkLpA0C 2024-09-17T22:44:25.851 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.LX0GkLpA0C 2024-09-17T22:44:25.935 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.LX0GkLpA0C is 390KB 2024-09-17T22:44:26.034 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.LX0GkLpA0C 2024-09-17T22:44:26.049 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909540/data/mon.c.tgz 2024-09-17T22:44:26.049 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-09-17T22:44:26.103 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.Y6jyQskFfb 2024-09-17T22:44:26.103 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Y6jyQskFfb 2024-09-17T22:44:26.273 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.Y6jyQskFfb 2024-09-17T22:44:26.356 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.Y6jyQskFfb is 531KB 2024-09-17T22:44:26.454 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.Y6jyQskFfb 2024-09-17T22:44:26.469 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909540/data/mon.b.tgz 2024-09-17T22:44:26.470 DEBUG:teuthology.orchestra.run.smithi129:> mktemp 2024-09-17T22:44:26.488 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/tmp.oBHmVZMbZ9 2024-09-17T22:44:26.488 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.oBHmVZMbZ9 2024-09-17T22:44:26.686 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0666 /tmp/tmp.oBHmVZMbZ9 2024-09-17T22:44:26.770 DEBUG:teuthology.orchestra.remote:smithi129:/tmp/tmp.oBHmVZMbZ9 is 530KB 2024-09-17T22:44:26.868 DEBUG:teuthology.orchestra.run.smithi129:> rm -fr /tmp/tmp.oBHmVZMbZ9 2024-09-17T22:44:26.884 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-17T22:44:26.884 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T22:44:26.886 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T22:44:26.964 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T22:44:27.005 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T22:44:27.005 INFO:tasks.ceph:Archiving crash dumps... 2024-09-17T22:44:27.006 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909540/remote/smithi115/crash 2024-09-17T22:44:27.007 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T22:44:27.031 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909540/remote/smithi129/crash 2024-09-17T22:44:27.032 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T22:44:27.069 INFO:tasks.ceph:Compressing logs... 2024-09-17T22:44:27.070 DEBUG:teuthology.orchestra.run.smithi115:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T22:44:27.075 DEBUG:teuthology.orchestra.run.smithi129:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T22:44:27.155 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33508.log 2024-09-17T22:44:27.155 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-17T22:44:27.156 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33388.log 2024-09-17T22:44:27.156 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33508.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33508.log.gz 2024-09-17T22:44:27.156 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33428.log 2024-09-17T22:44:27.156 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33592.log 2024-09-17T22:44:27.156 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33388.log.gz 2024-09-17T22:44:27.156 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33428.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-17T22:44:27.156 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33428.log.gz 2024-09-17T22:44:27.157 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33592.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T22:44:27.157 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33592.log.gz 2024-09-17T22:44:27.157 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33548.log 2024-09-17T22:44:27.157 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-17T22:44:27.157 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33548.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33548.log.gz 2024-09-17T22:44:27.157 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-17T22:44:27.157 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T22:44:27.157 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-17T22:44:27.157 INFO:teuthology.orchestra.run.smithi129.stderr: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T22:44:27.157 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-17T22:44:27.157 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33639.log 2024-09-17T22:44:27.157 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33468.log 2024-09-17T22:44:27.158 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33639.log: 85.5% -- replaced with /var/log/ceph/ceph.log.gz 0.0% 2024-09-17T22:44:27.158 INFO:teuthology.orchestra.run.smithi129.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33639.log.gz 2024-09-17T22:44:27.158 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33679.log 2024-09-17T22:44:27.158 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.33468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33468.log.gz/var/log/ceph/ceph-client.admin.33679.log: 2024-09-17T22:44:27.158 INFO:teuthology.orchestra.run.smithi129.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33679.log.gz 2024-09-17T22:44:27.216 INFO:teuthology.orchestra.run.smithi129.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-17T22:44:27.234 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-17T22:44:27.234 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42674.log 2024-09-17T22:44:27.234 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39850.log 2024-09-17T22:44:27.234 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.42674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38264.log 2024-09-17T22:44:27.234 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42674.log.gz 2024-09-17T22:44:27.234 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51061.log 2024-09-17T22:44:27.234 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39850.log.gz 2024-09-17T22:44:27.234 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38264.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38264.log.gz 2024-09-17T22:44:27.234 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38092.log 2024-09-17T22:44:27.234 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57776.log 2024-09-17T22:44:27.235 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51061.log.gz 2024-09-17T22:44:27.235 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38092.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38092.log.gz 2024-09-17T22:44:27.235 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47415.log 2024-09-17T22:44:27.235 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41388.log 2024-09-17T22:44:27.235 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57776.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57776.log.gz 2024-09-17T22:44:27.235 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54665.log 2024-09-17T22:44:27.235 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47415.log.gz 2024-09-17T22:44:27.235 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59203.log 2024-09-17T22:44:27.235 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41388.log.gz 2024-09-17T22:44:27.235 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43396.log 2024-09-17T22:44:27.235 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54665.log.gz 2024-09-17T22:44:27.236 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50980.log 2024-09-17T22:44:27.236 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59203.log.gz 2024-09-17T22:44:27.236 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43396.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.46351.log 2024-09-17T22:44:27.236 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43396.log.gz 2024-09-17T22:44:27.236 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41564.log 2024-09-17T22:44:27.236 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50980.log.gz 2024-09-17T22:44:27.236 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/objectstore_tool.46351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59371.log 2024-09-17T22:44:27.236 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57607.log 2024-09-17T22:44:27.236 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41564.log.gz 2024-09-17T22:44:27.236 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59371.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.59371.log.gz 2024-09-17T22:44:27.237 INFO:teuthology.orchestra.run.smithi115.stderr: /var/log/ceph/ceph-client.admin.48446.log 2024-09-17T22:44:27.237 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42109.log 2024-09-17T22:44:27.237 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57607.log.gz 2024-09-17T22:44:27.237 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55900.log 2024-09-17T22:44:27.237 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48446.log.gz 2024-09-17T22:44:27.237 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.42109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41507.log 2024-09-17T22:44:27.237 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42109.log.gz 2024-09-17T22:44:27.237 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55900.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.55900.log.gz /var/log/ceph/ceph-client.admin.57781.log 2024-09-17T22:44:27.237 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-09-17T22:44:27.237 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37560.log 2024-09-17T22:44:27.237 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41507.log.gz 2024-09-17T22:44:27.238 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55858.log 2024-09-17T22:44:27.238 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57781.log.gz 2024-09-17T22:44:27.238 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58363.log 2024-09-17T22:44:27.238 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37560.log.gz 2024-09-17T22:44:27.238 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55858.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55858.log.gz 2024-09-17T22:44:27.238 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38132.log 2024-09-17T22:44:27.238 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38470.log 2024-09-17T22:44:27.238 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58363.log.gz 2024-09-17T22:44:27.238 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55080.log 2024-09-17T22:44:27.238 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38132.log.gz 2024-09-17T22:44:27.238 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57784.log 2024-09-17T22:44:27.239 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38470.log.gz 2024-09-17T22:44:27.239 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59035.log 2024-09-17T22:44:27.239 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55080.log.gz 2024-09-17T22:44:27.239 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59455.log 2024-09-17T22:44:27.239 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57784.log.gz 2024-09-17T22:44:27.239 INFO:teuthology.orchestra.run.smithi115.stderr: 94.9% -- replaced with /var/log/ceph/objectstore_tool.46351.log.gz 2024-09-17T22:44:27.239 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59035.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.37243.log 2024-09-17T22:44:27.239 INFO:teuthology.orchestra.run.smithi115.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59035.log.gz 2024-09-17T22:44:27.239 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56935.log 2024-09-17T22:44:27.239 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59455.log.gz 2024-09-17T22:44:27.239 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37243.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.37243.log.gz 2024-09-17T22:44:27.240 INFO:teuthology.orchestra.run.smithi115.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57787.log 2024-09-17T22:44:27.240 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50207.log 2024-09-17T22:44:27.240 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56935.log.gz 2024-09-17T22:44:27.240 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58405.log 2024-09-17T22:44:27.240 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57787.log.gz 2024-09-17T22:44:27.240 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54720.log 2024-09-17T22:44:27.240 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50207.log.gz 2024-09-17T22:44:27.240 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38742.log 2024-09-17T22:44:27.240 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58405.log.gz 2024-09-17T22:44:27.240 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38881.log 2024-09-17T22:44:27.240 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54720.log.gz 2024-09-17T22:44:27.241 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57019.log 2024-09-17T22:44:27.241 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38742.log.gz 2024-09-17T22:44:27.241 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51846.log 2024-09-17T22:44:27.241 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38881.log.gz 2024-09-17T22:44:27.241 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58909.log 2024-09-17T22:44:27.241 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57019.log.gz 2024-09-17T22:44:27.241 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54384.log 2024-09-17T22:44:27.241 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51846.log.gz 2024-09-17T22:44:27.241 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49729.log 2024-09-17T22:44:27.241 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58909.log.gz 2024-09-17T22:44:27.242 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54384.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.54384.log.gz --verbose 2024-09-17T22:44:27.242 INFO:teuthology.orchestra.run.smithi115.stderr: -- /var/log/ceph/ceph-osd.1.log 2024-09-17T22:44:27.242 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58120.log 2024-09-17T22:44:27.242 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49729.log.gz 2024-09-17T22:44:27.242 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52044.log 2024-09-17T22:44:27.242 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54902.log 2024-09-17T22:44:27.242 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58120.log.gz 2024-09-17T22:44:27.242 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54505.log 2024-09-17T22:44:27.242 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52044.log.gz 2024-09-17T22:44:27.242 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54902.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54902.log.gz 2024-09-17T22:44:27.243 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39589.log 2024-09-17T22:44:27.243 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49120.log 2024-09-17T22:44:27.243 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54505.log.gz 2024-09-17T22:44:27.243 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56846.log 0.0% 2024-09-17T22:44:27.243 INFO:teuthology.orchestra.run.smithi115.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39589.log.gz 2024-09-17T22:44:27.243 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53791.log 2024-09-17T22:44:27.243 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49120.log.gz 2024-09-17T22:44:27.243 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37551.log 2024-09-17T22:44:27.243 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56846.log.gz 2024-09-17T22:44:27.243 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53791.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.53791.log.gz 2024-09-17T22:44:27.244 INFO:teuthology.orchestra.run.smithi115.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58741.log 2024-09-17T22:44:27.244 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58321.log 2024-09-17T22:44:27.244 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37551.log.gz 2024-09-17T22:44:27.244 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58741.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58741.log.gzgzip 2024-09-17T22:44:27.244 INFO:teuthology.orchestra.run.smithi115.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44515.log 2024-09-17T22:44:27.244 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53716.log 2024-09-17T22:44:27.244 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58321.log.gz 2024-09-17T22:44:27.244 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.44515.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.44515.log.gz --verbose 2024-09-17T22:44:27.244 INFO:teuthology.orchestra.run.smithi115.stderr: -- /var/log/ceph/ceph-client.admin.38272.log 2024-09-17T22:44:27.244 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38270.log 2024-09-17T22:44:27.245 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53716.log.gz 2024-09-17T22:44:27.245 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58615.log 2024-09-17T22:44:27.245 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38272.log.gz 2024-09-17T22:44:27.245 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38270.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38270.log.gz 2024-09-17T22:44:27.245 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40575.log 2024-09-17T22:44:27.245 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38596.log 2024-09-17T22:44:27.245 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58615.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58615.log.gz 2024-09-17T22:44:27.245 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.40575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52458.log 2024-09-17T22:44:27.245 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40575.log.gz 2024-09-17T22:44:27.245 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57523.log 2024-09-17T22:44:27.245 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38596.log.gz 2024-09-17T22:44:27.246 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52458.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.52458.log.gz --verbose 2024-09-17T22:44:27.246 INFO:teuthology.orchestra.run.smithi115.stderr: -- /var/log/ceph/ceph-client.admin.59413.log 2024-09-17T22:44:27.246 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57103.log 2024-09-17T22:44:27.246 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57523.log.gz 2024-09-17T22:44:27.246 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42492.log 2024-09-17T22:44:27.246 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59413.log.gz 2024-09-17T22:44:27.246 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58993.log 2024-09-17T22:44:27.246 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57103.log.gz 2024-09-17T22:44:27.246 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.42492.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42492.log.gz 2024-09-17T22:44:27.246 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.45395.log 2024-09-17T22:44:27.247 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59077.log 2024-09-17T22:44:27.247 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58993.log.gz 2024-09-17T22:44:27.247 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/objectstore_tool.45395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58699.log 2024-09-17T22:44:27.247 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59077.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27358.log 2024-09-17T22:44:27.247 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59077.log.gz 2024-09-17T22:44:27.247 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54240.log 2024-09-17T22:44:27.247 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58699.log.gz 2024-09-17T22:44:27.247 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph.tmp-client.admin.27358.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27358.log.gz 2024-09-17T22:44:27.248 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57271.log 2024-09-17T22:44:27.248 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57355.log 2024-09-17T22:44:27.248 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54240.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54240.log.gz 2024-09-17T22:44:27.248 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57271.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57271.log.gz 2024-09-17T22:44:27.248 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41288.log 2024-09-17T22:44:27.248 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37894.log 2024-09-17T22:44:27.248 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57355.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57355.log.gz 2024-09-17T22:44:27.248 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41288.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41288.log.gz 2024-09-17T22:44:27.248 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51712.log 2024-09-17T22:44:27.248 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49161.log 2024-09-17T22:44:27.248 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37894.log.gz 2024-09-17T22:44:27.249 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51712.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51712.log.gz 2024-09-17T22:44:27.249 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57061.log 2024-09-17T22:44:27.249 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47530.log 2024-09-17T22:44:27.249 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49161.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49161.log.gz 2024-09-17T22:44:27.249 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57061.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57061.log.gz 2024-09-17T22:44:27.249 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50145.log 2024-09-17T22:44:27.249 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42613.log 2024-09-17T22:44:27.249 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47530.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47530.log.gz 2024-09-17T22:44:27.249 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50145.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50145.log.gz 2024-09-17T22:44:27.249 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48085.log 2024-09-17T22:44:27.249 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50783.log 2024-09-17T22:44:27.250 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.42613.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42613.log.gz 2024-09-17T22:44:27.250 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48085.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48085.log.gz 2024-09-17T22:44:27.250 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52901.log 2024-09-17T22:44:27.250 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43256.log 2024-09-17T22:44:27.250 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50783.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50783.log.gz 2024-09-17T22:44:27.250 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52901.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52901.log.gz 2024-09-17T22:44:27.250 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40775.log 2024-09-17T22:44:27.250 INFO:teuthology.orchestra.run.smithi115.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.45395.log.gz 2024-09-17T22:44:27.250 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50464.log 2024-09-17T22:44:27.250 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43256.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43256.log.gz 2024-09-17T22:44:27.250 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.40775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53512.log 2024-09-17T22:44:27.251 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40775.log.gz 2024-09-17T22:44:27.251 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52840.log 2024-09-17T22:44:27.251 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50464.log.gz 2024-09-17T22:44:27.251 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42390.log 2024-09-17T22:44:27.251 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53512.log.gz 2024-09-17T22:44:27.251 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42049.log 2024-09-17T22:44:27.251 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52840.log.gz 2024-09-17T22:44:27.251 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.42390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39362.log 2024-09-17T22:44:27.251 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42390.log.gz 2024-09-17T22:44:27.252 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.42049.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42049.log.gz 2024-09-17T22:44:27.252 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53851.log 2024-09-17T22:44:27.252 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40295.log 2024-09-17T22:44:27.252 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39362.log.gz 2024-09-17T22:44:27.252 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53851.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53851.log.gz 2024-09-17T22:44:27.252 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40874.log 2024-09-17T22:44:27.252 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56893.log 2024-09-17T22:44:27.252 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.40295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40295.log.gz 2024-09-17T22:44:27.253 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.40874.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40874.log.gz 2024-09-17T22:44:27.253 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T22:44:27.253 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58118.log 2024-09-17T22:44:27.253 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56893.log.gz 2024-09-17T22:44:27.253 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58121.log 2024-09-17T22:44:27.253 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58118.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58118.log.gz 2024-09-17T22:44:27.253 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48343.log 2024-09-17T22:44:27.253 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41000.log 2024-09-17T22:44:27.253 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58121.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58121.log.gz 2024-09-17T22:44:27.254 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48343.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48343.log.gz 2024-09-17T22:44:27.254 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39788.log 2024-09-17T22:44:27.254 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52363.log 2024-09-17T22:44:27.254 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41000.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41000.log.gz 2024-09-17T22:44:27.254 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39788.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39788.log.gzgzip 2024-09-17T22:44:27.254 INFO:teuthology.orchestra.run.smithi115.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37883.log 2024-09-17T22:44:27.254 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55732.log 2024-09-17T22:44:27.254 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52363.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T22:44:27.254 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52363.log.gz 2024-09-17T22:44:27.254 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38512.log 2024-09-17T22:44:27.255 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37883.log.gz 2024-09-17T22:44:27.255 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55732.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55732.log.gz 2024-09-17T22:44:27.255 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33587.log 2024-09-17T22:44:27.255 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39263.log 2024-09-17T22:44:27.255 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38512.log.gz 2024-09-17T22:44:27.255 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.33587.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33587.log.gz 2024-09-17T22:44:27.255 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38263.log 2024-09-17T22:44:27.255 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55774.log 2024-09-17T22:44:27.255 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39263.log.gz 2024-09-17T22:44:27.255 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38263.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38263.log.gz 2024-09-17T22:44:27.255 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41956.log 2024-09-17T22:44:27.255 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55420.log 2024-09-17T22:44:27.256 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55774.log.gz 2024-09-17T22:44:27.256 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41956.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41956.log.gz 2024-09-17T22:44:27.256 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48923.log 2024-09-17T22:44:27.256 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47747.log 2024-09-17T22:44:27.256 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55420.log.gz 2024-09-17T22:44:27.256 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37559.log 2024-09-17T22:44:27.256 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48923.log.gz 2024-09-17T22:44:27.256 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47747.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47747.log.gz 2024-09-17T22:44:27.256 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40916.log 2024-09-17T22:44:27.256 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58783.log 2024-09-17T22:44:27.257 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37559.log.gz 2024-09-17T22:44:27.257 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.40916.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40916.log.gz 2024-09-17T22:44:27.257 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38131.log 2024-09-17T22:44:27.257 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57789.log 2024-09-17T22:44:27.257 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58783.log.gz 2024-09-17T22:44:27.257 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57777.log 2024-09-17T22:44:27.257 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38131.log.gz 2024-09-17T22:44:27.257 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57789.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57789.log.gz 2024-09-17T22:44:27.257 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48385.log 2024-09-17T22:44:27.257 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37162.log 2024-09-17T22:44:27.257 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57777.log.gz 2024-09-17T22:44:27.257 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48385.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48385.log.gz 2024-09-17T22:44:27.258 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59329.log 2024-09-17T22:44:27.258 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39846.log 2024-09-17T22:44:27.258 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37162.log.gz 2024-09-17T22:44:27.258 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59329.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59329.log.gz 2024-09-17T22:44:27.258 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50506.log 2024-09-17T22:44:27.258 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53472.log 2024-09-17T22:44:27.258 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39846.log.gz 2024-09-17T22:44:27.258 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50506.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50506.log.gz 2024-09-17T22:44:27.258 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55037.log 2024-09-17T22:44:27.258 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33496.log 2024-09-17T22:44:27.258 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53472.log.gz 2024-09-17T22:44:27.258 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43120.log 2024-09-17T22:44:27.259 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55037.log.gz 2024-09-17T22:44:27.259 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.33496.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33496.log.gz 2024-09-17T22:44:27.259 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47849.log 2024-09-17T22:44:27.259 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39141.log 2024-09-17T22:44:27.259 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43120.log.gz 2024-09-17T22:44:27.259 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47849.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47849.log.gz 2024-09-17T22:44:27.259 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40233.log 2024-09-17T22:44:27.259 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55400.log 2024-09-17T22:44:27.259 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39141.log.gz 2024-09-17T22:44:27.259 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.40233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40233.log.gz 2024-09-17T22:44:27.259 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52086.log 2024-09-17T22:44:27.259 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37336.log 2024-09-17T22:44:27.260 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55400.log.gz 2024-09-17T22:44:27.260 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52086.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52086.log.gz 2024-09-17T22:44:27.260 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.43572.log 2024-09-17T22:44:27.260 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58489.log 2024-09-17T22:44:27.260 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37336.log.gz 2024-09-17T22:44:27.260 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/objectstore_tool.43572.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-17T22:44:27.260 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58489.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58489.log.gz 2024-09-17T22:44:27.260 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52737.log 2024-09-17T22:44:27.260 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37294.log 2024-09-17T22:44:27.260 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.52737.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52737.log.gz 2024-09-17T22:44:27.260 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51218.log 2024-09-17T22:44:27.260 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41124.log 2024-09-17T22:44:27.261 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37294.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37294.log.gz 2024-09-17T22:44:27.261 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37895.log 2024-09-17T22:44:27.261 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51218.log.gz 2024-09-17T22:44:27.261 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58657.log 2024-09-17T22:44:27.261 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41124.log.gz 2024-09-17T22:44:27.261 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47692.log 2024-09-17T22:44:27.261 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37895.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37895.log.gz 2024-09-17T22:44:27.261 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38879.log 2024-09-17T22:44:27.261 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58657.log.gz 2024-09-17T22:44:27.261 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47692.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47692.log.gz 2024-09-17T22:44:27.261 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54997.log 2024-09-17T22:44:27.261 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37218.log 2024-09-17T22:44:27.262 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38879.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38879.log.gz 2024-09-17T22:44:27.262 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54068.log 2024-09-17T22:44:27.262 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54997.log.gz 2024-09-17T22:44:27.262 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37218.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37218.log.gz 2024-09-17T22:44:27.262 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38796.log 2024-09-17T22:44:27.262 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48801.log 2024-09-17T22:44:27.262 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54068.log.gz 2024-09-17T22:44:27.262 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.0.38796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37893.log 2024-09-17T22:44:27.262 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48801.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48801.log.gz 2024-09-17T22:44:27.262 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57439.log 2024-09-17T22:44:27.262 INFO:teuthology.orchestra.run.smithi115.stderr: 95.7% -- replaced with /var/log/ceph/ceph-client.0.38796.log.gz 2024-09-17T22:44:27.263 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37893.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40693.log 2024-09-17T22:44:27.263 INFO:teuthology.orchestra.run.smithi115.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37893.log.gz 2024-09-17T22:44:27.263 INFO:teuthology.orchestra.run.smithi115.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.43572.log.gz 2024-09-17T22:44:27.263 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54168.log 2024-09-17T22:44:27.263 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57439.log.gz 2024-09-17T22:44:27.263 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.40693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57788.log 2024-09-17T22:44:27.263 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40693.log.gz 2024-09-17T22:44:27.263 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43018.log 2024-09-17T22:44:27.263 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54168.log.gz 2024-09-17T22:44:27.263 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57788.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57788.log.gz 2024-09-17T22:44:27.263 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-17T22:44:27.263 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43018.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43018.log.gz 2024-09-17T22:44:27.263 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37378.log 2024-09-17T22:44:27.264 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50838.log 2024-09-17T22:44:27.264 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37378.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37378.log.gz 2024-09-17T22:44:27.264 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42712.log 2024-09-17T22:44:27.264 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54088.log 2024-09-17T22:44:27.264 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50838.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50838.log.gz 2024-09-17T22:44:27.264 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.42712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38554.log 2024-09-17T22:44:27.264 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42712.log.gz 2024-09-17T22:44:27.264 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54088.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54088.log.gz 2024-09-17T22:44:27.264 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58125.log 2024-09-17T22:44:27.264 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58951.log 2024-09-17T22:44:27.264 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38554.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38554.log.gz 2024-09-17T22:44:27.264 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47220.log 2024-09-17T22:44:27.265 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58125.log.gz 2024-09-17T22:44:27.265 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37772.log 2024-09-17T22:44:27.265 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58951.log.gz 2024-09-17T22:44:27.265 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59245.log 2024-09-17T22:44:27.265 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47220.log.gz 2024-09-17T22:44:27.265 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58447.log 2024-09-17T22:44:27.265 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37772.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37772.log.gz 2024-09-17T22:44:27.265 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40958.log 2024-09-17T22:44:27.265 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59245.log.gz 2024-09-17T22:44:27.265 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58447.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58447.log.gz 2024-09-17T22:44:27.265 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-17T22:44:27.266 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42248.log 2024-09-17T22:44:27.266 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.40958.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40958.log.gz 2024-09-17T22:44:27.266 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37552.log 2024-09-17T22:44:27.266 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.42248.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42248.log.gz 2024-09-17T22:44:27.266 INFO:teuthology.orchestra.run.smithi115.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38156.log 2024-09-17T22:44:27.266 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37552.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37552.log.gz 2024-09-17T22:44:27.266 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41965.log 2024-09-17T22:44:27.266 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38156.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38156.log.gz 2024-09-17T22:44:27.266 INFO:teuthology.orchestra.run.smithi115.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52662.log 2024-09-17T22:44:27.266 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41965.log.gz 2024-09-17T22:44:27.266 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43597.log 2024-09-17T22:44:27.266 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52662.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52662.log.gz 2024-09-17T22:44:27.267 INFO:teuthology.orchestra.run.smithi115.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37896.log 2024-09-17T22:44:27.267 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43597.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43597.log.gz 2024-09-17T22:44:27.267 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56977.log 2024-09-17T22:44:27.267 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37896.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37896.log.gz 2024-09-17T22:44:27.267 INFO:teuthology.orchestra.run.smithi115.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41186.log 2024-09-17T22:44:27.267 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.56977.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52639.log 2024-09-17T22:44:27.267 INFO:teuthology.orchestra.run.smithi115.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56977.log.gz 2024-09-17T22:44:27.267 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57565.log 2024-09-17T22:44:27.267 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41186.log.gz 2024-09-17T22:44:27.267 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53098.log 2024-09-17T22:44:27.267 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52639.log.gz 2024-09-17T22:44:27.267 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37558.log 2024-09-17T22:44:27.267 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57565.log: /var/log/ceph/ceph-client.admin.53098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33194.log 2024-09-17T22:44:27.268 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57565.log.gz 2024-09-17T22:44:27.268 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53098.log.gz 2024-09-17T22:44:27.268 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37558.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37558.log.gz 2024-09-17T22:44:27.268 INFO:teuthology.orchestra.run.smithi115.stderr: -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-17T22:44:27.268 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.33194.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33194.log.gz 2024-09-17T22:44:27.268 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52779.log 2024-09-17T22:44:27.268 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40357.log 2024-09-17T22:44:27.268 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52779.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57481.log 2024-09-17T22:44:27.268 INFO:teuthology.orchestra.run.smithi115.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52779.log.gz 2024-09-17T22:44:27.268 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38428.log 2024-09-17T22:44:27.268 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.40357.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40357.log.gz 2024-09-17T22:44:27.268 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57481.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57481.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55181.log 2024-09-17T22:44:27.269 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-09-17T22:44:27.269 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58573.log 2024-09-17T22:44:27.269 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38428.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38428.log.gz 2024-09-17T22:44:27.269 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55181.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37887.log 2024-09-17T22:44:27.269 INFO:teuthology.orchestra.run.smithi115.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55181.log.gz 2024-09-17T22:44:27.269 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57187.log 2024-09-17T22:44:27.269 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58573.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58573.log.gz 2024-09-17T22:44:27.269 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37887.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37887.log.gz 2024-09-17T22:44:27.269 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37776.log 2024-09-17T22:44:27.269 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54444.log 2024-09-17T22:44:27.269 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57187.log: /var/log/ceph/ceph-client.admin.37776.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37776.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57187.log.gz 2024-09-17T22:44:27.269 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-09-17T22:44:27.269 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48447.log 2024-09-17T22:44:27.270 INFO:teuthology.orchestra.run.smithi115.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-17T22:44:27.270 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51333.log 2024-09-17T22:44:27.270 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54444.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54444.log.gz 2024-09-17T22:44:27.270 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T22:44:27.270 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48447.log: /var/log/ceph/ceph-client.admin.51333.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38638.log 2024-09-17T22:44:27.270 INFO:teuthology.orchestra.run.smithi115.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51333.log.gz 2024-09-17T22:44:27.270 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48447.log.gz 2024-09-17T22:44:27.270 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51435.log 2024-09-17T22:44:27.270 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57145.log 2024-09-17T22:44:27.270 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38638.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38638.log.gz 2024-09-17T22:44:27.270 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51435.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51435.log.gz 2024-09-17T22:44:27.270 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41062.log 2024-09-17T22:44:27.271 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57145.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57145.log.gz 2024-09-17T22:44:27.271 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37548.log 2024-09-17T22:44:27.271 INFO:teuthology.orchestra.run.smithi115.stderr: 92.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T22:44:27.271 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49078.log 2024-09-17T22:44:27.271 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41062.log: /var/log/ceph/ceph-client.admin.37548.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41062.log.gz 2024-09-17T22:44:27.271 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37548.log.gz 2024-09-17T22:44:27.271 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50701.log 2024-09-17T22:44:27.271 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49078.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49078.log.gz 2024-09-17T22:44:27.271 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40356.log 2024-09-17T22:44:27.272 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50701.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50701.log.gz 2024-09-17T22:44:27.272 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51907.log 2024-09-17T22:44:27.272 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.40356.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40356.log.gz 2024-09-17T22:44:27.272 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40090.log 2024-09-17T22:44:27.272 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51907.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51907.log.gz 2024-09-17T22:44:27.272 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53375.log 2024-09-17T22:44:27.272 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.40090.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40090.log.gz 2024-09-17T22:44:27.272 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58531.log 2024-09-17T22:44:27.272 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53375.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53375.log.gz 2024-09-17T22:44:27.272 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41886.log 2024-09-17T22:44:27.272 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58531.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58531.log.gz 2024-09-17T22:44:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54128.log 2024-09-17T22:44:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41886.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41886.log.gz 2024-09-17T22:44:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48524.log 2024-09-17T22:44:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54128.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54128.log.gz 2024-09-17T22:44:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50006.log 2024-09-17T22:44:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48524.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48524.log.gz 2024-09-17T22:44:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47990.log 2024-09-17T22:44:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50006.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50006.log.gz 2024-09-17T22:44:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39303.log 2024-09-17T22:44:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47990.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47990.log.gz 2024-09-17T22:44:27.273 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48746.log 2024-09-17T22:44:27.274 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39303.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39303.log.gz 2024-09-17T22:44:27.274 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59119.log 2024-09-17T22:44:27.274 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48746.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48746.log.gz 2024-09-17T22:44:27.274 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41826.log 2024-09-17T22:44:27.274 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59119.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59119.log.gz 2024-09-17T22:44:27.274 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39992.log 2024-09-17T22:44:27.274 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.41826.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41826.log.gz 2024-09-17T22:44:27.274 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50047.log 2024-09-17T22:44:27.274 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39992.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39992.log.gz 2024-09-17T22:44:27.274 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57397.log 2024-09-17T22:44:27.274 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50047.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50047.log.gz 2024-09-17T22:44:27.274 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38680.log 2024-09-17T22:44:27.275 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57397.log.gz 2024-09-17T22:44:27.275 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58122.log 2024-09-17T22:44:27.275 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38680.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38680.log.gz 2024-09-17T22:44:27.275 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58004.log 2024-09-17T22:44:27.275 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58122.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58122.log.gz 2024-09-17T22:44:27.275 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51610.log 2024-09-17T22:44:27.275 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58004.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58004.log.gz 2024-09-17T22:44:27.275 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52220.log 2024-09-17T22:44:27.275 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.51610.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51610.log.gz 2024-09-17T22:44:27.275 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53427.log 2024-09-17T22:44:27.275 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.52220.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52220.log.gz 2024-09-17T22:44:27.276 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58124.log 2024-09-17T22:44:27.276 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53427.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53427.log.gz 2024-09-17T22:44:27.276 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49452.log 2024-09-17T22:44:27.276 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58124.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58124.log.gz 2024-09-17T22:44:27.276 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54324.log 2024-09-17T22:44:27.276 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49452.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49452.log.gz 2024-09-17T22:44:27.276 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58825.log 2024-09-17T22:44:27.276 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54324.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54324.log.gz 2024-09-17T22:44:27.276 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59161.log 2024-09-17T22:44:27.276 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58825.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58825.log.gz 2024-09-17T22:44:27.276 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57229.log 2024-09-17T22:44:27.276 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59161.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59161.log.gz 2024-09-17T22:44:27.277 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58867.log 2024-09-17T22:44:27.277 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57229.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57229.log.gz 2024-09-17T22:44:27.277 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40824.log 2024-09-17T22:44:27.277 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58867.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58867.log.gz 2024-09-17T22:44:27.277 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39382.log 2024-09-17T22:44:27.277 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.40824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40824.log.gz 2024-09-17T22:44:27.277 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49771.log 2024-09-17T22:44:27.277 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39382.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39382.log.gz 2024-09-17T22:44:27.277 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55816.log 2024-09-17T22:44:27.277 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49771.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49771.log.gz 2024-09-17T22:44:27.278 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57785.log 2024-09-17T22:44:27.278 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55816.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55816.log.gz 2024-09-17T22:44:27.278 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49573.log 2024-09-17T22:44:27.278 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57785.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57785.log.gz 2024-09-17T22:44:27.278 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42429.log 2024-09-17T22:44:27.278 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49573.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49573.log.gz 2024-09-17T22:44:27.278 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47200.log 2024-09-17T22:44:27.278 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.42429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42429.log.gz 2024-09-17T22:44:27.278 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50346.log 2024-09-17T22:44:27.278 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.47200.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47200.log.gz 2024-09-17T22:44:27.278 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45887.log 2024-09-17T22:44:27.279 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50346.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50346.log.gz 2024-09-17T22:44:27.279 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48106.log 2024-09-17T22:44:27.279 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.45887.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45887.log.gz 2024-09-17T22:44:27.279 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58005.log 2024-09-17T22:44:27.279 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.48106.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48106.log.gz 2024-09-17T22:44:27.279 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38880.log 2024-09-17T22:44:27.279 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.58005.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58005.log.gz 2024-09-17T22:44:27.279 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39690.log 2024-09-17T22:44:27.279 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.38880.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38880.log.gz 2024-09-17T22:44:27.279 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37556.log 2024-09-17T22:44:27.279 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.39690.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39690.log.gz 2024-09-17T22:44:27.280 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42916.log 2024-09-17T22:44:27.280 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37556.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37556.log.gz 2024-09-17T22:44:27.280 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42715.log 2024-09-17T22:44:27.280 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.42916.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42916.log.gz 2024-09-17T22:44:27.280 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55120.log 2024-09-17T22:44:27.280 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.42715.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42715.log.gz 2024-09-17T22:44:27.280 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53128.log 2024-09-17T22:44:27.280 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.55120.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55120.log.gz 2024-09-17T22:44:27.280 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.44594.log 2024-09-17T22:44:27.280 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.53128.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53128.log.gz 2024-09-17T22:44:27.280 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49397.log 2024-09-17T22:44:27.281 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/objectstore_tool.44594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59495.log 2024-09-17T22:44:27.281 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.49397.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-17T22:44:27.281 INFO:teuthology.orchestra.run.smithi115.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49397.log.gz 2024-09-17T22:44:27.281 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59495.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59495.log.gz 2024-09-17T22:44:27.281 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55637.log 2024-09-17T22:44:27.281 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37555.log 2024-09-17T22:44:27.281 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.55637.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55637.log.gz 2024-09-17T22:44:27.281 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57313.log 2024-09-17T22:44:27.281 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37555.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37555.log.gz 2024-09-17T22:44:27.281 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37059.log 2024-09-17T22:44:27.281 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.57313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57313.log.gz 2024-09-17T22:44:27.282 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54284.log 2024-09-17T22:44:27.282 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.37059.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37059.log.gz 2024-09-17T22:44:27.282 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42551.log 2024-09-17T22:44:27.282 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.54284.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54284.log.gz 2024-09-17T22:44:27.282 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43177.log 2024-09-17T22:44:27.282 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.42551.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42551.log.gz 2024-09-17T22:44:27.282 INFO:teuthology.orchestra.run.smithi115.stderr: 95.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50103.log 2024-09-17T22:44:27.282 INFO:teuthology.orchestra.run.smithi115.stderr: -- replaced with /var/log/ceph/objectstore_tool.44594.log.gz 2024-09-17T22:44:27.282 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.43177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43177.log.gz 2024-09-17T22:44:27.282 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59287.log 2024-09-17T22:44:27.283 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.50103.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50103.log.gz 2024-09-17T22:44:27.283 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.59287.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59287.log.gz 2024-09-17T22:44:27.283 INFO:teuthology.orchestra.run.smithi115.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-17T22:44:27.283 INFO:teuthology.orchestra.run.smithi129.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-17T22:44:27.466 INFO:teuthology.orchestra.run.smithi115.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-17T22:44:28.061 INFO:teuthology.orchestra.run.smithi129.stderr: 94.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-17T22:44:28.495 INFO:teuthology.orchestra.run.smithi115.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-17T22:44:29.501 INFO:teuthology.orchestra.run.smithi129.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-17T22:44:29.555 INFO:teuthology.orchestra.run.smithi129.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-17T22:44:30.220 INFO:teuthology.orchestra.run.smithi115.stderr: 96.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-17T22:44:30.665 INFO:teuthology.orchestra.run.smithi129.stderr: 96.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-17T22:44:30.667 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-17T22:44:30.667 INFO:teuthology.orchestra.run.smithi129.stderr:real 0m3.539s 2024-09-17T22:44:30.668 INFO:teuthology.orchestra.run.smithi129.stderr:user 0m9.055s 2024-09-17T22:44:30.668 INFO:teuthology.orchestra.run.smithi129.stderr:sys 0m0.394s 2024-09-17T22:44:31.489 INFO:teuthology.orchestra.run.smithi115.stderr: 96.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-17T22:44:32.177 INFO:teuthology.orchestra.run.smithi115.stderr: 96.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-17T22:44:32.178 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-09-17T22:44:32.179 INFO:teuthology.orchestra.run.smithi115.stderr:real 0m5.092s 2024-09-17T22:44:32.179 INFO:teuthology.orchestra.run.smithi115.stderr:user 0m13.872s 2024-09-17T22:44:32.179 INFO:teuthology.orchestra.run.smithi115.stderr:sys 0m0.658s 2024-09-17T22:44:32.179 INFO:tasks.ceph:Archiving logs... 2024-09-17T22:44:32.179 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909540/remote/smithi115/log 2024-09-17T22:44:32.180 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T22:44:33.950 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909540/remote/smithi129/log 2024-09-17T22:44:33.951 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T22:44:35.131 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T22:44:35.140 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T22:44:35.140 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T22:44:35.143 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T22:44:35.231 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T22:44:35.239 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T22:44:35.240 DEBUG:teuthology.orchestra.run.smithi115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T22:44:35.242 DEBUG:teuthology.orchestra.run.smithi129:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T22:44:35.257 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-09-17T22:44:35.259 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-09-17T22:44:35.261 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T22:44:35.261 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-09-17T22:44:35.261 INFO:teuthology.orchestra.run.smithi115.stdout:^* hv01.front.sepia.ceph.com 3 6 377 9 -158us[ -156us] +/- 62ms 2024-09-17T22:44:35.261 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 -441us[ -441us] +/- 62ms 2024-09-17T22:44:35.261 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 +828us[ +828us] +/- 84ms 2024-09-17T22:44:35.261 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T22:44:35.264 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T22:44:35.264 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-09-17T22:44:35.265 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 9 -150us[ -147us] +/- 62ms 2024-09-17T22:44:35.265 INFO:teuthology.orchestra.run.smithi129.stdout:^* hv02.front.sepia.ceph.com 3 6 377 7 -436us[ -433us] +/- 62ms 2024-09-17T22:44:35.265 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 +838us[ +840us] +/- 84ms 2024-09-17T22:44:35.265 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T22:44:35.265 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T22:44:35.274 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T22:44:35.274 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T22:44:35.323 DEBUG:teuthology.orchestra.run.smithi115:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T22:44:35.403 DEBUG:teuthology.orchestra.run.smithi129:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T22:44:35.480 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T22:44:35.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T22:44:35.511 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T22:44:35.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T22:44:35.541 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T22:44:35.553 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T22:44:35.598 INFO:teuthology.task.internal:Duration was 628.746848 seconds 2024-09-17T22:44:35.598 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T22:44:35.648 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T22:44:35.649 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T22:44:35.651 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T22:44:35.689 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T22:44:35.691 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T22:44:36.157 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T22:44:36.158 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi115.front.sepia.ceph.com 2024-09-17T22:44:36.158 DEBUG:teuthology.orchestra.run.smithi115:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T22:44:36.178 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi129.front.sepia.ceph.com 2024-09-17T22:44:36.178 DEBUG:teuthology.orchestra.run.smithi129:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T22:44:36.224 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T22:44:36.225 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T22:44:36.227 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T22:44:36.291 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T22:44:36.291 DEBUG:teuthology.orchestra.run.smithi115:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T22:44:36.294 DEBUG:teuthology.orchestra.run.smithi129:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T22:44:36.443 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T22:44:36.453 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T22:44:36.453 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T22:44:36.478 DEBUG:teuthology.orchestra.run.smithi129:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T22:44:36.503 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T22:44:36.533 DEBUG:teuthology.orchestra.run.smithi115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T22:44:36.536 DEBUG:teuthology.orchestra.run.smithi129:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T22:44:36.556 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = core 2024-09-17T22:44:36.566 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = core 2024-09-17T22:44:36.596 DEBUG:teuthology.orchestra.run.smithi115:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T22:44:36.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T22:44:36.652 DEBUG:teuthology.orchestra.run.smithi129:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T22:44:36.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T22:44:36.668 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T22:44:36.678 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T22:44:36.678 DEBUG:teuthology.misc:Transferring archived files from smithi115:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909540/remote/smithi115 2024-09-17T22:44:36.679 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T22:44:36.732 DEBUG:teuthology.misc:Transferring archived files from smithi129:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909540/remote/smithi129 2024-09-17T22:44:36.732 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T22:44:36.765 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T22:44:36.765 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T22:44:36.767 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T22:44:36.814 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T22:44:36.827 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T22:44:36.827 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T22:44:36.858 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T22:44:36.858 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T22:44:36.861 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T22:44:36.873 INFO:teuthology.orchestra.run.smithi115.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 22:44 /home/ubuntu/cephtest 2024-09-17T22:44:36.876 INFO:teuthology.orchestra.run.smithi129.stdout: 265861 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 22:44 /home/ubuntu/cephtest 2024-09-17T22:44:36.877 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T22:44:36.908 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T22:44:37.009 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache} duration: 628.7468478679657 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-17T22:44:37.010 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T22:44:37.096 INFO:teuthology.run:pass